Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(122)

Side by Side Diff: third_party/gsutil/gslib/addlhelp/subdirs.py

Issue 2280023003: depot_tools: Remove third_party/gsutil (Closed)
Patch Set: Created 4 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
(Empty)
1 # Copyright 2012 Google Inc. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 # http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14
15 from gslib.help_provider import HELP_NAME
16 from gslib.help_provider import HELP_NAME_ALIASES
17 from gslib.help_provider import HELP_ONE_LINE_SUMMARY
18 from gslib.help_provider import HelpProvider
19 from gslib.help_provider import HELP_TEXT
20 from gslib.help_provider import HelpType
21 from gslib.help_provider import HELP_TYPE
22
23 _detailed_help_text = ("""
24 <B>OVERVIEW</B>
25 This section provides details about how subdirectories work in gsutil.
26 Most users probably don't need to know these details, and can simply use
27 the commands (like cp -R) that work with subdirectories. We provide this
28 additional documentation to help users understand how gsutil handles
29 subdirectories differently than most GUI / web-based tools (e.g., why
30 those other tools create "dir_$folder$" objects), and also to explain cost and
31 performance implications of the gsutil approach, for those interested in such
32 details.
33
34 gsutil provides the illusion of a hierarchical file tree atop the "flat"
35 name space supported by the Google Cloud Storage service. To the service,
36 the object gs://bucket/abc/def/ghi.txt is just an object that happens to have
37 "/" characters in its name. There are no "abc" or "abc/def" directories;
38 just a single object with the given name.
39
40 gsutil achieves the hierarchical file tree illusion by applying a variety of
41 rules, to try to make naming work the way users would expect. For example, in
42 order to determine whether to treat a destination URI as an object name or the
43 root of a directory under which objects should be copied gsutil uses these
44 rules:
45
46 1. If the destination object ends with a "/" gsutil treats it as a directory.
47 For example, if you run the command:
48
49 gsutil cp file gs://bucket/abc/
50
51 gsutil will create the object gs://bucket/abc/file.
52
53 2. If you attempt to copy multiple source files to a destination URI, gsutil
54 treats the destination URI as a directory. For example, if you run
55 the command:
56
57 gsutil cp -R dir gs://bucket/abc
58
59 gsutil will create objects like gs://bucket/abc/dir/file1, etc. (assuming
60 file1 is a file under the source dir).
61
62 3. If neither of the above rules applies, gsutil performs a bucket listing to
63 determine if the target of the operation is a prefix match to the
64 specified string. For example, if you run the command:
65
66 gsutil cp file gs://bucket/abc
67
68 gsutil will make a bucket listing request for the named bucket, using
69 delimiter="/" and prefix="abc". It will then examine the bucket listing
70 results and determine whether there are objects in the bucket whose path
71 starts with gs://bucket/abc/, to determine whether to treat the target as
72 an object name or a directory name. In turn this impacts the name of the
73 object you create: If the above check indicates there is an "abc" directory
74 you will end up with the object gs://bucket/abc/file; otherwise you will
75 end up with the object gs://bucket/abc. (See "HOW NAMES ARE CONSTRUCTED"
76 under "gsutil help cp" for more details.)
77
78 This rule-based approach stands in contrast to the way many tools work, which
79 create objects to mark the existence of folders (such as "dir_$folder$").
80 gsutil understands several conventions used by such tools but does not
81 require such marker objects to implement naming behavior consistent with
82 UNIX commands.
83
84 A downside of the gsutil approach is it requires an extra bucket listing
85 before performing the needed cp or mv command. However those listings are
86 relatively inexpensive, because they use delimiter and prefix parameters to
87 limit result data. Moreover, gsutil makes only one bucket listing request
88 per cp/mv command, and thus amortizes the bucket listing cost across all
89 transferred objects (e.g., when performing a recursive copy of a directory
90 to the cloud).
91 """)
92
93
94 class CommandOptions(HelpProvider):
95 """Additional help about subdirectory handling in gsutil."""
96
97 help_spec = {
98 # Name of command or auxiliary help info for which this help applies.
99 HELP_NAME : 'subdirs',
100 # List of help name aliases.
101 HELP_NAME_ALIASES : ['dirs', 'directory', 'directories', 'folder',
102 'folders', 'hierarchy', 'subdir', 'subdirectory',
103 'subdirectories'],
104 # Type of help:
105 HELP_TYPE : HelpType.ADDITIONAL_HELP,
106 # One line summary of this help.
107 HELP_ONE_LINE_SUMMARY : 'How subdirectories work in gsutil',
108 # The full help text.
109 HELP_TEXT : _detailed_help_text,
110 }
OLDNEW
« no previous file with comments | « third_party/gsutil/gslib/addlhelp/projects.py ('k') | third_party/gsutil/gslib/addlhelp/support.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698