OLD | NEW |
| (Empty) |
1 # Copyright 2012 Google Inc. All Rights Reserved. | |
2 # | |
3 # Licensed under the Apache License, Version 2.0 (the "License"); | |
4 # you may not use this file except in compliance with the License. | |
5 # You may obtain a copy of the License at | |
6 # | |
7 # http://www.apache.org/licenses/LICENSE-2.0 | |
8 # | |
9 # Unless required by applicable law or agreed to in writing, software | |
10 # distributed under the License is distributed on an "AS IS" BASIS, | |
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
12 # See the License for the specific language governing permissions and | |
13 # limitations under the License. | |
14 | |
15 from gslib.help_provider import HELP_NAME | |
16 from gslib.help_provider import HELP_NAME_ALIASES | |
17 from gslib.help_provider import HELP_ONE_LINE_SUMMARY | |
18 from gslib.help_provider import HelpProvider | |
19 from gslib.help_provider import HELP_TEXT | |
20 from gslib.help_provider import HelpType | |
21 from gslib.help_provider import HELP_TYPE | |
22 | |
23 _detailed_help_text = (""" | |
24 <B>OVERVIEW</B> | |
25 This section provides details about how subdirectories work in gsutil. | |
26 Most users probably don't need to know these details, and can simply use | |
27 the commands (like cp -R) that work with subdirectories. We provide this | |
28 additional documentation to help users understand how gsutil handles | |
29 subdirectories differently than most GUI / web-based tools (e.g., why | |
30 those other tools create "dir_$folder$" objects), and also to explain cost and | |
31 performance implications of the gsutil approach, for those interested in such | |
32 details. | |
33 | |
34 gsutil provides the illusion of a hierarchical file tree atop the "flat" | |
35 name space supported by the Google Cloud Storage service. To the service, | |
36 the object gs://bucket/abc/def/ghi.txt is just an object that happens to have | |
37 "/" characters in its name. There are no "abc" or "abc/def" directories; | |
38 just a single object with the given name. | |
39 | |
40 gsutil achieves the hierarchical file tree illusion by applying a variety of | |
41 rules, to try to make naming work the way users would expect. For example, in | |
42 order to determine whether to treat a destination URI as an object name or the | |
43 root of a directory under which objects should be copied gsutil uses these | |
44 rules: | |
45 | |
46 1. If the destination object ends with a "/" gsutil treats it as a directory. | |
47 For example, if you run the command: | |
48 | |
49 gsutil cp file gs://bucket/abc/ | |
50 | |
51 gsutil will create the object gs://bucket/abc/file. | |
52 | |
53 2. If you attempt to copy multiple source files to a destination URI, gsutil | |
54 treats the destination URI as a directory. For example, if you run | |
55 the command: | |
56 | |
57 gsutil cp -R dir gs://bucket/abc | |
58 | |
59 gsutil will create objects like gs://bucket/abc/dir/file1, etc. (assuming | |
60 file1 is a file under the source dir). | |
61 | |
62 3. If neither of the above rules applies, gsutil performs a bucket listing to | |
63 determine if the target of the operation is a prefix match to the | |
64 specified string. For example, if you run the command: | |
65 | |
66 gsutil cp file gs://bucket/abc | |
67 | |
68 gsutil will make a bucket listing request for the named bucket, using | |
69 delimiter="/" and prefix="abc". It will then examine the bucket listing | |
70 results and determine whether there are objects in the bucket whose path | |
71 starts with gs://bucket/abc/, to determine whether to treat the target as | |
72 an object name or a directory name. In turn this impacts the name of the | |
73 object you create: If the above check indicates there is an "abc" directory | |
74 you will end up with the object gs://bucket/abc/file; otherwise you will | |
75 end up with the object gs://bucket/abc. (See "HOW NAMES ARE CONSTRUCTED" | |
76 under "gsutil help cp" for more details.) | |
77 | |
78 This rule-based approach stands in contrast to the way many tools work, which | |
79 create objects to mark the existence of folders (such as "dir_$folder$"). | |
80 gsutil understands several conventions used by such tools but does not | |
81 require such marker objects to implement naming behavior consistent with | |
82 UNIX commands. | |
83 | |
84 A downside of the gsutil approach is it requires an extra bucket listing | |
85 before performing the needed cp or mv command. However those listings are | |
86 relatively inexpensive, because they use delimiter and prefix parameters to | |
87 limit result data. Moreover, gsutil makes only one bucket listing request | |
88 per cp/mv command, and thus amortizes the bucket listing cost across all | |
89 transferred objects (e.g., when performing a recursive copy of a directory | |
90 to the cloud). | |
91 """) | |
92 | |
93 | |
94 class CommandOptions(HelpProvider): | |
95 """Additional help about subdirectory handling in gsutil.""" | |
96 | |
97 help_spec = { | |
98 # Name of command or auxiliary help info for which this help applies. | |
99 HELP_NAME : 'subdirs', | |
100 # List of help name aliases. | |
101 HELP_NAME_ALIASES : ['dirs', 'directory', 'directories', 'folder', | |
102 'folders', 'hierarchy', 'subdir', 'subdirectory', | |
103 'subdirectories'], | |
104 # Type of help: | |
105 HELP_TYPE : HelpType.ADDITIONAL_HELP, | |
106 # One line summary of this help. | |
107 HELP_ONE_LINE_SUMMARY : 'How subdirectories work in gsutil', | |
108 # The full help text. | |
109 HELP_TEXT : _detailed_help_text, | |
110 } | |
OLD | NEW |