OLD | NEW |
| (Empty) |
1 # -*- coding: utf-8 -*- | |
2 # Copyright 2012 Google Inc. All Rights Reserved. | |
3 # | |
4 # Licensed under the Apache License, Version 2.0 (the "License"); | |
5 # you may not use this file except in compliance with the License. | |
6 # You may obtain a copy of the License at | |
7 # | |
8 # http://www.apache.org/licenses/LICENSE-2.0 | |
9 # | |
10 # Unless required by applicable law or agreed to in writing, software | |
11 # distributed under the License is distributed on an "AS IS" BASIS, | |
12 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
13 # See the License for the specific language governing permissions and | |
14 # limitations under the License. | |
15 """Additional help about subdirectory handling in gsutil.""" | |
16 | |
17 from __future__ import absolute_import | |
18 | |
19 from gslib.help_provider import HelpProvider | |
20 | |
21 _DETAILED_HELP_TEXT = (""" | |
22 <B>OVERVIEW</B> | |
23 This section provides details about how subdirectories work in gsutil. | |
24 Most users probably don't need to know these details, and can simply use | |
25 the commands (like cp -r) that work with subdirectories. We provide this | |
26 additional documentation to help users understand how gsutil handles | |
27 subdirectories differently than most GUI / web-based tools (e.g., why | |
28 those other tools create "dir_$folder$" objects), and also to explain cost and | |
29 performance implications of the gsutil approach, for those interested in such | |
30 details. | |
31 | |
32 gsutil provides the illusion of a hierarchical file tree atop the "flat" | |
33 name space supported by the Google Cloud Storage service. To the service, | |
34 the object gs://your-bucket/abc/def/ghi.txt is just an object that happens to | |
35 have "/" characters in its name. There are no "abc" or "abc/def" directories; | |
36 just a single object with the given name. This | |
37 `diagram <https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.
png>` | |
38 illustrates how gsutil provides a hierarchical view of objects in a bucket. | |
39 | |
40 gsutil achieves the hierarchical file tree illusion by applying a variety of | |
41 rules, to try to make naming work the way users would expect. For example, in | |
42 order to determine whether to treat a destination URI as an object name or the | |
43 root of a directory under which objects should be copied gsutil uses these | |
44 rules: | |
45 | |
46 1. If the destination object ends with a "/" gsutil treats it as a directory. | |
47 For example, if you run the command: | |
48 | |
49 gsutil cp your-file gs://your-bucket/abc/ | |
50 | |
51 gsutil will create the object gs://your-bucket/abc/file. | |
52 | |
53 2. If the destination object is XYZ and an object exists called XYZ_$folder$ | |
54 gsutil treats XYZ as a directory. For example, if you run the command: | |
55 | |
56 gsutil cp your-file gs://your-bucket/abc | |
57 | |
58 and there exists an object called abc_$folder$, gsutil will create the | |
59 object gs://your-bucket/abc/file. | |
60 | |
61 3. If you attempt to copy multiple source files to a destination URI, gsutil | |
62 treats the destination URI as a directory. For example, if you run | |
63 the command: | |
64 | |
65 gsutil cp -r your-dir gs://your-bucket/abc | |
66 | |
67 gsutil will create objects like gs://your-bucket/abc/your-dir/file1, etc. | |
68 (assuming file1 is a file under the source directory). | |
69 | |
70 4. If none of the above rules applies, gsutil performs a bucket listing to | |
71 determine if the target of the operation is a prefix match to the | |
72 specified string. For example, if you run the command: | |
73 | |
74 gsutil cp your-file gs://your-bucket/abc | |
75 | |
76 gsutil will make a bucket listing request for the named bucket, using | |
77 delimiter="/" and prefix="abc". It will then examine the bucket listing | |
78 results and determine whether there are objects in the bucket whose path | |
79 starts with gs://your-bucket/abc/, to determine whether to treat the target | |
80 as an object name or a directory name. In turn this impacts the name of the | |
81 object you create: If the above check indicates there is an "abc" directory | |
82 you will end up with the object gs://your-bucket/abc/file; otherwise you | |
83 will end up with the object gs://your-bucket/abc. (See | |
84 "HOW NAMES ARE CONSTRUCTED" under "gsutil help cp" for more details.) | |
85 | |
86 This rule-based approach stands in contrast to the way many tools work, which | |
87 create objects to mark the existence of folders (such as "dir_$folder$"). | |
88 gsutil understands several conventions used by such tools but does not | |
89 require such marker objects to implement naming behavior consistent with | |
90 UNIX commands. | |
91 | |
92 A downside of the gsutil approach is it requires an extra bucket listing | |
93 before performing the needed cp or mv command. However those listings are | |
94 relatively inexpensive, because they use delimiter and prefix parameters to | |
95 limit result data. Moreover, gsutil makes only one bucket listing request | |
96 per cp/mv command, and thus amortizes the bucket listing cost across all | |
97 transferred objects (e.g., when performing a recursive copy of a directory | |
98 to the cloud). | |
99 """) | |
100 | |
101 | |
102 class CommandOptions(HelpProvider): | |
103 """Additional help about subdirectory handling in gsutil.""" | |
104 | |
105 # Help specification. See help_provider.py for documentation. | |
106 help_spec = HelpProvider.HelpSpec( | |
107 help_name='subdirs', | |
108 help_name_aliases=[ | |
109 'dirs', 'directory', 'directories', 'folder', 'folders', 'hierarchy', | |
110 'subdir', 'subdirectory', 'subdirectories'], | |
111 help_type='additional_help', | |
112 help_one_line_summary='How Subdirectories Work', | |
113 help_text=_DETAILED_HELP_TEXT, | |
114 subcommand_help_text={}, | |
115 ) | |
OLD | NEW |