| OLD | NEW |
| (Empty) |
| 1 # -*- coding: utf-8 -*- | |
| 2 # Copyright 2012 Google Inc. All Rights Reserved. | |
| 3 # | |
| 4 # Licensed under the Apache License, Version 2.0 (the "License"); | |
| 5 # you may not use this file except in compliance with the License. | |
| 6 # You may obtain a copy of the License at | |
| 7 # | |
| 8 # http://www.apache.org/licenses/LICENSE-2.0 | |
| 9 # | |
| 10 # Unless required by applicable law or agreed to in writing, software | |
| 11 # distributed under the License is distributed on an "AS IS" BASIS, | |
| 12 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 13 # See the License for the specific language governing permissions and | |
| 14 # limitations under the License. | |
| 15 """Additional help about subdirectory handling in gsutil.""" | |
| 16 | |
| 17 from __future__ import absolute_import | |
| 18 | |
| 19 from gslib.help_provider import HelpProvider | |
| 20 | |
| 21 _DETAILED_HELP_TEXT = (""" | |
| 22 <B>OVERVIEW</B> | |
| 23 This section provides details about how subdirectories work in gsutil. | |
| 24 Most users probably don't need to know these details, and can simply use | |
| 25 the commands (like cp -r) that work with subdirectories. We provide this | |
| 26 additional documentation to help users understand how gsutil handles | |
| 27 subdirectories differently than most GUI / web-based tools (e.g., why | |
| 28 those other tools create "dir_$folder$" objects), and also to explain cost and | |
| 29 performance implications of the gsutil approach, for those interested in such | |
| 30 details. | |
| 31 | |
| 32 gsutil provides the illusion of a hierarchical file tree atop the "flat" | |
| 33 name space supported by the Google Cloud Storage service. To the service, | |
| 34 the object gs://your-bucket/abc/def/ghi.txt is just an object that happens to | |
| 35 have "/" characters in its name. There are no "abc" or "abc/def" directories; | |
| 36 just a single object with the given name. This | |
| 37 `diagram <https://cloud.google.com/storage/images/gsutil-subdirectories-thumb.
png>` | |
| 38 illustrates how gsutil provides a hierarchical view of objects in a bucket. | |
| 39 | |
| 40 gsutil achieves the hierarchical file tree illusion by applying a variety of | |
| 41 rules, to try to make naming work the way users would expect. For example, in | |
| 42 order to determine whether to treat a destination URI as an object name or the | |
| 43 root of a directory under which objects should be copied gsutil uses these | |
| 44 rules: | |
| 45 | |
| 46 1. If the destination object ends with a "/" gsutil treats it as a directory. | |
| 47 For example, if you run the command: | |
| 48 | |
| 49 gsutil cp your-file gs://your-bucket/abc/ | |
| 50 | |
| 51 gsutil will create the object gs://your-bucket/abc/file. | |
| 52 | |
| 53 2. If the destination object is XYZ and an object exists called XYZ_$folder$ | |
| 54 gsutil treats XYZ as a directory. For example, if you run the command: | |
| 55 | |
| 56 gsutil cp your-file gs://your-bucket/abc | |
| 57 | |
| 58 and there exists an object called abc_$folder$, gsutil will create the | |
| 59 object gs://your-bucket/abc/file. | |
| 60 | |
| 61 3. If you attempt to copy multiple source files to a destination URI, gsutil | |
| 62 treats the destination URI as a directory. For example, if you run | |
| 63 the command: | |
| 64 | |
| 65 gsutil cp -r your-dir gs://your-bucket/abc | |
| 66 | |
| 67 gsutil will create objects like gs://your-bucket/abc/your-dir/file1, etc. | |
| 68 (assuming file1 is a file under the source directory). | |
| 69 | |
| 70 4. If none of the above rules applies, gsutil performs a bucket listing to | |
| 71 determine if the target of the operation is a prefix match to the | |
| 72 specified string. For example, if you run the command: | |
| 73 | |
| 74 gsutil cp your-file gs://your-bucket/abc | |
| 75 | |
| 76 gsutil will make a bucket listing request for the named bucket, using | |
| 77 delimiter="/" and prefix="abc". It will then examine the bucket listing | |
| 78 results and determine whether there are objects in the bucket whose path | |
| 79 starts with gs://your-bucket/abc/, to determine whether to treat the target | |
| 80 as an object name or a directory name. In turn this impacts the name of the | |
| 81 object you create: If the above check indicates there is an "abc" directory | |
| 82 you will end up with the object gs://your-bucket/abc/file; otherwise you | |
| 83 will end up with the object gs://your-bucket/abc. (See | |
| 84 "HOW NAMES ARE CONSTRUCTED" under "gsutil help cp" for more details.) | |
| 85 | |
| 86 This rule-based approach stands in contrast to the way many tools work, which | |
| 87 create objects to mark the existence of folders (such as "dir_$folder$"). | |
| 88 gsutil understands several conventions used by such tools but does not | |
| 89 require such marker objects to implement naming behavior consistent with | |
| 90 UNIX commands. | |
| 91 | |
| 92 A downside of the gsutil approach is it requires an extra bucket listing | |
| 93 before performing the needed cp or mv command. However those listings are | |
| 94 relatively inexpensive, because they use delimiter and prefix parameters to | |
| 95 limit result data. Moreover, gsutil makes only one bucket listing request | |
| 96 per cp/mv command, and thus amortizes the bucket listing cost across all | |
| 97 transferred objects (e.g., when performing a recursive copy of a directory | |
| 98 to the cloud). | |
| 99 """) | |
| 100 | |
| 101 | |
| 102 class CommandOptions(HelpProvider): | |
| 103 """Additional help about subdirectory handling in gsutil.""" | |
| 104 | |
| 105 # Help specification. See help_provider.py for documentation. | |
| 106 help_spec = HelpProvider.HelpSpec( | |
| 107 help_name='subdirs', | |
| 108 help_name_aliases=[ | |
| 109 'dirs', 'directory', 'directories', 'folder', 'folders', 'hierarchy', | |
| 110 'subdir', 'subdirectory', 'subdirectories'], | |
| 111 help_type='additional_help', | |
| 112 help_one_line_summary='How Subdirectories Work', | |
| 113 help_text=_DETAILED_HELP_TEXT, | |
| 114 subcommand_help_text={}, | |
| 115 ) | |
| OLD | NEW |