| OLD | NEW |
| (Empty) |
| 1 # Copyright 2012 Google Inc. All Rights Reserved. | |
| 2 # | |
| 3 # Licensed under the Apache License, Version 2.0 (the "License"); | |
| 4 # you may not use this file except in compliance with the License. | |
| 5 # You may obtain a copy of the License at | |
| 6 # | |
| 7 # http://www.apache.org/licenses/LICENSE-2.0 | |
| 8 # | |
| 9 # Unless required by applicable law or agreed to in writing, software | |
| 10 # distributed under the License is distributed on an "AS IS" BASIS, | |
| 11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 12 # See the License for the specific language governing permissions and | |
| 13 # limitations under the License. | |
| 14 | |
| 15 from gslib.help_provider import HELP_NAME | |
| 16 from gslib.help_provider import HELP_NAME_ALIASES | |
| 17 from gslib.help_provider import HELP_ONE_LINE_SUMMARY | |
| 18 from gslib.help_provider import HelpProvider | |
| 19 from gslib.help_provider import HELP_TEXT | |
| 20 from gslib.help_provider import HelpType | |
| 21 from gslib.help_provider import HELP_TYPE | |
| 22 | |
| 23 _detailed_help_text = (""" | |
| 24 <B>OVERVIEW</B> | |
| 25 This section provides details about how subdirectories work in gsutil. | |
| 26 Most users probably don't need to know these details, and can simply use | |
| 27 the commands (like cp -R) that work with subdirectories. We provide this | |
| 28 additional documentation to help users understand how gsutil handles | |
| 29 subdirectories differently than most GUI / web-based tools (e.g., why | |
| 30 those other tools create "dir_$folder$" objects), and also to explain cost and | |
| 31 performance implications of the gsutil approach, for those interested in such | |
| 32 details. | |
| 33 | |
| 34 gsutil provides the illusion of a hierarchical file tree atop the "flat" | |
| 35 name space supported by the Google Cloud Storage service. To the service, | |
| 36 the object gs://bucket/abc/def/ghi.txt is just an object that happens to have | |
| 37 "/" characters in its name. There are no "abc" or "abc/def" directories; | |
| 38 just a single object with the given name. | |
| 39 | |
| 40 gsutil achieves the hierarchical file tree illusion by applying a variety of | |
| 41 rules, to try to make naming work the way users would expect. For example, in | |
| 42 order to determine whether to treat a destination URI as an object name or the | |
| 43 root of a directory under which objects should be copied gsutil uses these | |
| 44 rules: | |
| 45 | |
| 46 1. If the destination object ends with a "/" gsutil treats it as a directory. | |
| 47 For example, if you run the command: | |
| 48 | |
| 49 gsutil cp file gs://bucket/abc/ | |
| 50 | |
| 51 gsutil will create the object gs://bucket/abc/file. | |
| 52 | |
| 53 2. If you attempt to copy multiple source files to a destination URI, gsutil | |
| 54 treats the destination URI as a directory. For example, if you run | |
| 55 the command: | |
| 56 | |
| 57 gsutil cp -R dir gs://bucket/abc | |
| 58 | |
| 59 gsutil will create objects like gs://bucket/abc/dir/file1, etc. (assuming | |
| 60 file1 is a file under the source dir). | |
| 61 | |
| 62 3. If neither of the above rules applies, gsutil performs a bucket listing to | |
| 63 determine if the target of the operation is a prefix match to the | |
| 64 specified string. For example, if you run the command: | |
| 65 | |
| 66 gsutil cp file gs://bucket/abc | |
| 67 | |
| 68 gsutil will make a bucket listing request for the named bucket, using | |
| 69 delimiter="/" and prefix="abc". It will then examine the bucket listing | |
| 70 results and determine whether there are objects in the bucket whose path | |
| 71 starts with gs://bucket/abc/, to determine whether to treat the target as | |
| 72 an object name or a directory name. In turn this impacts the name of the | |
| 73 object you create: If the above check indicates there is an "abc" directory | |
| 74 you will end up with the object gs://bucket/abc/file; otherwise you will | |
| 75 end up with the object gs://bucket/abc. (See "HOW NAMES ARE CONSTRUCTED" | |
| 76 under "gsutil help cp" for more details.) | |
| 77 | |
| 78 This rule-based approach stands in contrast to the way many tools work, which | |
| 79 create objects to mark the existence of folders (such as "dir_$folder$"). | |
| 80 gsutil understands several conventions used by such tools but does not | |
| 81 require such marker objects to implement naming behavior consistent with | |
| 82 UNIX commands. | |
| 83 | |
| 84 A downside of the gsutil approach is it requires an extra bucket listing | |
| 85 before performing the needed cp or mv command. However those listings are | |
| 86 relatively inexpensive, because they use delimiter and prefix parameters to | |
| 87 limit result data. Moreover, gsutil makes only one bucket listing request | |
| 88 per cp/mv command, and thus amortizes the bucket listing cost across all | |
| 89 transferred objects (e.g., when performing a recursive copy of a directory | |
| 90 to the cloud). | |
| 91 """) | |
| 92 | |
| 93 | |
| 94 class CommandOptions(HelpProvider): | |
| 95 """Additional help about subdirectory handling in gsutil.""" | |
| 96 | |
| 97 help_spec = { | |
| 98 # Name of command or auxiliary help info for which this help applies. | |
| 99 HELP_NAME : 'subdirs', | |
| 100 # List of help name aliases. | |
| 101 HELP_NAME_ALIASES : ['dirs', 'directory', 'directories', 'folder', | |
| 102 'folders', 'hierarchy', 'subdir', 'subdirectory', | |
| 103 'subdirectories'], | |
| 104 # Type of help: | |
| 105 HELP_TYPE : HelpType.ADDITIONAL_HELP, | |
| 106 # One line summary of this help. | |
| 107 HELP_ONE_LINE_SUMMARY : 'How subdirectories work in gsutil', | |
| 108 # The full help text. | |
| 109 HELP_TEXT : _detailed_help_text, | |
| 110 } | |
| OLD | NEW |