| OLD | NEW |
| (Empty) |
| 1 # Copyright 2012 Google Inc. All Rights Reserved. | |
| 2 # | |
| 3 # Licensed under the Apache License, Version 2.0 (the "License"); | |
| 4 # you may not use this file except in compliance with the License. | |
| 5 # You may obtain a copy of the License at | |
| 6 # | |
| 7 # http://www.apache.org/licenses/LICENSE-2.0 | |
| 8 # | |
| 9 # Unless required by applicable law or agreed to in writing, software | |
| 10 # distributed under the License is distributed on an "AS IS" BASIS, | |
| 11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 12 # See the License for the specific language governing permissions and | |
| 13 # limitations under the License. | |
| 14 | |
| 15 from gslib.help_provider import HELP_NAME | |
| 16 from gslib.help_provider import HELP_NAME_ALIASES | |
| 17 from gslib.help_provider import HELP_ONE_LINE_SUMMARY | |
| 18 from gslib.help_provider import HelpProvider | |
| 19 from gslib.help_provider import HELP_TEXT | |
| 20 from gslib.help_provider import HelpType | |
| 21 from gslib.help_provider import HELP_TYPE | |
| 22 | |
| 23 _detailed_help_text = (""" | |
| 24 <B>OVERVIEW OF METADATA</B> | |
| 25 Objects can have associated metadata, which control aspects of how | |
| 26 GET requests are handled, including Content-Type, Cache-Control, | |
| 27 Content-Disposition, and Content-Encoding (discussed in more detail in | |
| 28 the subsections below). In addition, you can set custom metadata that | |
| 29 can be used by applications (e.g., tagging that particular objects possess | |
| 30 some property). | |
| 31 | |
| 32 There are two ways to set metadata on objects: | |
| 33 | |
| 34 - at upload time you can specify one or more headers to associate with | |
| 35 objects, using the gsutil -h option. For example, the following command | |
| 36 would cause gsutil to set the Content-Type and Cache-Control for each | |
| 37 of the files being uploaded: | |
| 38 | |
| 39 gsutil -h "Content-Type:text/html" -h "Cache-Control:public, max-age=360
0" cp -r images gs://bucket/images | |
| 40 | |
| 41 Note that -h is an option on the gsutil command, not the cp sub-command. | |
| 42 | |
| 43 - You can set or remove metadata fields from already uploaded objects using | |
| 44 the gsutil setmeta command. See "gsutil help setmeta". | |
| 45 | |
| 46 More details about specific pieces of metadata are discussed below. | |
| 47 | |
| 48 | |
| 49 <B>CONTENT TYPE</B> | |
| 50 The most commonly set metadata is Content-Type (also known as MIME type), | |
| 51 which allows browsers to render the object properly. | |
| 52 gsutil sets the Content-Type | |
| 53 automatically at upload time, based on each filename extension. For | |
| 54 example, uploading files with names ending in .txt will set Content-Type | |
| 55 to text/plain. If you're running gsutil on Linux or MacOS and would prefer | |
| 56 to have content type set based on naming plus content examination, see the | |
| 57 use_magicfile configuration variable in the gsutil/boto configuration file | |
| 58 (See also "gsutil help config"). In general, using use_magicfile is more | |
| 59 robust and configurable, but is not available on Windows. | |
| 60 | |
| 61 If you specify a -h header when uploading content (like the example gsutil | |
| 62 command given in the previous section), it overrides the Content-Type that | |
| 63 would have been set based on filename extension or content. This can be | |
| 64 useful if the Content-Type detection algorithm doesn't work as desired | |
| 65 for some of your files. | |
| 66 | |
| 67 You can also completely suppress content type detection in gsutil, by | |
| 68 specifying an empty string on the Content-Type header: | |
| 69 | |
| 70 gsutil -h 'Content-Type:' cp -r images gs://bucket/images | |
| 71 | |
| 72 In this case, the Google Cloud Storage service will attempt to detect | |
| 73 the content type. In general this approach will work better than using | |
| 74 filename extension-based content detection in gsutil, because the list of | |
| 75 filename extensions is kept more current in the server-side content detection | |
| 76 system than in the Python library upon which gsutil content type detection | |
| 77 depends. (For example, at the time of writing this, the filename extension | |
| 78 ".webp" was recognized by the server-side content detection system, but | |
| 79 not by gsutil.) | |
| 80 | |
| 81 | |
| 82 <B>CACHE-CONTROL</B> | |
| 83 Another commonly set piece of metadata is Cache-Control, which allows | |
| 84 you to control whether and for how long browser and Internet caches are | |
| 85 allowed to cache your objects. Cache-Control only applies to objects with | |
| 86 a public-read ACL. Non-public data are not cacheable. | |
| 87 | |
| 88 Here's an example of uploading an object set to allow caching: | |
| 89 | |
| 90 gsutil -h "Cache-Control:public,max-age=3600" cp -a public-read -r html gs:/
/bucket/html | |
| 91 | |
| 92 This command would upload all files in the html directory (and subdirectories) | |
| 93 and make them publicly readable and cacheable, with cache expiration of | |
| 94 one hour. | |
| 95 | |
| 96 Note that if you allow caching, at download time you may see older versions | |
| 97 of objects after uploading a newer replacement object. Note also that because | |
| 98 objects can be cached at various places on the Internet there is no way to | |
| 99 force a cached object to expire globally (unlike the way you can force your | |
| 100 browser to refresh its cache). | |
| 101 | |
| 102 | |
| 103 <B>CONTENT-ENCODING</B> | |
| 104 You could specify Content-Encoding to indicate that an object is compressed, | |
| 105 using a command like: | |
| 106 | |
| 107 gsutil -h "Content-Encoding:gzip" cp *.gz gs://bucket/compressed | |
| 108 | |
| 109 Note that Google Cloud Storage does not compress or decompress objects. If | |
| 110 you use this header to specify a compression type or compression algorithm | |
| 111 (for example, deflate), Google Cloud Storage preserves the header but does | |
| 112 not compress or decompress the object. Instead, you need to ensure that | |
| 113 the files have been compressed using the specified Content-Encoding before | |
| 114 using gsutil to upload them. | |
| 115 | |
| 116 For compressible content, using Content-Encoding:gzip saves network and | |
| 117 storage costs, and improves content serving performance (since most browsers | |
| 118 are able to decompress objects served this way). | |
| 119 | |
| 120 Note also that gsutil provides an easy way to cause content to be compressed | |
| 121 and stored with Content-Encoding:gzip: see the -z option in "gsutil help cp". | |
| 122 | |
| 123 | |
| 124 <B>CONTENT-DISPOSITION</B> | |
| 125 You can set Content-Disposition on your objects, to specify presentation | |
| 126 information about the data being transmitted. Here's an example: | |
| 127 | |
| 128 gsutil -h 'Content-Disposition:attachment; filename=filename.ext' \\ | |
| 129 cp -r attachments gs://bucket/attachments | |
| 130 | |
| 131 Setting the Content-Disposition allows you to control presentation style | |
| 132 of the content, for example determining whether an attachment should be | |
| 133 automatically displayed vs should require some form of action from the user to | |
| 134 open it. See http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.5.1 | |
| 135 for more details about the meaning of Content-Disposition. | |
| 136 | |
| 137 | |
| 138 <B>CUSTOM METADATA</B> | |
| 139 You can add your own custom metadata (e.g,. for use by your application) | |
| 140 to an object by setting a header that starts with "x-goog-meta", for example: | |
| 141 | |
| 142 gsutil -h x-goog-meta-reviewer:jane cp mycode.java gs://bucket/reviews | |
| 143 | |
| 144 You can add multiple differently named custom metadata fields to each object. | |
| 145 | |
| 146 | |
| 147 <B>SETTABLE FIELDS; FIELD VALUES</B> | |
| 148 You can't set some metadata fields, such as ETag and Content-Length. The | |
| 149 fields you can set are: | |
| 150 - Cache-Control | |
| 151 - Content-Disposition | |
| 152 - Content-Encoding | |
| 153 - Content-Language | |
| 154 - Content-MD5 | |
| 155 - Content-Type | |
| 156 - Any field starting with X-GOOG-META- (i.e., custom metadata). | |
| 157 | |
| 158 Header names are case-insensitive. | |
| 159 | |
| 160 X-GOOG-META- fields can have data set to arbitrary Unicode values. All | |
| 161 other fields must have ASCII values. | |
| 162 | |
| 163 | |
| 164 <B>VIEWING CURRENTLY SET METADATA</B> | |
| 165 You can see what metadata is currently set on an object by using: | |
| 166 | |
| 167 gsutil ls -L gs://the_bucket/the_object | |
| 168 """) | |
| 169 | |
| 170 | |
| 171 class CommandOptions(HelpProvider): | |
| 172 """Additional help about object metadata.""" | |
| 173 | |
| 174 help_spec = { | |
| 175 # Name of command or auxiliary help info for which this help applies. | |
| 176 HELP_NAME : 'metadata', | |
| 177 # List of help name aliases. | |
| 178 HELP_NAME_ALIASES : ['cache-control', 'caching', 'content type', | |
| 179 'mime type', 'mime', 'type'], | |
| 180 # Type of help: | |
| 181 HELP_TYPE : HelpType.ADDITIONAL_HELP, | |
| 182 # One line summary of this help. | |
| 183 HELP_ONE_LINE_SUMMARY : 'Working with object metadata', | |
| 184 # The full help text. | |
| 185 HELP_TEXT : _detailed_help_text, | |
| 186 } | |
| OLD | NEW |