OLD | NEW |
| (Empty) |
1 # Copyright 2012 Google Inc. All Rights Reserved. | |
2 # | |
3 # Licensed under the Apache License, Version 2.0 (the "License"); | |
4 # you may not use this file except in compliance with the License. | |
5 # You may obtain a copy of the License at | |
6 # | |
7 # http://www.apache.org/licenses/LICENSE-2.0 | |
8 # | |
9 # Unless required by applicable law or agreed to in writing, software | |
10 # distributed under the License is distributed on an "AS IS" BASIS, | |
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
12 # See the License for the specific language governing permissions and | |
13 # limitations under the License. | |
14 | |
15 from gslib.help_provider import HELP_NAME | |
16 from gslib.help_provider import HELP_NAME_ALIASES | |
17 from gslib.help_provider import HELP_ONE_LINE_SUMMARY | |
18 from gslib.help_provider import HelpProvider | |
19 from gslib.help_provider import HELP_TEXT | |
20 from gslib.help_provider import HelpType | |
21 from gslib.help_provider import HELP_TYPE | |
22 | |
23 _detailed_help_text = (""" | |
24 <B>OVERVIEW OF METADATA</B> | |
25 Objects can have associated metadata, which control aspects of how | |
26 GET requests are handled, including Content-Type, Cache-Control, | |
27 Content-Disposition, and Content-Encoding (discussed in more detail in | |
28 the subsections below). In addition, you can set custom metadata that | |
29 can be used by applications (e.g., tagging that particular objects possess | |
30 some property). | |
31 | |
32 There are two ways to set metadata on objects: | |
33 | |
34 - at upload time you can specify one or more headers to associate with | |
35 objects, using the gsutil -h option. For example, the following command | |
36 would cause gsutil to set the Content-Type and Cache-Control for each | |
37 of the files being uploaded: | |
38 | |
39 gsutil -h "Content-Type:text/html" -h "Cache-Control:public, max-age=360
0" cp -r images gs://bucket/images | |
40 | |
41 Note that -h is an option on the gsutil command, not the cp sub-command. | |
42 | |
43 - You can set or remove metadata fields from already uploaded objects using | |
44 the gsutil setmeta command. See "gsutil help setmeta". | |
45 | |
46 More details about specific pieces of metadata are discussed below. | |
47 | |
48 | |
49 <B>CONTENT TYPE</B> | |
50 The most commonly set metadata is Content-Type (also known as MIME type), | |
51 which allows browsers to render the object properly. | |
52 gsutil sets the Content-Type | |
53 automatically at upload time, based on each filename extension. For | |
54 example, uploading files with names ending in .txt will set Content-Type | |
55 to text/plain. If you're running gsutil on Linux or MacOS and would prefer | |
56 to have content type set based on naming plus content examination, see the | |
57 use_magicfile configuration variable in the gsutil/boto configuration file | |
58 (See also "gsutil help config"). In general, using use_magicfile is more | |
59 robust and configurable, but is not available on Windows. | |
60 | |
61 If you specify a -h header when uploading content (like the example gsutil | |
62 command given in the previous section), it overrides the Content-Type that | |
63 would have been set based on filename extension or content. This can be | |
64 useful if the Content-Type detection algorithm doesn't work as desired | |
65 for some of your files. | |
66 | |
67 You can also completely suppress content type detection in gsutil, by | |
68 specifying an empty string on the Content-Type header: | |
69 | |
70 gsutil -h 'Content-Type:' cp -r images gs://bucket/images | |
71 | |
72 In this case, the Google Cloud Storage service will attempt to detect | |
73 the content type. In general this approach will work better than using | |
74 filename extension-based content detection in gsutil, because the list of | |
75 filename extensions is kept more current in the server-side content detection | |
76 system than in the Python library upon which gsutil content type detection | |
77 depends. (For example, at the time of writing this, the filename extension | |
78 ".webp" was recognized by the server-side content detection system, but | |
79 not by gsutil.) | |
80 | |
81 | |
82 <B>CACHE-CONTROL</B> | |
83 Another commonly set piece of metadata is Cache-Control, which allows | |
84 you to control whether and for how long browser and Internet caches are | |
85 allowed to cache your objects. Cache-Control only applies to objects with | |
86 a public-read ACL. Non-public data are not cacheable. | |
87 | |
88 Here's an example of uploading an object set to allow caching: | |
89 | |
90 gsutil -h "Cache-Control:public,max-age=3600" cp -a public-read -r html gs:/
/bucket/html | |
91 | |
92 This command would upload all files in the html directory (and subdirectories) | |
93 and make them publicly readable and cacheable, with cache expiration of | |
94 one hour. | |
95 | |
96 Note that if you allow caching, at download time you may see older versions | |
97 of objects after uploading a newer replacement object. Note also that because | |
98 objects can be cached at various places on the Internet there is no way to | |
99 force a cached object to expire globally (unlike the way you can force your | |
100 browser to refresh its cache). | |
101 | |
102 | |
103 <B>CONTENT-ENCODING</B> | |
104 You could specify Content-Encoding to indicate that an object is compressed, | |
105 using a command like: | |
106 | |
107 gsutil -h "Content-Encoding:gzip" cp *.gz gs://bucket/compressed | |
108 | |
109 Note that Google Cloud Storage does not compress or decompress objects. If | |
110 you use this header to specify a compression type or compression algorithm | |
111 (for example, deflate), Google Cloud Storage preserves the header but does | |
112 not compress or decompress the object. Instead, you need to ensure that | |
113 the files have been compressed using the specified Content-Encoding before | |
114 using gsutil to upload them. | |
115 | |
116 For compressible content, using Content-Encoding:gzip saves network and | |
117 storage costs, and improves content serving performance (since most browsers | |
118 are able to decompress objects served this way). | |
119 | |
120 Note also that gsutil provides an easy way to cause content to be compressed | |
121 and stored with Content-Encoding:gzip: see the -z option in "gsutil help cp". | |
122 | |
123 | |
124 <B>CONTENT-DISPOSITION</B> | |
125 You can set Content-Disposition on your objects, to specify presentation | |
126 information about the data being transmitted. Here's an example: | |
127 | |
128 gsutil -h 'Content-Disposition:attachment; filename=filename.ext' \\ | |
129 cp -r attachments gs://bucket/attachments | |
130 | |
131 Setting the Content-Disposition allows you to control presentation style | |
132 of the content, for example determining whether an attachment should be | |
133 automatically displayed vs should require some form of action from the user to | |
134 open it. See http://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.5.1 | |
135 for more details about the meaning of Content-Disposition. | |
136 | |
137 | |
138 <B>CUSTOM METADATA</B> | |
139 You can add your own custom metadata (e.g,. for use by your application) | |
140 to an object by setting a header that starts with "x-goog-meta", for example: | |
141 | |
142 gsutil -h x-goog-meta-reviewer:jane cp mycode.java gs://bucket/reviews | |
143 | |
144 You can add multiple differently named custom metadata fields to each object. | |
145 | |
146 | |
147 <B>SETTABLE FIELDS; FIELD VALUES</B> | |
148 You can't set some metadata fields, such as ETag and Content-Length. The | |
149 fields you can set are: | |
150 - Cache-Control | |
151 - Content-Disposition | |
152 - Content-Encoding | |
153 - Content-Language | |
154 - Content-MD5 | |
155 - Content-Type | |
156 - Any field starting with X-GOOG-META- (i.e., custom metadata). | |
157 | |
158 Header names are case-insensitive. | |
159 | |
160 X-GOOG-META- fields can have data set to arbitrary Unicode values. All | |
161 other fields must have ASCII values. | |
162 | |
163 | |
164 <B>VIEWING CURRENTLY SET METADATA</B> | |
165 You can see what metadata is currently set on an object by using: | |
166 | |
167 gsutil ls -L gs://the_bucket/the_object | |
168 """) | |
169 | |
170 | |
171 class CommandOptions(HelpProvider): | |
172 """Additional help about object metadata.""" | |
173 | |
174 help_spec = { | |
175 # Name of command or auxiliary help info for which this help applies. | |
176 HELP_NAME : 'metadata', | |
177 # List of help name aliases. | |
178 HELP_NAME_ALIASES : ['cache-control', 'caching', 'content type', | |
179 'mime type', 'mime', 'type'], | |
180 # Type of help: | |
181 HELP_TYPE : HelpType.ADDITIONAL_HELP, | |
182 # One line summary of this help. | |
183 HELP_ONE_LINE_SUMMARY : 'Working with object metadata', | |
184 # The full help text. | |
185 HELP_TEXT : _detailed_help_text, | |
186 } | |
OLD | NEW |