Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(460)

Side by Side Diff: third_party/gsutil/gslib/addlhelp/versions.py

Issue 1380943003: Roll version of gsutil to 4.15. (Closed) Base URL: https://github.com/catapult-project/catapult.git@master
Patch Set: rebase Created 5 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # -*- coding: utf-8 -*- 1 # -*- coding: utf-8 -*-
2 # Copyright 2012 Google Inc. All Rights Reserved. 2 # Copyright 2012 Google Inc. All Rights Reserved.
3 # 3 #
4 # Licensed under the Apache License, Version 2.0 (the "License"); 4 # Licensed under the Apache License, Version 2.0 (the "License");
5 # you may not use this file except in compliance with the License. 5 # you may not use this file except in compliance with the License.
6 # You may obtain a copy of the License at 6 # You may obtain a copy of the License at
7 # 7 #
8 # http://www.apache.org/licenses/LICENSE-2.0 8 # http://www.apache.org/licenses/LICENSE-2.0
9 # 9 #
10 # Unless required by applicable law or agreed to in writing, software 10 # Unless required by applicable law or agreed to in writing, software
(...skipping 125 matching lines...) Expand 10 before | Expand all | Expand 10 after
136 136
137 Note that there is no limit to the number of older versions of an object you 137 Note that there is no limit to the number of older versions of an object you
138 will create if you continue to upload to the same object in a versioning- 138 will create if you continue to upload to the same object in a versioning-
139 enabled bucket. It is your responsibility to delete versions beyond the ones 139 enabled bucket. It is your responsibility to delete versions beyond the ones
140 you want to retain. 140 you want to retain.
141 141
142 142
143 <B>COPYING VERSIONED BUCKETS</B> 143 <B>COPYING VERSIONED BUCKETS</B>
144 You can copy data between two versioned buckets, using a command like: 144 You can copy data between two versioned buckets, using a command like:
145 145
146 gsutil cp -r gs://bucket1/* gs://bucket2 146 gsutil cp -r -A gs://bucket1/* gs://bucket2
147 147
148 When run using versioned buckets, this command will cause every object version 148 When run using versioned buckets, this command will cause every object version
149 to be copied. The copies made in gs://bucket2 will have different generation 149 to be copied. The copies made in gs://bucket2 will have different generation
150 numbers (since a new generation is assigned when the object copy is made), 150 numbers (since a new generation is assigned when the object copy is made),
151 but the object sort order will remain consistent. For example, gs://bucket1 151 but the object sort order will remain consistent. For example, gs://bucket1
152 might contain: 152 might contain:
153 153
154 % gsutil ls -la gs://bucket1 10 2013-06-06T02:33:11Z 154 % gsutil ls -la gs://bucket1 10 2013-06-06T02:33:11Z
155 53 2013-02-02T22:30:57Z gs://bucket1/file#1359844257574000 metageneration =1 155 53 2013-02-02T22:30:57Z gs://bucket1/file#1359844257574000 metageneration =1
156 12 2013-02-02T22:30:57Z gs://bucket1/file#1359844257615000 metageneration =1 156 12 2013-02-02T22:30:57Z gs://bucket1/file#1359844257615000 metageneration =1
157 97 2013-02-02T22:30:57Z gs://bucket1/file#1359844257665000 metageneration =1 157 97 2013-02-02T22:30:57Z gs://bucket1/file#1359844257665000 metageneration =1
158 158
159 and after the copy, gs://bucket2 might contain: 159 and after the copy, gs://bucket2 might contain:
160 160
161 % gsutil ls -la gs://bucket2 161 % gsutil ls -la gs://bucket2
162 53 2013-06-06T02:33:11Z gs://bucket2/file#1370485991580000 metageneration =1 162 53 2013-06-06T02:33:11Z gs://bucket2/file#1370485991580000 metageneration =1
163 12 2013-06-06T02:33:14Z gs://bucket2/file#1370485994328000 metageneration =1 163 12 2013-06-06T02:33:14Z gs://bucket2/file#1370485994328000 metageneration =1
164 97 2013-06-06T02:33:17Z gs://bucket2/file#1370485997376000 metageneration =1 164 97 2013-06-06T02:33:17Z gs://bucket2/file#1370485997376000 metageneration =1
165 165
166 Note that the object versions are in the same order (as can be seen by the 166 Note that the object versions are in the same order (as can be seen by the
167 same sequence of sizes in both listings), but the generation numbers (and 167 same sequence of sizes in both listings), but the generation numbers (and
168 timestamps) are newer in gs://bucket2. 168 timestamps) are newer in gs://bucket2.
169 169
170 WARNING: If you use the gsutil -m option when copying the objects (to parallel
171 copy the data), object version ordering will NOT be preserved. All object
172 versions will be copied, but (for example) the latest/live version in the
173 destination bucket might be from one of the earlier versions in the source
174 bucket (and similarly, other versions may be out of order). When copying
175 versioned data it is advisable not to use the gsutil -m option.
176 170
177 171
178 <B>CONCURRENCY CONTROL</B> 172 <B>CONCURRENCY CONTROL</B>
179 If you are building an application using Google Cloud Storage, you may need to 173 If you are building an application using Google Cloud Storage, you may need to
180 be careful about concurrency control. Normally gsutil itself isn't used for 174 be careful about concurrency control. Normally gsutil itself isn't used for
181 this purpose, but it's possible to write scripts around gsutil that perform 175 this purpose, but it's possible to write scripts around gsutil that perform
182 concurrency control. 176 concurrency control.
183 177
184 For example, suppose you want to implement a "rolling update" system using 178 For example, suppose you want to implement a "rolling update" system using
185 gsutil, where a periodic job computes some data and uploads it to the cloud. 179 gsutil, where a periodic job computes some data and uploads it to the cloud.
(...skipping 78 matching lines...) Expand 10 before | Expand all | Expand 10 after
264 258
265 # Help specification. See help_provider.py for documentation. 259 # Help specification. See help_provider.py for documentation.
266 help_spec = HelpProvider.HelpSpec( 260 help_spec = HelpProvider.HelpSpec(
267 help_name='versions', 261 help_name='versions',
268 help_name_aliases=['concurrency', 'concurrency control'], 262 help_name_aliases=['concurrency', 'concurrency control'],
269 help_type='additional_help', 263 help_type='additional_help',
270 help_one_line_summary='Object Versioning and Concurrency Control', 264 help_one_line_summary='Object Versioning and Concurrency Control',
271 help_text=_DETAILED_HELP_TEXT, 265 help_text=_DETAILED_HELP_TEXT,
272 subcommand_help_text={}, 266 subcommand_help_text={},
273 ) 267 )
OLDNEW
« no previous file with comments | « third_party/gsutil/gslib/addlhelp/throttling.py ('k') | third_party/gsutil/gslib/addlhelp/wildcards.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698