Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(826)

Side by Side Diff: third_party/gsutil/CHANGES.md

Issue 1377933002: [catapult] - Copy Telemetry's gsutilz over to third_party. (Closed) Base URL: https://github.com/catapult-project/catapult.git@master
Patch Set: Rename to gsutil. Created 5 years, 2 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | third_party/gsutil/CHECKSUM » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 Release 4.13 (release date: 2015-06-03)
2 =======================================
3 New Features
4 ------------
5 - Added -U flag to cp and rsync commands to allow skipping of unsupported
6 object types.
7 - Added support for Google Developer Shell credentials.
8
9 Bug Fixes
10 ---------
11 - Precondition headers (x-goog-if-...) are now respected for the setmeta
12 command.
13 - Fixed an index out of range error that could occur with an empty
14 parallel composite upload tracker file.
15 - The stat command outputs errors to stderr instead of stdout.
16 - Fixed two possible sources of ResumableUploadStartOverException from
17 httplib2 and oauth2client.
18 - Fixed a bug in the compose command where a missing source object resulted
19 in an error message claiming the destination object was missing.
20
21 Other Changes
22 -------------
23 - Added a help section on throttling gsutil.
24 - Resumable uploads will now start over if a PUT to the upload ID returns
25 an HTTP 404. Previously this behavior applied only to an HTTP 410.
26 - XML API resumable uploads now retry on HTTP 429 errors, matching the
27 behavior of JSON API resumable uploads.
28 - Improved response to process kill signals, reducing the likelihood of
29 leaving orphaned child processes and temporary files.
30 - Bucket lifecycle configuration now works for S3.
31 - Removed the deprecated setmeta -n option.
32
33
34 Release 4.12 (release date: 2015-04-20)
35 =======================================
36 New Features
37 ------------
38 - Added support for JSON-format service account private key files.
39 - Added support for the Rewrite API (JSON API only). This is used for
40 all copies within the Google Cloud and supports copying objects across
41 storage classes and/or locations.
42
43 Bug Fixes
44 ---------
45 - Fixed a bug that could cause downloads to have a hash mismatch (and deletion
46 of the corrupted file) when resumed across process breaks via a tracker file.
47
48 Other Changes
49 -------------
50 - Updated documentation and examples for several topics including
51 acl, cp, dev, signurl, stat, and wildcards.
52
53
54 Release 4.11 (release date: 2015-03-10)
55 =======================================
56 New Features
57 ------------
58 - Added Nearline storage class support to the mb command.
59
60 Bug Fixes
61 ---------
62 - Fixed a bug for streaming uploads that could occasionally cause a HTTP 410
63 from the service or a hash mismatch (and deletion of the corrutped file).
64 - Fixed an OverflowError that occurred when uploading files > 4GiB on a 32-bit
65 operating system.
66
67 Other Changes
68 -------------
69 - Added documentation around using the Content-MD5 header to extend integrity
70 checking to include checksums computed by a client-side content pipeline.
71
72
73 Release 4.10 (release date: 2015-03-03)
74 =======================================
75 Bug Fixes
76 ---------
77 - Fixed a bug that could cause undetected data corruption (preserving incorrect
78 data) if a streaming upload encountered a service error on non-8KiB-aligned
79 boundary.
80 - Fixed a bug that caused downloads to be truncated if the connection broke,
81 resulting in a hash mismatch (and deletion of the corrupted file) for that
82 download.
83 - Fixed a format string arguments error that occurred if a download exhausted
84 all retries.
85
86 Other Changes
87 -------------
88 - The lifecycle command now accepts JSON input in the form of
89 "{ "lifecycle": { "rule" ..." in addition to "{ "rule": ...".
90 - Improved access token expiry logic for GCE credentials.
91
92
93 Release 4.9 (release date: 2015-02-13)
94 =======================================
95 New Features
96 ------------
97 - When using the JSON API, the ch acl/defacl subcommand now supports
98 project groups via the -p flag. For details, see "gsutil help acl ch".
99
100 Bug Fixes
101 ---------
102 - Fixed a bug that caused daisy-chain copies (including cross-provider
103 copies) for files large than 100MiB to fail.
104 - Fixed a bug that caused streaming uploads than ran for longer than
105 an hour to fail with HTTP 400s.
106 - Fixed a bug where perfdiag would not properly clean up its test files.
107 - Fixed a bug where using ls with the XML API could mistakenly report bucket
108 configuration as present.
109
110 Other Changes
111 -------------
112 - Updated documentation for metadata, retries, security, and subdirs.
113 - Tracker files are no longer written for small downloads.
114
115
116 Release 4.8 (release date: 2015-01-23)
117 =======================================
118 New Features
119 ------------
120 - gsutil now supports HTTP proxy configuration via the http_proxy,
121 https_proxy, or HTTPS_PROXY environment variables. This configuration
122 is used only if proxy configuration is not present in the .boto
123 configuration file.
124 - gsutil rsync now supports regex-based source and destination URL
125 exclusion via the -x flag.
126 - The rm command now supports arguments on stdin via the -I flag.
127
128 Bug Fixes
129 ---------
130 - Fixed a bug where perfdiag would fail if netstat was not available.
131 - Fixed a bug where temporary ca_certs files were not being cleaned up.
132 - Fixed a bug in rsync to unnecessarily remove or write objects, in some
133 cases leaving the destination in a non-synchronized state.
134 caused rsync to unnecessarily remove or rewrite objects.
135 - Fixed a bug where rsync temporary listing files were not being
136 cleaned up when the rsync process was killed.
137 - Fixed a bug where rsync would remove destination URLs if listing the
138 source encountered a non-retryable failure (for example, if the source
139 did not exist).
140 - Fixed a bug where mv would fail for some Unicode filenames.
141 - Fixed a bug where mv would remove the source URL after skipping the
142 destination URL.
143 - Fixed a bug that caused daisy chain uploads to hang if the download thread
144 raised an exception.
145 - Fixed a bug where acl ch would return a zero exit code even if it failed.
146 - Fixed a bug that sometimes caused the progress display to render multiple
147 times at the end of an upload or download.
148
149 Other Changes
150 -------------
151 - Resumable uploads of files using the JSON API now send their data in a
152 single request, making separate HTTP calls only when resuming is necessary.
153 - The test command now runs tests in parallel by default, and test
154 parallelism on Windows is now supported.
155 - All non-streaming downloads are now resumable (and retryable) by default,
156 regardless of size.
157 - Canned ACLs and canned default object ACLs are now supported in the JSON
158 API (previously they would fall back to using the XML API).
159 - Google Compute Engine service account credential tokens are now cached,
160 avoiding unnecessary refreshes.
161 - Improved detection of the Google Compute Engine metadata server,
162 particularly when using the -m flag for multiprocessing.
163 - Added new help sections about filename encoding and security/privacy
164 considerations.
165 - Download progress is now displayed for small XML API downloads.
166
167
168 Release 4.7 (release date: 2014-11-17)
169 =======================================
170 New Features
171 ------------
172 - Tab completion now works on gs:// URLs (for Cloud SDK installs only).
173 To install via Cloud SDK, see https://cloud.google.com/sdk/#Quick_Start
174 - Streaming uploads (with source URL "-") using the JSON API now buffer
175 in-memory up to 100MB, allowing large streams to be retried in the event
176 of network flakiness or service errors.
177 - Resumable uploads that receive a fatal service error (typically a 410)
178 are now automatically retried from the beginning.
179
180 Bug Fixes
181 ---------
182 - Fixed an apitools bug that impacted upload performance and caused
183 "Retrying from byte __ after exception" to print after every 100MiB.
184 - Fixed _$folder$ placeholder object detection on versioned buckets.
185 - Removed an unnecessary credential check on load which increased
186 startup time by over one second in some cases.
187 - SignURL now properly retries when checking if the signed object is
188 readable.
189 - Files with both Content-Encoding and Content-Type gzip are now properly
190 removed when hash validation fails (only one of the two should be set).
191 - The x-goog-if-generation-match:0 header now works properly with the XML API.
192 - Fixed a bug that caused "defacl ch" on a bucket with a private ACL to fail.
193 - The rm command now properly supports precondition headers.
194 - Fixed a bug that caused large streaming uploads to fail with the message
195 "Failure: length too large" when using the JSON API.
196 - Fixed a bug that caused JSON lifecycle configurations with createdBefore
197 conditions to fail with a DecodeError.
198
199 Other Changes
200 -------------
201 - Byte counts now display accurate abbreviations of binary sizing. For example,
202 messages previously labeled MB are now properly labeled MiB to indicate
203 2**20 bytes. Only the labeling changed - the actual sizes were always binary.
204 - Improved Cloud SDK integration, including improved error messages
205 and instructions.
206 - The num_retries .boto configuration value now applies to all requests
207 (previously it was ignored for JSON API requests).
208 - rsync now works with non-existent destination subdirectories.
209 - Raised the default resumable upload threshold from 2MB to 8MB to
210 improve performance.
211 - Benign retry messages now print only when debug mode is enabled via the
212 top-level -d flag.
213 - The top-level -q flag now suppresses suggestions to use the -m flag.
214 - Command synopsis is now output when the wrong number of arguments are
215 provided.
216 - Removed dependency on google-api-python-client module, added dependencies on
217 oauth2client and six modules.
218
219 Release 4.6 (release date: 2014-09-08)
220 =======================================
221
222 Bug Fixes
223 ---------
224 - Fixed a TypeError bug that occurred in perfdiag write throughput tests.
225 - Fixed an rsync bug that caused invalid symlinks to abort the transfer
226 even when -e option was specified.
227 - Fixed a perfdiag assumption that ipaddrlist was populated.
228 - Fixed an AttributeError when setting an invalid canned ACL with defacl set.
229 - Fixed a bug where non-resumable uploads would include payload in debug output
230 when for running in debug mode (-D).
231
232 Other Changes
233 -------------
234 - Added the proxy_rdns configuration variable for clients that
235 do DNS lookups via a proxy.
236 - Added the state_dir configuration variable for choosing the location of
237 gsutil's internal state files, including resumable transfer tracker files.
238 resumable_tracker_dir configuration variable is now deprecated.
239 - Added DNS, connection latency, and proxy use information to perfdiag
240 command.
241 - perfdiag command will not perform DNS lookups if they are disabled in
242 boto config.
243 - perfdiag command will now only attempt to delete uploaded objects when
244 running write tests.
245 - Added code coverage support to test command.
246 - rsync -d now succeeds on a 404 for a to-be-deleted object (for example, when
247 the object was already deleted by an external process).
248
249 Release 4.5 (release date: 2014-08-14)
250 =======================================
251
252 Bug Fixes
253 ---------
254 - Fixed a bug that caused resumable uploads to restart if gsutil was
255 terminated with CTRL-C.
256 - Fixed a bug in defacl ch command that caused a failure when updating
257 an existing default object ACL entry.
258 - Fixed an invalid literal bug during rsync file listing.
259 - Made several improvements to JSON upload stability, including fixing a bug
260 which occasionally caused resumable upload hashes not to catch up properly.
261 - All JSON calls now have socket timeouts, eliminating hangs under
262 flaky network conditions.
263 - Fixed a bug where acl ch -g AllAuthenticatedUsers would instead add
264 AllUsers.
265 - Fixed a bug that caused object custom metadata not to be preserved when
266 copying in the cloud.
267 - Fixed a bug where du -s did not properly elide subdirectories.
268
269 Other Changes
270 -------------
271 - Parallel composite uploads are now disabled by default until crcmod is
272 available in major Linux distributions. To re-enable the setting from
273 prior versions, in the [GSUtil] section of your .boto config file, add:
274 parallel_composite_upload_threshold=150M
275 - Non-wildcarded URLs for existing objects now use Get before trying List
276 (as in gsutil3), and thus are not subject to eventual listing consistency.
277 - gsutil -D now redacts proxy configuration values in the output.
278
279 Release 4.4 (release date: 2014-07-17)
280 =======================================
281
282 New Features
283 ------------
284 - Added the hash command, which can calculate hashes of local files.
285 gsutil already calculates hashes for integrity checking, but this allows
286 the user to separately calculate the MD5 and CRC32c hashes of a local file.
287
288 Bug Fixes
289 ---------
290 - Many improvements to JSON API media transfers, including far
291 more robust retry logic.
292 - Fixed "File changed during upload: EOF..." errors on XML resumable uploads.
293 - Fixed rsync command to read and write index files in binary mode.
294 - Fix potential TypeError in _CheckAndHandleCredentialException.
295 - Fixed possible data corruption when using JSON API uploads for
296 small files with lines starting with "From:", which would cause
297 integrity checks to fail.
298 - Fixed gsutil cp to skip directory placeholders when downloading, avoiding
299 "directory exists where the file needs to be created" errors.
300 - Fixed daisy chain cp/rsync for files >= 100MB.
301 - Fixed a bug in JSON proxy support where the proxy info was sometimes unused.
302 - Fixed a bug where an acl get on a private default object ACL returned an
303 error instead of a blank ACL.
304 - Fixed a JSON API issue with large HTTP responses on 32-bit systems.
305
306 Other Changes
307 -------------
308 - Improved object listing performance when using the XML API.
309 - Improved various error messages.
310 - Improved progress display during media transfer.
311 - Switched to truncated exponential backoff for retries.
312 - Improved OS-specific ulimit checks.
313 - Added some information such as OS and Cloud SDK wrapping to gsutil version,
314 and changed the output format to be more uniform.
315 - Daisy chain cp/rsync now supports resumable uploads.
316 - Improved proxy support for proxy username and passwords.
317 - x-amz headers are now supported for cp, rsync, and setmeta. x-amz-meta
318 headers continue to be supported as well.
319
320 Release 4.3 (release date: 2014-06-10)
321 =======================================
322
323 Bug Fixes
324 ---------
325 - Fix acl/defacl ch changing the role of an existing group.
326 - Fix unicode and 404 errors when using manifests.
327 - Fix parallelism configuration bug that limited gsutil rsync to two threads
328 and could lead to rsync hangs. "gsutil -m rsync" runs much faster, and rsync
329 uploads of large local files are now faster via parallel composite upload.
330 Parallel composite uploads of large files are also faster.
331 - Fix rsync bug with parallel composite uploads.
332 - Fix TypeError that could occur when running the cp command with no
333 credentials.
334
335 Other Changes
336 -------------
337 - Progress indicators for -m cp/rsync commands are now more readable.
338 - Added command being run to gsutil -d/-D output.
339 - Lowered default parallelism for 'gsutil -m test' and added hang detection.
340
341 Release 4.2 (release date: 2014-06-05)
342 =======================================
343
344 New Features
345 ------------
346 - Added parallel test execution support to test command, ex: "gsutil -m test"
347
348 Bug Fixes
349 ---------
350 - Fix failure during retry of an XML download.
351 - Moved to boto release 2.29.1 fixing boto authentication erroneously
352 reporting OAuth2 credentials as invalid.
353 - Fix parallel composite uploads when using only a single process and thread.
354 - Fix an invalid seek during daisy chain operation that affected file copy
355 from Google Cloud Storage -> S3 for files greater than 8KB in size.
356 - Fix "gsutil acl ch" with AllUsers or AllAuthenticatedUsers groups.
357 - Fix some copy errors writing new lines to the manifest file.
358 - Fix "gsutil test" return code to properly be 0 on success.
359
360 Other Changes
361 -------------
362 - "gsutil cp -z" now ignores whitespace in the extension list.
363
364 Release 4.1 (release date: 2014-05-28)
365 =======================================
366
367 Bug Fixes
368 ---------
369 - Fixed a bug in parallel composite uploads where uploads with
370 existing components would fail.
371 - Moved gcs-oauth2-boto-plugin to version 1.5, fixing a bug in the PyPi gsutil
372 distribution that would cause gsutil to unnecessarily attempt to query
373 the Google Compute Engine metadata service.
374
375 Other Changes
376 -------------
377 - Parallel composite uploads no longer specify an if-not-match precondition
378 when uploading component parts.
379 - Parallel composite uploads no longer calculate a CRC32c hash prior to
380 uploading component parts (these are still validated by an MD5 hash).
381 - Removed apitools dependency on gflags.
382
383 Release 4.0 (release date: 2014-05-27)
384 =======================================
385
386 Major New Gsutil Version - Backwards-Incompatible Changes
387 ------------------------------
388 - The Google Cloud Storage configuration data supported by the acl, cors,
389 and lifecycle commands now uses the JSON format instead of the older XML
390 format. gsutil 4.0 will fail and provide conversion instructions if an XML
391 configuration file is provided as an argument for a gs:// URL.
392 - gsutil no longer accepts arbitrary headers via the global -h flag.
393 Documented headers for gsutil commands are still supported; for the
394 full list of supported headers, see "gsutil help command_opts".
395 - The compose command will now default the destination object's
396 Content-Type to the Content-Type of the first source object if none
397 is provided via the -h global flag.
398 - The long-deprecated -t and -q options have been removed from the cp command.
399 - The perfdiag command no longer supports adding a host header.
400 - Having OAuth2 User Account credentials and OAuth2 Service Account
401 credentials configured simultaneously will now fail with an error message
402 to avoid confusion. Also, a single invalid credential will fail with an
403 error message. See "gsutil help creds" for details.
404 - Bucket relocate scripts have been removed.
405 - Downloading object names ending with '/' is no longer supported to avoid
406 problems this caused for directores using the Google Cloud Console.
407 - rm -r now implies rm -ra (removing all object versions recursively).
408 - All commands using the global -m option or a force option (such as
409 rm -f or cp -c) will now return a non-zero exit code if there are any
410 failures during the operation.
411 - MD5 and CRC32c values are now represented in base64 encoding instead
412 of hex encoding (this includes manifest files).
413
414 New Features
415 ------------
416 - The Google Cloud Storage JSON API (v1) is now the default API used
417 by gsutil for all commands targeting gs:// URLs. The JSON API is more
418 bandwidth efficient than the older XML API when transferring metadata
419 and does not require separate calls to preserve object ACLs when copying.
420 The XML API will automatically be used when accessing s3:// URLs.
421 - The Google Cloud Storage XML API can be used in lieu of the JSON API
422 by setting 'prefer_api = xml' in the GSUtil section of your boto config file.
423 - Added the rsync command that can synchronize cloud and local directories.
424 - Added the signurl command that can generate Google Cloud Storage signed URLs.
425 - The perfdiag command now supports a listing latency test.
426 - The rb command now supports a -f flag allowing it to continue when errors
427 are encountered.
428 - The test command now supports a -s flag that runs tests against S3.
429
430 Other Changes
431 -------------
432 - All python files not under a third_party directory are now pylint-clean,
433 with the exception of TODO-format and a handful of warnings in root-level
434 files. As part of the de-linting process, many edge-case bugs were
435 identified and fixed.
436 - The ls command now operates depth-first (as in Unix ls) instead
437 of breadth-first.
438 - Daisy-chain copying does not currently support resumable uploads.
439 - Several compatibility improvements for Windows and S3.
440
441
442 Release 3.42 (release-date: 2014-01-15)
443 =======================================
444
445 Other Changes
446 -------------
447
448 - Fixed potential bug with update command on CentOS.
449
450
451 Release 3.41 (release-date: 2014-01-14)
452 =======================================
453
454 Other Changes
455 -------------
456
457 - Changes to protect security of resumable upload IDs.
458
459
460 Release 3.40: Skipped
461
462
463 Release 3.39: Skipped
464
465
466 Release 3.38 (release-date: 2013-11-25)
467
468 Bug Fixes
469 ---------
470
471 - Fix to include version number in user-agent string.
472 - Fix bug wherein -m flag or parallel uploads caused crash on systems without
473 /dev/shm.
474 - Fix SSL errors and invalid results with perfdiag -c and -k rthru test.
475 - Fixed cases where parallel composite uploads could leave orphaned components.
476 - Fix bug attempting to stat objects you don't have auth to read.
477 - Fixed bug breaking defacl's -d option.
478
479
480 Other Changes
481 -------------
482
483 - Fixed gsutil config doc.
484 - Fixed references to old command names; fix defacl ch example.
485 - Improved error messages for deprecated command aliases.
486 - Updated gsutil support info.
487
488
489 New Features
490 ------------
491
492 - Enabled -R flag for recursion with setmeta command.
493
494
495 Release 3.37 (release-date: 2013-09-25)
496 =======================================
497
498 Bug Fixes
499 ---------
500
501 - Fix parsing of -R for "acl ch" and chacl commands.
502 - Fixed import statement of unittest2 which caused installations using Python
503 2.6 without unittest2 installed to fail when starting up gsutil.
504
505
506 Other Changes
507 -------------
508
509 - Fixed tests so they pass on Windows and package installs.
510 - Add a root logging handler manually instead of relying on basicConfig.
511 - Fix apiclient import statement.
512 - Exponential backoff for access token requests.
513 - Fix flakiness in test TearDown to account for eventual consistency of object
514 listings.
515
516
517 Release 3.36 (release-date: 2013-09-18)
518 =======================================
519
520 Bug Fixes
521 ---------
522
523 - Fix bug when a 400 or 403 exception has no detail.
524 - Fix bugs with config -e and config -o.
525
526
527 Other Changes
528 -------------
529
530 - Clarify stat command documentation regarding trailing slashes.
531 - Add Generation and Metageneration to gsutil stat output.
532
533
534 New Features
535 ------------
536
537 - Set config values from command line with -o.
538
539
540 Release 3.35 (release-date: 2013-09-09)
541 =======================================
542
543 Bug Fixes
544 ---------
545
546 - Fix streaming upload to S3 and provide more useful stack traces multi-threaded failures.
547 - Fix race condition in test_rm.
548 - Fix retry decorator during test bucket cleanup.
549 - Fixed cat bug that caused version to be ignored in URIs.
550 - Don't decode -p or -h values other than x-goog-meta-. Fixes ability to use str ing project names.
551 - Update bucket_relocate.sh to work on GCE.
552 - Fix recursive uploading from subdirectories with unexpanded wildcard as source URI.
553 - Make gsutil error text include <Message> content.
554 - Change shebang line back to python because this doesn't work on some systems.
555 - Fix hash_algs differences in perfdiag.
556 - Update Python version check and shebang line.
557 - Enforce project_id entry in config command; provide friendly error if missing proj ID.
558 - Use transcoding-invariant headers when available in gs.Key.
559 - Make gsutil cp not fail if unable to check versioning config on dest bucket.
560 - Make gsutil detect when config fails because of proxy and prompt for proxy con fig.
561 - Avoid checking metageneration attribute when long-listing S3 objects.
562 - Exclude the no-op auth handler as indicating credentials are configured.
563
564
565 New Features
566 ------------
567
568 - Implemented gsutil stat command.
569
570
571 Other Changes
572 -------------
573
574 - Consolidate config-related commands.
575 - Changed rm -r gs://bucket to delete bucket at end.
576 - Various doc cleanup and improvement.
577 - Warn user before updating to major new version. Also fixed minor version compa rison bug, and added tests.
578 - Change max component count to 1024.
579 - Add retry-decorator as a submodule.
580 - Explicitly state control chars to avoid in gsutil naming documentation.
581 - Make config command recommend project strings.
582 - Made long listing format a little better looking.
583 - Allow --help flag for subcommands.
584 - Implement help for subcommands and add OPTIONS sections for subcommands.
585 - Add more detailed error message to notification watchbucket command.
586 - Add notification URL configuration for notification tests.
587 - Refactor to use upstream retry_decorator as external dependency.
588 - Distribute cacerts file with gsutil.
589 - Updated gsutil help to point to Google Cloud Console instead of older APIs con sole.
590 - Make gsutil pass bundled cacerts.txt to oauth2client; stop checking SHA1 of ce rts, now that we no longer depend on boto distribution.
591 - Move all TTY checks to a common util function and mock it for update tests.
592 - Fix duplicate entry created in .gitmodules.
593 - Fix unit test breakage because VERSION file is old.
594 - Fix test using ? glob with ObjectToURI.
595 - Fix update tests that fail for package installs.
596 - Change bucket delete teardown to try more times.
597 - Fix tests that perform operations on bucket listings.
598 - Keep package install set to True unless VERSION file doesn't exist.
599 - Fix handling of non-numeric version strings in update test.
600
601
602 Release 3.34 (release-date: 2013-07-18)
603 =======================================
604
605 Bug Fixes
606 ---------
607
608 - Fixed a bug where the no-op authentication handler was being loaded after
609 other authentication plugins, causing the no-op handler to be chosen instead
610 of other valid credentials.
611
612
613 Release 3.33 (release-date: 2013-07-16)
614 =======================================
615
616 Bug Fixes
617 ---------
618
619 - Added .git* to MANIFEST.in excludes and fixed cp doc typo. This was needed to
620 overcome problem caused by accidental inclusion of .git* files in release,
621 which caused the update command no longer to allow updates (since starting
622 in 3.32 it checks whether the user has any extraneous files in the gsutil
623 directory before updating)
624
625
626 Release 3.32 (release-date: 2013-07-16)
627 =======================================
628
629 New Features
630 ------------
631
632 - Added support for getting and setting lifecycle configuration for buckets.
633 - Implemented Parallel Composite Uploads.
634 - Added a new du command that displays object size, similar to Linux du.
635
636
637 Bug Fixes
638 ---------
639
640 - Fixed a bug when using ls -R on objects with trailing slashes. Closes #93.
641 - Fixed so won't crash in perfdiag when nslookup is missing or gethostbyname
642 fails.
643 - Smartly compare version strings during autoupdate check.
644 - Made header handling for upload case-insensitive.
645 - Re-enabled software update check for users with no credentials configured.
646 - Fixed incorrectly-generated password editing comment in service account
647 config. Fixes #146.
648
649
650 Other Changes
651 -------------
652
653 - Improved flow when encounter auth failure for GCE service account with no
654 configured storage scopes:
655 1. Changed HasConfiguredCredentials() logic not to include
656 has_auth_plugins as part of the evaluated expression, since that will
657 always evaluate to true under GCE (since GCE configures its internal
658 service account plugin under /etc/boto.cfg).
659 2. Changed ConfigureNoOpAuthIfNeeded logic so we configure no-op auth
660 plugin even if there is a config_file list, since GCE always configures
661 /etc/boto.cfg, even if user has no storage scopes configured.
662 3. Additional changes:
663 a. Removed assertion of oauth access token cache check log from
664 test_Doption.py, which may not be true sometimes (e.g., if user is
665 using HMAC creds).
666 b. Removed remnants of CONFIG_REQUIRED left over from earlier CL.
667 c. Merged dupe _ConfigureNoOpAuthIfNeeded functions from two code files,
668 moved to util.py.
669 - Fixed confusing gsutil rm "Omitting" message.
670 - Wrapped long gsutil update message.
671 - Silenced additional possible perfdiag errors.
672 - Improved perfdiag performance by only generating one chunk of random file.
673 - Changed to swallow broken pipe errors when piping gsutil to other programs.
674 - Made DotfulBucketNameNotUnderTld error message more user friendly.
675 - Extracted function for building ACL error text from main try/except loop,
676 for better readability.
677 - Disallowed gsutil update when user data present in gsutil dir.
678 - Plumbed accept-encoding into HEAD requests in ls -L command.
679 - Updated README and moved ReleaseNotes.txt to CHANGES.md.
680 - Updated crcmod docs with link to Windows installer.
681 - Updated documentation regarding gzip content-encoding.
682 - Removed StorageUri parse check for lone ':' (interferes with using filenames
683 containing ':')
684 - Added tests for gsutil update check and fixed bug for bad file contents.
685 - Set accept-encoding and handle gzip on-the-fly encoding.
686
687
688 Release 3.31 (release-date: 2013-06-10)
689 =======================================
690
691 New Features
692 ------------
693
694 - Implemented consumption of manifest files for cp.
695 - Add ETag to ls -l and make ls -b variants more efficient.
696 - Expand the manifest path to allow for tildes in paths.
697 - Added bucket_relocate.sh script to gsutil.
698
699
700 Bug Fixes
701 ---------
702
703 - Fix gsutil cp -R to copy all versioned objects.
704 - Fixed bug where gsutil cp -D -n caused precondition failure.
705 - Fixed gsutil daisy-chain copy to allow preserving ACLs when copying within
706 same provider.
707 - Fix identification of non-MD5 ETags.
708 - Fixed bugs where gsutil -q cp and gsutil cp -q sometimes weren't quiet.
709 - Fixed unicode error when constructing tracker filename from non-ASCII
710 Unicode filename.
711 - Fixed that noclobber would not resume partial resumable downloads.
712 - Fixed bug when running gsutil cp -Dp by user other than object owner.
713 - Properly encode metageneration and etag in ls output with -a and -e.
714 - Update resumable threshold stated in gsutil help prod.
715
716
717 Other Changes
718 -------------
719
720 - ls -Lb no longer shows total # files/total size of bucket, so that ls -Lb
721 instead provides an efficient way to view just the metadata for large buckets.
722 - Catch and ignore EEXIST error when creating gsutil tracker dir.
723 - Add note to gsutil update doc about auto-update checks being disabled with
724 gsutil -q option.
725 - Disable hashing and increase buffer size in perfdiag.
726 - Added better error messages for service account auth.
727 - Make perfdiag behave more like normal gsutil, with multi-threading option.
728 - Changed so auto-update check/prompt aren't made if gsutil -q specified.
729 - Changed gsutil mb command to clarify that EU means European Union.
730 - Added doc warnings about losing version ordering if using gsutil -m cp
731 between versioned buckets; removed trailing whitespace.
732 - Added to gsutil cp -L doc to describe how to build a reliable script for
733 copying many objects.
734
735
736 Release 3.30 (release-date: 2013-06-10)
737 =======================================
738
739 - Abandoned.
740
741
742 Release 3.29 (release-date: 2013-05-13)
743 =======================================
744
745 Bug Fixes
746 ---------
747
748 - Fixed incorrect package installation detection that resulted in not being
749 able to run the update command while running gsutil from a symlink.
750
751 Other Changes
752 -------------
753
754 - Added a test for debug mode (gsutil -D) output.
755 - List numbering and title case fixes in additional help pages.
756 - Removed dateutil module dependency from cp command test.
757 - Updated documentation to clarify that public-read objects are cached for 1
758 hour by default.
759 - Added a filter to suppress "module was already imported" warnings that were
760 sometimes printed while running gsutil on Google Compute Engine instances.
761
762
763 Release 3.28 (release-date: 2013-05-07)
764 =======================================
765
766 New Features
767 ------------
768
769 - Added support for new Object Change Notifications feature.
770
771 Bug Fixes
772 ---------
773
774 - Fixed problem where gsutil update command didn’t take default action.
775 - Fixed a problem with the update command sometimes triggering an additional
776 update command.
777
778 Other Changes
779 -------------
780
781 - Add packaging information to version output.
782 - Removed fancy_urllib, since it is no longer used.
783 - Changed num_retries default for resumable downloads to 6.
784 - Don’t check for newer software version if gs_host is specified in boto
785 config file.
786 - Modified oauth2client logging behavior to be consistent with gsutil.
787 - Added gs_port configuration option.
788 - Skip update tests when SSL is disabled.
789
790
791 Release 3.27 (release-date: 2013-04-25)
792 =======================================
793
794 New Features
795 ------------
796
797 - Added a human readable option (-h) to ls command.
798 - Changed WildcardIterator not to materialize list of all matching files from
799 directory listing (so works faster when walking over large directories)
800 - Added -f option to setacl command to allow command to continue after errors
801 encountered.
802 - Add manifest log support for the cp command.
803 - Added never option for check_hashes_config; fixed bug that assumes an ETag
804 is always returned from server.
805 - Made gsutil provide friendlier error message if attempting non-public data
806 access with missing credentials.
807 - Set 70 second default socket timeout for httplib.
808 - Add ability to run a single test class or function with the test command.
809
810 Bug Fixes
811 ---------
812
813 - Don't check for updates if the user has no credentials configured. This
814 fixes a bug for users without credentials trying to use gsutil for first
815 time.
816 - Fixed case where chacl command incorrectly recognized an email address as a
817 domain.
818 - Fix setmeta command for S3 objects.
819 - Fixed bug where wildcarded dest URI attempted string op on Key object.
820 - Fixed case where gsutil -q outputted progress output when doing a streaming
821 upload.
822 - Error handling for out of space during downloads.
823 - Include ISO 8601-required "Z" at end of timestamp string for gsutil ls -l,
824 to be spec-compliant.
825 - Removed deprecated setmeta syntax and fixed unicode issues.
826 - Changed update command not to suggest running sudo if running under Cygwin.
827 - Removed references to deprecated gs-discussion forum from gsutil built-in
828 help.
829 - Add literal quotes around CORS config example URL in gsutil setcors help to
830 avoid having example URL turn into an HREF in auto-generated doc.
831
832 Other Changes
833 -------------
834
835 - Added proper setup.py to make gsutil installable via PyPi.
836 - Added warning to gsutil built-in help that delete operations cannot be
837 undone.
838 - Replaced gsutil's OAuth2 client implementation with oauth2client.
839 - Updates to perfdiag.
840 - Updated config help about currently supported settings.
841 - Fixes to setup.py and modified version command.
842 - Move gslib/commands/cred_types.py to gslib, so only Command subclasses live
843 in gslib/commands.
844 - Updated gsutil setmeta help no longer to warn that setmeta with versioning
845 enabled creates a new object.
846
847
848 Release 3.26 (release-date: 2013-03-25)
849 =======================================
850
851 New Features
852 ------------
853
854 - Added support for object composition.
855 - Added support for external service accounts.
856 - Changed gsutil to check for available updates periodically (only while
857 stdin, stderr, stdout are connected to a TTY, so as not to interfere with
858 cron jobs).
859 - Added chdefacl command.
860 - Made gsutil built-in help available under
861 https://developers.google.com/storage/docs/gsutil
862 - Add a command suggestion when the command name is not found.
863 - Added byte suffix parsing to the -s parameter of perfdiag.
864 - Added --help support to subcommands. Fixes #96.
865 - Updated perfdiag command to track availability and record TCP settings.
866 - Added metadata parameter to perfdiag command.
867 - Added support for specifying byte range to cat command.
868 - Output more bucket metadata on ls -Lb.
869 - Implemented gsutil -q (global quiet) option (fixes issue #130). Also changed
870 gsutil to output all progress indicators using logging levels. Also changed
871 help command not to output bold escape sequences and not use PAGER if stdout
872 is not a tty, which also fixes bug that caused gsutil help test to fail.
873 - Plumbed https_validate_certificates through to OAuth2 plugin handler,
874 allowing control over cert validation for OAuth2 requests
875 - Fixed ISO 639.1 ref in config command help text
876
877 Bug Fixes
878 ---------
879
880 - Fixed bug where gsutil cp -D didn't preserve metadata
881 - Fixed problem where gsutil -m is hard to interrupt (partial fix for issue
882 #99 - only for Linux/MacOS; problem still exists for Windows).
883 - Fixed broken reference to boto_lib_dir in update command.
884 - Made changing ACL not retry on 400 error.
885 - Fixed name expansion bug for case where uri_strs is itself an iterator
886 (issue #131); implemented additional naming unit test for this case.
887 - Fixed flaky gsutil rm test
888 - Fixed a bug in the chacl command that made it so you couldn't delete the
889 AllAuthenticatedUsers group from an ACL.
890
891 Other Changes
892 -------------
893
894 - Refactored gsutil main function into gslib, with gsutil being a thin
895 wrapper.
896 - Added a test for the update command.
897 - Renamed gsutil meta_generation params to metageneration, for consistency
898 with GCS docs.
899 - Removed .pyc files from tarball/zipfile.
900 - Added new root certs to cacerts.txt, to provide additional flexibility
901 in the future.
902
903
904 Release 3.25 (release-date: 2013-02-21)
905 =======================================
906
907 Bug Fixes
908 ---------
909
910 - Fixed two version-specific URI bugs:
911 1. gsutil cp -r gs://bucket1 gs://bucket2 would create objects in bucket2
912 with names corresponding to version-specific URIs in bucket1 (e.g.,
913 gs://bucket2/obj#1361417568482000, where the "#1361417568482000" part was
914 part of the object name, not the object's generation).
915
916 This problem similarly caused gsutil cp -r gs://bucket1 ./dir to create
917 files names corresponding to version-specific URIs in bucket1.
918 2. gsutil rm -a gs://bucket/obj would attempt to delete the same object
919 twice, getting a NoSuchKey error on the second attempt.
920
921
922 Release 3.24 (release-date: 2013-02-19)
923 =======================================
924
925 Bug Fixes
926 ---------
927
928 - Fixed bug that caused attempt to dupe-encode a unicode filename.
929
930 Other Changes
931 -------------
932
933 - Refactored retry logic from setmeta and chacl to use @Retry decorator.
934 - Moved @Retry decorator to third_party.
935 - Fixed flaky tests.
936
937
938 Release 3.23 (release-date: 2013-02-16)
939 =======================================
940
941 Bug Fixes
942 ---------
943
944 - Make version-specific URI parsing more robust. This fixes a bug where
945 listing buckets in certain cases would result in the error
946 'BucketStorageUri' object has no attribute 'version_specific_uri'
947
948
949 Release 3.22 (release-date: 2013-02-15)
950 =======================================
951
952 New Features
953 ------------
954
955 - Implemented new chacl command, which makes it easy to add and remove bucket
956 and object ACL grants without having to edit XML (like the older setacl
957 command).
958 - Implemented new "daisy-chain" copying mode, which allows cross-provider
959 copies to run without buffering to local disk, and to use resumable uploads.
960 This copying mode also allows copying between locations and between storage
961 classes, using the new gsutil cp -D option. (Daisy-chain copying is the
962 default when copying between providers, but must be explicitly requested for
963 the other cases to keep costs and performance expectations clear.)
964 - Implemented new perfdiag command to run a diagnostic test against
965 a bucket, collect system information, and report results. Useful
966 when working with Google Cloud Storage team to resolve questions
967 about performance.
968 - Added SIGQUIT (^\) handler, to allow breakpointing a running gsutil.
969
970 Bug Fixes
971 ---------
972
973 - Fixed bug where gsutil setwebcfg signature didn't match with
974 HMAC authentication.
975 - Fixed ASCII codec decode error when constructing tracker filename
976 from non-7bit ASCII input filename.
977 - Changed boto auth plugin framework to allow multiple plugins
978 supporting requested capability, which fixes gsutil exception
979 that used to happen where a GCE user had a service account
980 configured and then ran gsutil config.
981 - Changed Command.Apply method to be resilient to name expansion
982 exceptions. Before this change, if an exception was raised
983 during iteration of NameExpansionResult, the parent process
984 would immediately stop execution, causing the
985 _EOF_NAME_EXPANSION_RESULT to never be sent to child processes.
986 This resulted in the process hanging forever.
987 - Fixed various bugs for gsutil running on Windows:
988 - Fixed various places from a hard-coded '/' to os.sep.
989 - Fixed a bug in the cp command where it was using the destination
990 URI's .delim property instead of the source URI.
991 - Fixed a bug in the cp command's _SrcDstSame function by
992 simplifying it to use os.path.normpath.
993 - Fixed windows bug in tests/util.py _NormalizeURI function.
994 - Fixed ZeroDivisionError sometimes happening during unit tests
995 on Windows.
996
997 - Fixed gsutil rm bug that caused exit status 1 when encountered
998 non-existent URI.
999 - Fixed support for gsutil cp file -.
1000 - Added preconditions and retry logic to setmeta command, to
1001 enforce concurrency control.
1002 - Fixed bug in copying subdirs to subdirs.
1003 - Fixed cases where boto debug_level caused too much or too little
1004 logging:
1005 - resumable and one-shot uploads weren't showing response headers
1006 when connection.debug > 0.
1007 - payload was showing up in debug output when connection.debug
1008 < 4 for streaming uploads.
1009
1010 - Removed XML parsing from setacl. The previous implementation
1011 relied on loose XML handling, which could truncate what it sends
1012 to the service, allowing invalid XML to be specified by the
1013 user. Instead now the ACL XML is passed verbatim and we rely
1014 on server-side schema enforcement.
1015 - Added user-agent header to resumable uploads.
1016 - Fixed reporting bits/s when it was really bytes/s.
1017 - Changed so we now pass headers with API version & project ID
1018 to create_bucket().
1019 - Made "gsutil rm -r gs://bucket/folder" remove xyz_$folder$ object
1020 (which is created by various GUI tools).
1021 - Fixed bug where gsutil binary was shipped with protection 750
1022 instead of 755.
1023
1024 Other Changes
1025 -------------
1026
1027 - Reworked versioned object handling:
1028 - Removed need for commands to specify -v option to parse
1029 versions. Versioned URIs are now uniformly handled by all
1030 commands.
1031 - Refactored StorageUri parsing that had been split across
1032 storage_uri and convenience; made versioned URIs render with
1033 version string so StorageUri is round-trippable (boto change).
1034 - Implemented gsutil cp -v option for printing the version-specific
1035 URI that was just created.
1036 - Added error detail for attempt to delete non-empty versioned
1037 bucket. Also added versioning state to ls -L -b gs://bucket
1038 output.
1039 - Changed URI parsing to use pre-compiled regex's.
1040 - Other bug fixes.
1041
1042 - Rewrote/deepened/improved various parts of built-in help:
1043 - Updated 'gsutil help dev'.
1044 - Fixed help command handling when terminal does not have the
1045 number of rows set.
1046 - Rewrote versioning help.
1047 - Added gsutil help text for common 403 AccountProblem error.
1048 - Added text to 'gsutil help dev' about legal agreement needed
1049 with code submissions.
1050 - Fixed various other typos.
1051 - Updated doc for cp command regarding metadata not being
1052 preserved when copying between providers.
1053 - Fixed gsutil ls command documentation typo for the -L option.
1054 - Added HTTP scheme to doc/examples for gsutil setcors command.
1055 - Changed minimum version in documentation from 2.5 to 2.6 since
1056 gsutil no longer works in Python 2.5.
1057 - Cleaned up/clarify/deepen various other parts of gsutil
1058 built-in documentation.
1059
1060 - Numerous improvements to testing infrastructure:
1061 - Completely refactored infrastructure, allowing deeper testing
1062 and more readable test code, and enabling better debugging
1063 output when tests fail.
1064 - Moved gslib/test_*.py unit tests to gslib/tests module.
1065 - Made all tests (unit and integration, per-command and modules
1066 (like naming) run from single gsutil test command.
1067 - Moved TempDir functions from GsUtilIntegrationTestCase to
1068 GsUtilTestCase.
1069 - Made test runner message show the test function being run.
1070 - Added file path support to ObjectToURI function.
1071 - Disabled the test command if running on Python 2.6 and unittest2
1072 is not available instead of breaking all of gsutil.
1073 - Changed to pass GCS V2 API and project_id from boto config
1074 if necessary in integration_testcase#CreateBucket().
1075 - Fixed unit tests by using a GS-specific mocking class to
1076 override the S3 provider.
1077 - Added friendlier error message if test path munging fails.
1078 - Fixed bug where gsutil test only cleaned up first few test files.
1079 - Implemented setacl integration tests.
1080 - Implemented StorageUri parsing unit tests.
1081 - Implemented test for gsutil cp -D.
1082 - Implemented setacl integration tests.
1083 - Implemented tests for reading and seeking past end of file.
1084 - Implemented and tests for it in new tests module.
1085 - Changed cp tests that don't specify a Content-Type to check
1086 for new binary/octet-stream default instead of server-detected
1087 mime type.
1088
1089 - Changed gsutil mv to allow moving local files/dirs to the cloud.
1090 Previously this was disallowed in the belief we should be
1091 conservative about deleting data from local disk but there are
1092 legitimate use cases for moving data from a local dir to the
1093 cloud, it's clear to the user this would remove data from the
1094 local disk, and allowing it makes the tool behavior more
1095 consistent with what users would expect.
1096 - Changed gsutil update command to insist on is_secure and
1097 https_validate_certificates.
1098 - Fixed release no longer to include extraneous boto dirs in
1099 top-level of gsutil distribution (like bin/ and docs/).
1100 - Changed resumable upload threshold from 1 MB to 2 MB.
1101 - Removed leftover cloudauth and cloudreader dirs. Sample code
1102 now lives at https://github.com/GoogleCloudPlatform.
1103 - Updated copyright notice on code files.
1104
1105
1106 Release 3.21 (release-date: 2012-12-10)
1107 =======================================
1108
1109 New Features
1110 ------------
1111
1112 - Added the ability for the cp command to continue even if there is an
1113 error. This can be activated with the -c flag.
1114 - Added support for specifying src args for gsutil cp on stdin (-I option)
1115
1116 Bug Fixes
1117 ---------
1118
1119 - Fixed gsutil test cp, which assumed it was run from gsutil install dir.
1120 - Mods so we send generation subresource only when user requested
1121 version parsing (-v option for cp and cat commands).
1122
1123 Other Changes
1124 -------------
1125
1126 - Updated docs about using setmeta with versioning enabled.
1127 - Changed GCS endpoint in boto to storage.googleapis.com.
1128
1129
1130 Release 3.20 (release-date: 2012-11-30)
1131 =======================================
1132
1133 New Features
1134 ------------
1135
1136 - Added a noclobber (-n) setting for the cp command. Existing objects/files
1137 will not be overwritten when using this setting.
1138
1139 Bug Fixes
1140 ---------
1141
1142 - Fixed off-by-one error when reporting bytes transferred.
1143
1144 Other Changes
1145 -------------
1146
1147 - Improved versioning support for the remove command.
1148 - Improved test runner support.
1149
1150
1151 Release 3.19 (release-date: 2012-11-26)
1152 =======================================
1153
1154 New Features
1155 ------------
1156
1157 - Added support for object versions.
1158 - Added support for storage classes (including Durable Reduced Availability).
1159
1160 Bug Fixes
1161 ---------
1162 - Fixed problem where cp -q prevented resumable uploads from being performed.
1163 - Made setwebcfg and setcors tests robust wrt XML formatting variation.
1164
1165 Other Changes
1166 -------------
1167
1168 - Incorporated vapier@ mods to make version command not fail if CHECKSUM file
1169 missing.
1170 - Refactored gsutil such that most functionality exists in boto.
1171 - Updated gsutil help dev instructions for how to check out source.
1172
1173
1174 Release 3.18 (release-date: 2012-09-19)
1175 =======================================
1176
1177 Bug Fixes
1178 ---------
1179
1180 - Fixed resumable upload boundary condition when handling POST request
1181 when server already has complete file, which resulted in an infinite
1182 loop that consumed 100% of the CPU.
1183 - Fixed one more place that outputted progress info when gsutil cp -q
1184 specified (during streaming uploads).
1185
1186 Other Changes
1187 -------------
1188
1189 - Updated help text for "gsutil help setmeta" and "gsutil help metadata", to
1190 clarify and deepen parts of the documentation.
1191
1192
1193 Release 3.17 (release-date: 2012-08-17)
1194 =======================================
1195
1196 Bug Fixes
1197 ---------
1198
1199 - Fixed race condition when multiple threads attempt to get an OAuth2 refresh
1200 token concurrently.
1201
1202 Other Changes
1203 -------------
1204
1205 - Implemented simplified syntax for setmeta command. The old syntax still
1206 works but is now deprecated.
1207 - Added help to gsutil cp -z option, to describe how to change where temp
1208 files are written.
1209
1210
1211 Release 3.16 (release-date: 2012-08-13)
1212 =======================================
1213
1214 Bug Fixes
1215 ---------
1216
1217 - Added info to built-in help for setmeta command, to explain the syntax
1218 needed when running from Windows.
1219
1220
1221 Release 3.15 (release-date: 2012-08-12)
1222 =======================================
1223
1224 New Features
1225 ------------
1226
1227 - Implemented gsutil setmeta command.
1228 - Made gsutil understand bucket subdir conventions used by various tools
1229 (like GCS Manager and CloudBerry) so if you cp or mv to a subdir you
1230 created with one of those tools it will work as expected.
1231 - Added support for Windows drive letter-prefaced paths when using Storage
1232 URIs.
1233
1234 Bug Fixes
1235 ---------
1236
1237 - Fixed performance bug when downloading a large object with Content-
1238 Encoding:gzip, where decompression attempted to load the entire object
1239 in memory. Also added "Uncompressing" log output if file is larger than
1240 50M, to make it clear the download hasn't stalled.
1241 - Fixed naming bug when performing gsutil mv from a bucket subdir to
1242 and existing bucket subdir.
1243 - Fixed bug that caused cross-provider copies into Google Cloud Storage to
1244 fail.
1245 - Made change needed to make resumable transfer progress messages not print
1246 when running gsutil cp -q.
1247 - Fixed copy/paste error in config file documentation for
1248 https_validate_certificates option.
1249 - Various typo fixes.
1250
1251 Other Changes
1252 -------------
1253
1254 - Changed gsutil to unset http_proxy environment variable if it's set,
1255 because it confuses boto. (Proxies should instead be configured via the
1256 boto config file.)
1257
1258
1259 Release 3.14 (release-date: 2012-07-28)
1260 =======================================
1261
1262 New Features
1263 ------------
1264
1265 - Added cp -q option, to support quiet operation from cron jobs.
1266 - Made config command restore backed up file if there was a failure or user
1267 hits ^C.
1268
1269 Bug Fixes
1270 ---------
1271
1272 - Fixed bug where gsutil cp -R from a source directory didn't generate
1273 correct destination path.
1274 - Fixed file handle leak in gsutil cp -z
1275 - Fixed bug that caused cp -a option not to work when copying in the cloud.
1276 - Fixed bug that caused '/-' to be appended to object name for streaming
1277 uploads.
1278 - Revert incorrect line I changed in previous CL, that attempted to
1279 get fp from src_key object. The real fix that's needed is described in
1280 https://github.com/GoogleCloudPlatform/gsutil/issues/72.
1281
1282 Other Changes
1283 -------------
1284
1285 - Changed logging to print "Copying..." and Content-Type on same line;
1286 refactored content type and log handling.
1287
1288
1289 Release 3.13 (release-date: 2012-07-19)
1290 =======================================
1291
1292 Bug Fixes
1293 ---------
1294
1295 - Included the fix to make 'gsutil config' honor BOTO_CONFIG environment
1296 variable (which was intended to be included in Release 3.12)
1297
1298
1299 Release 3.11 (release-date: 2012-06-28)
1300 =======================================
1301
1302 New Features
1303 ------------
1304
1305 - Added support for configuring website buckets.
1306
1307 Bug Fixes
1308 ---------
1309
1310 - Fixed bug that caused simultaneous resumable downloads of the same source
1311 object to use the same tracker file.
1312 - Changed language code spec pointer from Wikipedia to loc.gov (for
1313 Content-Language header).
1314
1315
1316 Release 3.10 (release-date: 2012-06-19)
1317 =======================================
1318
1319 New Features
1320 ------------
1321
1322 - Added support for setting and listing Content-Language header.
1323
1324 Bug Fixes
1325 ---------
1326
1327 - Fixed bug that caused getacl/setacl commands to get a character encoding
1328 exception when ACL content contained content not representable in ISO-8859-1
1329 character set.
1330 - Fixed gsutil update not to fail under Windows exclusive file locking.
1331 - Fixed gsutil ls -L to continue past 403 errors.
1332 - Updated gsutil tests and also help dev with instructions on how to run
1333 boto tests, based on recent test refactoring done to in boto library.
1334 - Cleaned up parts of cp help text.
1335
1336
1337 Release 3.9 (release-date: 2012-05-24)
1338 ======================================
1339
1340 Bug Fixes
1341 ---------
1342
1343 - Fixed bug that caused extra "file:/" to be included in pathnames with
1344 doing gsutil cp -R on Windows.
1345
1346
1347 Release 3.8 (release-date: 2012-05-20)
1348 ======================================
1349
1350 Bug Fixes
1351 ---------
1352
1353 - Fixed problem with non-ASCII filename characters not setting encoding before
1354 attempting to hash for generating resumable transfer filename.
1355
1356
1357 Release 3.7 (release-date: 2012-05-11)
1358 ======================================
1359
1360 Bug Fixes
1361 ---------
1362
1363 - Fixed handling of HTTPS tunneling through a proxy.
1364
1365
1366 Release 3.6 (release-date: 2012-05-09)
1367 ======================================
1368
1369 Bug Fixes
1370 ---------
1371
1372 - Fixed bug that caused wildcards spanning directories not to work.
1373 - Fixed bug that gsutil cp -z not to find available tmp space correctly
1374 under Windows.
1375
1376
1377 Release 3.5 (release-date: 2012-04-30)
1378 ======================================
1379
1380 Performance Improvement
1381 -----------------------
1382
1383 - Change by Evan Worley to calculate MD5s incrementally during uploads and
1384 downloads. This reduces overall transfer time substantially for large
1385 objects.
1386
1387 Bug Fixes
1388 ---------
1389
1390 - Fixed bug where uploading and moving multiple files to a bucket subdirectory
1391 didn't work as intended.
1392 (https://github.com/GoogleCloudPlatform/gsutil/issues/92).
1393 - Fixed bug where gsutil cp -r sourcedir didn't copy to specified subdir
1394 if there is only one file in sourcedir.
1395 - Fixed bug where tracker file included a timestamp that caused it not to
1396 be recognized across sessions.
1397 - Fixed bug where gs://bucket/*/dir wildcard matches too many objects.
1398 - Fixed documentation errors in help associated with ACLs and projects.
1399 - Changed GCS ACL parsing to be case-insensitive.
1400 - Changed ls to print error and exit with non-0 status when wildcard matches
1401 nothing, to be more consistent with UNIX shell behavior.
1402
1403
1404 Release 3.4 (release-date: 2012-04-06)
1405 ======================================
1406
1407 Bug Fixes
1408 ---------
1409
1410 - Fixed problem where resumable uploads/downloads of objects with very long
1411 names would generate tracking files with names that exceeded local file
1412 system limits, making it impossible to complete resumable transfers for
1413 those objects. Solution was to build the tracking file name from a fixed
1414 prefix, SHA1 hash of the long filename, epoch timestamp and last 16
1415 chars of the long filename, which is guarantee to be a predictable and
1416 reasonable length.
1417 - Fixed minor bug in output from 'gsutil help dev' which advised executing
1418 an inconsequential test script (test_util.py).
1419
1420
1421 Release 3.3 (release-date: 2012-04-03)
1422 ======================================
1423
1424 Bug Fixes
1425 ---------
1426
1427 - Fixed problem where gsutil ver and debug flags crashed when used
1428 with newly generated boto config files.
1429 - Fixed gsutil bug in windows path handling, and make checksumming work
1430 across platforms.
1431 - Fixed enablelogging to translate -b URI param to plain bucket name in REST
1432 API request.
1433
1434
1435 Release 3.2 (release-date: 2012-03-30)
1436 ======================================
1437
1438 Bug Fixes
1439 ---------
1440
1441 - Fixed problem where gsutil didn't convert between OS-specific directory
1442 separators when copying individually-named files (issue 87).
1443 - Fixed problem where gsutil ls -R didn't work right if there was a key
1444 with a leading path (like /foo/bar/baz)
1445
1446
1447 Release 3.1 (release-date: 2012-03-20)
1448 ======================================
1449
1450 Bug Fixes
1451 ---------
1452
1453 - Removed erroneous setting of Content-Encoding when a gzip file is uploaded
1454 (vs running gsutil cp -z, when Content-Encoding should be set). This
1455 error caused users to get gsutil.tar.gz file uncompressed by the user
1456 agent (like wget) while downloading, making the file appear to be of the
1457 wrong size/content.
1458 - Fixed handling of gsutil help for Windows (previous code depended on
1459 termios and fcntl libs, which are Linux/MacOS-specific).
1460
1461
1462 Release 3.0 (release-date: 2012-03-20)
1463 ======================================
1464
1465 Important Notes
1466 ---------------
1467
1468 - Backwards-incompatible wildcard change:
1469 The '*' wildcard now only matches objects within a bucket directory. If
1470 you have scripts that depend on being able to match spanning multiple
1471 directories you need to use '**' instead. For example, the command:
1472
1473 gsutil cp gs://bucket/*.txt
1474
1475 will now only match .txt files in the top-level directory.
1476
1477 gsutil cp gs://bucket/**.txt
1478
1479 will match across all directories.
1480 - gsutil ls now lists one directory at a time. If you want to list all objects
1481 in a bucket, you can use:
1482
1483 gsutil ls gs://bucket/**
1484
1485 or:
1486
1487 gsutil ls -R gs://bucket
1488
1489 New Features
1490 ------------
1491
1492 - Built-in help for all commands and many additional topics. Try
1493 "gsutil help" for a list of available commands and topics.
1494 - A new hierarchical file tree abstraction layer, which makes the flat bucket
1495 name space look like a hierarchical file tree. This makes several things
1496 possible:
1497 - copying data to/from bucket sub-directories (see “gsutil help cp”).
1498 - distributing large uploads/downloads across many machines
1499 (see “gsutil help cp”)
1500 - renaming bucket sub-directories (see “gsutil help mv”).
1501 - listing individual bucket sub-directories and for listing directories
1502 recursively (see “gsutil help ls”).
1503 - setting ACLs for objects in a sub-directory (see “gsutil help setacl”).
1504
1505 - Support for per-directory (*) and recursive (**) wildcards. Essentially,
1506 ** works the way * did in previous gsutil releases, and * now behaves
1507 consistently with how it works in command interpreters (like bash). The
1508 ability to specify directory-only wildcards also enables a number of use
1509 cases, such as distributing large uploads/downloads by wildcarded name. See
1510 "gsutil help wildcards" for details.
1511 - Support for Cross-Origin Resource Sharing (CORS) configuration. See "gsutil
1512 help cors" for details.
1513 - Support for multi-threading and recursive operation for setacl command
1514 (see “gsutil help setacl”).
1515 - Ability to use the UNIX 'file' command to do content type recognition as
1516 an alternative to filename extensions.
1517 - Introduction of new end-to-end test suite.
1518 - The gsutil version command now computes a checksum of the code, to detect
1519 corruption and local modification when assisting with technical support.
1520 - The gsutil update command is no longer beta/experimental, and now also
1521 supports updating from named URIs (for early/test releases).
1522 - Changed gsutil ls -L to also print Content-Disposition header.
1523
1524 Bug Fixes
1525 ---------
1526
1527 - The gsutil cp -t option previously didn't work as documented, and instead
1528 Content-Type was always detected based on filename extension. Content-Type
1529 detection is now the default, the -t option is deprecated (to be removed in
1530 the future), and specifying a -h Content-Type header now correctly overrides
1531 the filename extension based handling. For details see "gsutil help
1532 metadata".
1533 - Fixed bug that caused multi-threaded mv command not to percolate failures
1534 during the cp phase to the rm phase, which could under some circumstances
1535 cause data that was not copied to be deleted.
1536 - Fixed bug that caused gsutil to use GET for ls -L requests. It now uses HEAD
1537 for ls -L requests, which is more efficient and faster.
1538 - Fixed bug that caused gsutil not to preserve metadata during
1539 copy-in-the-cloud.
1540 - Fixed bug that prevented setacl command from allowing DisplayName's in ACLs.
1541 - Fixed bug that caused gsutil/boto to suppress consecutive slashes in path
1542 names.
1543 - Fixed spec-non-compliant URI construction for resumable uploads.
1544 - Fixed bug that caused rm -f not to work.
1545 - Fixed UnicodeEncodeError that happened when redirecting gsutil ls output
1546 to a file with non-ASCII object names.
1547
1548 Other Changes
1549 -------------
1550
1551 - UserAgent sent in HTTP requests now includes gsutil version number and OS
1552 name.
1553 - Starting with this release users are able to get individual named releases
1554 from version-named objects: gs://pub/gsutil_<version>.tar.gz
1555 and gs://pub/gsutil_<version>.zip. The version-less counterparts
1556 (gs://pub/gsutil.tar.gz and gs://pub/gsutil.zip) will contain the latest
1557 release. Also, the gs://pub bucket is now publicly readable (so, anyone
1558 can list its contents).
1559
1560
1561 Release 2.0 (release-date: 2012-01-13)
1562 ======================================
1563
1564 New Features
1565 ------------
1566
1567 - Support for for two new installation modes: enterprise and RPM.
1568 Customers can now install gsutil one of three ways:
1569 - Individual user mode (previously the only available mode): unpacking from
1570 a gzipped tarball (gs://pub/gsutil.tar.gz) or zip file
1571 (gs://pub/gsutil.zip) and running the gsutil command in place in the
1572 unpacked gsutil directory.
1573 - Enterprise mode (new): unpacking as above, and then running the setup.py
1574 script in the unpacked gsutil directory. This allows a systems
1575 administrator to install gsutil in a central location, using the Python
1576 distutils facility. This mode is supported only on Linux and MacOS.
1577 - RPM mode (new). A RedHat RPM can be built from the gsutil.spec.in file
1578 in the unpacked gsutil directory, allowing it to be installed as part of
1579 a RedHat build.
1580
1581 - Note: v2.0 is the first numbered gsutil release. Previous releases
1582 were given timestamps for versions. Numbered releases enable downstream
1583 package builds (like RPMs) to define dependencies more easily.
1584 This is also the first version where we began including release notes.
OLDNEW
« no previous file with comments | « no previous file | third_party/gsutil/CHECKSUM » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698