[download_from_google_storage] Don't list ALL objects to check for ACLs

download_from_google_storage.py

 #!/usr/bin/env python
 # Copyright (c) 2012 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 """Download files from Google Storage based on SHA1 sums."""
 
 
 import hashlib
 import optparse
@@ skipping 87 matching lines @@
     if status_code_match:
       return (int(status_code_match.group(1)), out, err)
     if ('You are attempting to access protected data with '
           'no configured credentials.' in err):
       return (403, out, err)
     if 'No such object' in err:
       return (404, out, err)
     return (code, out, err)
 
 
 def check_bucket_permissions(base_url, gsutil):

[hinoka, 2015/02/12 18:03:54]

I'd actually be okay with removing this entirely.  The intention for this was to
early exit if it detects the lack of permissions.

In reality, bucket and object acls are separate, so having bucket permission
doesn't necessarily mean we have object permissions, and vice verca.  Its good
enough to see if "gsutil cp x y" fails later on, and produce a good message on
403.

-  code, _, ls_err = gsutil.check_call('ls', base_url)
+  code, _, ls_err = gsutil.check_call('ls', base_url + '/..')

[Michael Moss, 2015/02/12 16:39:21]

Based on your description of "only ls the bucket itself, and not its contents",
it sounds like 'ls -b' might be the proper solution.

   if code != 0:
     print >> sys.stderr, ls_err
   if code == 403:
     print >> sys.stderr, 'Got error 403 while authenticating to %s.' % base_url
     print >> sys.stderr, 'Try running "download_from_google_storage --config".'
   elif code == 404:
     print >> sys.stderr, '%s not found.' % base_url
   return code
 
 
@@ skipping 341 matching lines @@
       return code
 
   return download_from_google_storage(
       input_filename, base_url, gsutil, options.num_threads, options.directory,
       options.recursive, options.force, options.output, options.ignore_errors,
       options.sha1_file, options.verbose, options.auto_platform)
 
 
 if __name__ == '__main__':
   sys.exit(main(sys.argv))