Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(255)

Side by Side Diff: third_party/gsutil/gslib/command.py

Issue 86123002: Adds SSO auth to gsutil (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/tools/depot_tools
Patch Set: Bug fix or -> and Created 7 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « third_party/gsutil/README.chromium ('k') | third_party/gsutil/gslib/util.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2010 Google Inc. All Rights Reserved. 1 # Copyright 2010 Google Inc. All Rights Reserved.
2 # 2 #
3 # Licensed under the Apache License, Version 2.0 (the "License"); 3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License. 4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at 5 # You may obtain a copy of the License at
6 # 6 #
7 # http://www.apache.org/licenses/LICENSE-2.0 7 # http://www.apache.org/licenses/LICENSE-2.0
8 # 8 #
9 # Unless required by applicable law or agreed to in writing, software 9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS, 10 # distributed under the License is distributed on an "AS IS" BASIS,
(...skipping 151 matching lines...) Expand 10 before | Expand all | Expand 10 after
162 """ 162 """
163 test_steps = [] 163 test_steps = []
164 164
165 # Define a convenience property for command name, since it's used many places. 165 # Define a convenience property for command name, since it's used many places.
166 def _GetDefaultCommandName(self): 166 def _GetDefaultCommandName(self):
167 return self.command_spec[COMMAND_NAME] 167 return self.command_spec[COMMAND_NAME]
168 command_name = property(_GetDefaultCommandName) 168 command_name = property(_GetDefaultCommandName)
169 169
170 def __init__(self, command_runner, args, headers, debug, parallel_operations, 170 def __init__(self, command_runner, args, headers, debug, parallel_operations,
171 gsutil_bin_dir, boto_lib_dir, config_file_list, gsutil_ver, 171 gsutil_bin_dir, boto_lib_dir, config_file_list, gsutil_ver,
172 bucket_storage_uri_class, test_method=None): 172 bucket_storage_uri_class, test_method=None,
173 bypass_prodaccess=True):
173 """ 174 """
174 Args: 175 Args:
175 command_runner: CommandRunner (for commands built atop other commands). 176 command_runner: CommandRunner (for commands built atop other commands).
176 args: Command-line args (arg0 = actual arg, not command name ala bash). 177 args: Command-line args (arg0 = actual arg, not command name ala bash).
177 headers: Dictionary containing optional HTTP headers to pass to boto. 178 headers: Dictionary containing optional HTTP headers to pass to boto.
178 debug: Debug level to pass in to boto connection (range 0..3). 179 debug: Debug level to pass in to boto connection (range 0..3).
179 parallel_operations: Should command operations be executed in parallel? 180 parallel_operations: Should command operations be executed in parallel?
180 gsutil_bin_dir: Bin dir from which gsutil is running. 181 gsutil_bin_dir: Bin dir from which gsutil is running.
181 boto_lib_dir: Lib dir where boto runs. 182 boto_lib_dir: Lib dir where boto runs.
182 config_file_list: Config file list returned by _GetBotoConfigFileList(). 183 config_file_list: Config file list returned by _GetBotoConfigFileList().
183 gsutil_ver: Version string of currently running gsutil command. 184 gsutil_ver: Version string of currently running gsutil command.
184 bucket_storage_uri_class: Class to instantiate for cloud StorageUris. 185 bucket_storage_uri_class: Class to instantiate for cloud StorageUris.
185 Settable for testing/mocking. 186 Settable for testing/mocking.
186 test_method: Optional general purpose method for testing purposes. 187 test_method: Optional general purpose method for testing purposes.
187 Application and semantics of this method will vary by 188 Application and semantics of this method will vary by
188 command and test type. 189 command and test type.
190 bypass_prodaccess: Boolean to ignore the existance of prodaccess.
189 191
190 Implementation note: subclasses shouldn't need to define an __init__ 192 Implementation note: subclasses shouldn't need to define an __init__
191 method, and instead depend on the shared initialization that happens 193 method, and instead depend on the shared initialization that happens
192 here. If you do define an __init__ method in a subclass you'll need to 194 here. If you do define an __init__ method in a subclass you'll need to
193 explicitly call super().__init__(). But you're encouraged not to do this, 195 explicitly call super().__init__(). But you're encouraged not to do this,
194 because it will make changing the __init__ interface more painful. 196 because it will make changing the __init__ interface more painful.
195 """ 197 """
196 # Save class values from constructor params. 198 # Save class values from constructor params.
197 self.command_runner = command_runner 199 self.command_runner = command_runner
198 self.args = args 200 self.args = args
199 self.unparsed_args = args 201 self.unparsed_args = args
200 self.headers = headers 202 self.headers = headers
201 self.debug = debug 203 self.debug = debug
202 self.parallel_operations = parallel_operations 204 self.parallel_operations = parallel_operations
203 self.gsutil_bin_dir = gsutil_bin_dir 205 self.gsutil_bin_dir = gsutil_bin_dir
204 self.boto_lib_dir = boto_lib_dir 206 self.boto_lib_dir = boto_lib_dir
205 self.config_file_list = config_file_list 207 self.config_file_list = config_file_list
206 self.gsutil_ver = gsutil_ver 208 self.gsutil_ver = gsutil_ver
207 self.bucket_storage_uri_class = bucket_storage_uri_class 209 self.bucket_storage_uri_class = bucket_storage_uri_class
208 self.test_method = test_method 210 self.test_method = test_method
209 self.exclude_symlinks = False 211 self.exclude_symlinks = False
210 self.recursion_requested = False 212 self.recursion_requested = False
211 self.all_versions = False 213 self.all_versions = False
214 self.bypass_prodaccess = bypass_prodaccess
212 215
213 # Process sub-command instance specifications. 216 # Process sub-command instance specifications.
214 # First, ensure subclass implementation sets all required keys. 217 # First, ensure subclass implementation sets all required keys.
215 for k in self.REQUIRED_SPEC_KEYS: 218 for k in self.REQUIRED_SPEC_KEYS:
216 if k not in self.command_spec or self.command_spec[k] is None: 219 if k not in self.command_spec or self.command_spec[k] is None:
217 raise CommandException('"%s" command implementation is missing %s ' 220 raise CommandException('"%s" command implementation is missing %s '
218 'specification' % (self.command_name, k)) 221 'specification' % (self.command_name, k))
219 # Now override default command_spec with subclass-specified values. 222 # Now override default command_spec with subclass-specified values.
220 tmp = self._default_command_spec 223 tmp = self._default_command_spec
221 tmp.update(self.command_spec) 224 tmp.update(self.command_spec)
(...skipping 114 matching lines...) Expand 10 before | Expand all | Expand 10 after
336 storage_uri = self.UrisAreForSingleProvider(uri_args) 339 storage_uri = self.UrisAreForSingleProvider(uri_args)
337 if not storage_uri: 340 if not storage_uri:
338 raise CommandException('"%s" command spanning providers not allowed.' % 341 raise CommandException('"%s" command spanning providers not allowed.' %
339 self.command_name) 342 self.command_name)
340 343
341 # Determine whether acl_arg names a file containing XML ACL text vs. the 344 # Determine whether acl_arg names a file containing XML ACL text vs. the
342 # string name of a canned ACL. 345 # string name of a canned ACL.
343 if os.path.isfile(acl_arg): 346 if os.path.isfile(acl_arg):
344 acl_file = open(acl_arg, 'r') 347 acl_file = open(acl_arg, 'r')
345 acl_arg = acl_file.read() 348 acl_arg = acl_file.read()
346 349
347 # TODO: Remove this workaround when GCS allows 350 # TODO: Remove this workaround when GCS allows
348 # whitespace in the Permission element on the server-side 351 # whitespace in the Permission element on the server-side
349 acl_arg = re.sub(r'<Permission>\s*(\S+)\s*</Permission>', 352 acl_arg = re.sub(r'<Permission>\s*(\S+)\s*</Permission>',
350 r'<Permission>\1</Permission>', acl_arg) 353 r'<Permission>\1</Permission>', acl_arg)
351 354
352 acl_file.close() 355 acl_file.close()
353 self.canned = False 356 self.canned = False
354 else: 357 else:
355 # No file exists, so expect a canned ACL string. 358 # No file exists, so expect a canned ACL string.
356 canned_acls = storage_uri.canned_acls() 359 canned_acls = storage_uri.canned_acls()
357 if acl_arg not in canned_acls: 360 if acl_arg not in canned_acls:
358 raise CommandException('Invalid canned ACL "%s".' % acl_arg) 361 raise CommandException('Invalid canned ACL "%s".' % acl_arg)
(...skipping 276 matching lines...) Expand 10 before | Expand all | Expand 10 after
635 True if args_to_check contains any provider URIs. 638 True if args_to_check contains any provider URIs.
636 """ 639 """
637 for uri_str in args_to_check: 640 for uri_str in args_to_check:
638 if re.match('^[a-z]+://$', uri_str): 641 if re.match('^[a-z]+://$', uri_str):
639 return True 642 return True
640 return False 643 return False
641 644
642 def _ConfigureNoOpAuthIfNeeded(self): 645 def _ConfigureNoOpAuthIfNeeded(self):
643 """Sets up no-op auth handler if no boto credentials are configured.""" 646 """Sets up no-op auth handler if no boto credentials are configured."""
644 config = boto.config 647 config = boto.config
645 if not util.HasConfiguredCredentials(): 648 if not util.HasConfiguredCredentials(self.bypass_prodaccess):
646 if self.config_file_list: 649 if self.config_file_list:
647 if (config.has_option('Credentials', 'gs_oauth2_refresh_token') 650 if (config.has_option('Credentials', 'gs_oauth2_refresh_token')
648 and not HAVE_OAUTH2): 651 and not HAVE_OAUTH2):
649 raise CommandException( 652 raise CommandException(
650 'Your gsutil is configured with OAuth2 authentication ' 653 'Your gsutil is configured with OAuth2 authentication '
651 'credentials.\nHowever, OAuth2 is only supported when running ' 654 'credentials.\nHowever, OAuth2 is only supported when running '
652 'under Python 2.6 or later\n(unless additional dependencies are ' 655 'under Python 2.6 or later\n(unless additional dependencies are '
653 'installed, see README for details); you are running Python %s.' % 656 'installed, see README for details); you are running Python %s.' %
654 sys.version) 657 sys.version)
655 raise CommandException('You have no storage service credentials in any ' 658 raise CommandException('You have no storage service credentials in any '
(...skipping 57 matching lines...) Expand 10 before | Expand all | Expand 10 after
713 if num_threads > 1: 716 if num_threads > 1:
714 thread_pool.WaitCompletion() 717 thread_pool.WaitCompletion()
715 finally: 718 finally:
716 if num_threads > 1: 719 if num_threads > 1:
717 thread_pool.Shutdown() 720 thread_pool.Shutdown()
718 # If any shared variables (which means we are running in a separate OS 721 # If any shared variables (which means we are running in a separate OS
719 # process), increment value for each shared variable. 722 # process), increment value for each shared variable.
720 if shared_vars: 723 if shared_vars:
721 for (name, var) in shared_vars.items(): 724 for (name, var) in shared_vars.items():
722 var.value += getattr(self, name) 725 var.value += getattr(self, name)
OLDNEW
« no previous file with comments | « third_party/gsutil/README.chromium ('k') | third_party/gsutil/gslib/util.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698