Report more detailed error from download_from_google_storage.py when gsutil fails.

Report more detailed error from download_from_google_storage.py when gsutil fails.

BUG=

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=295416

[Nathan Parker, 2015-04-24 17:52:35]

nparker@chromium.org changed reviewers:
+ maruel@chromium.org

[Nathan Parker, 2015-04-24 18:35:17]

Tested with broken auth, with missing file, and then with everything working w/
gclient sync.

[M-A Ruel, 2015-04-24 18:39:21]

maruel@chromium.org changed reviewers:
+ hinoka@chromium.org
- maruel@chromium.org

[M-A Ruel, 2015-04-24 18:39:22]

Forwarding to Ryan.

[Nathan Parker, 2015-05-26 18:13:26]

Ryan -- Can you take a quick look?

[Nathan Parker, 2015-05-26 18:23:35]

Ryan -- Can you take a quick look?

[hinoka, 2015-05-26 18:35:27]

lgtm thanks

Sanity check with tests/download_from_google_storage_unittests.py

[Nathan Parker, 2015-05-26 20:41:45]

On 2015/05/26 18:35:27, hinoka wrote:
> lgtm thanks
> 
> Sanity check with tests/download_from_google_storage_unittests.py

That fails at HEAD for me.  Is it run by a bot somewhere?

[hinoka, 2015-05-26 20:47:27]

Actually 4 of them failed for me too, looks like the test expectations haven't
been updated :(

Go ahead and CQ this (this test isn't included in the CQ, which was probably
what caused the issue to begin with).  I'll update the tests and wedge it into
the CQ separately.

[Nathan Parker, 2015-05-26 21:04:48]

The CQ bit was checked by nparker@chromium.org

[Nathan Parker, 2015-05-26 21:07:59]

I started to fix the test, but ran into a problem I don't understand: It tries
to use --force-version on gsutil, but gsutil has no such option.  Maybe it's my
environment.   You can use the following patch to fix most of the tests.

 -- Nathan

diff --git a/tests/download_from_google_storage_unittests.py
b/tests/download_from_google_storage_unittests.py
index abdca72..469a864 100755
--- a/tests/download_from_google_storage_unittests.py
+++ b/tests/download_from_google_storage_unittests.py
@@ -72,8 +72,8 @@ class GstoolsUnitTests(unittest.TestCase):
     gsutil = download_from_google_storage.Gsutil(GSUTIL_DEFAULT_PATH, None)
     self.assertEqual(gsutil.path, GSUTIL_DEFAULT_PATH)
     code, _, err = gsutil.check_call()
-    self.assertEqual(code, 0)
     self.assertEqual(err, '')
+    self.assertEqual(code, 0)
 
   def test_get_sha1(self):
     lorem_ipsum = os.path.join(self.base_path, 'lorem_ipsum.txt')
@@ -169,12 +169,11 @@ class DownloadTests(unittest.TestCase):
         ('check_call',
             ('ls', input_filename)),
         ('check_call',
-            ('cp', '-q', input_filename, output_filename))]
+            ('cp', input_filename, output_filename))]
     if sys.platform != 'win32':
       expected_calls.append(
           ('check_call',
-           ('ls',
-            '-L',
+           ('stat',
             'gs://sometesturl/7871c8e24da15bad8b0be2c36edc9dc77e37727f')))
     expected_output = [
         '0> Downloading %s...' % output_filename]
@@ -210,7 +209,7 @@ class DownloadTests(unittest.TestCase):
         0, self.queue, False, self.base_url, self.gsutil,
         stdout_queue, self.ret_codes, True)
     expected_output = [
-        '0> File %s for %s does not exist, skipping.' % (
+        '0> Failed to fetch file %s for %s, skipping. [Err: ]' % (
             input_filename, output_filename),
     ]
     expected_calls = [
@@ -218,7 +217,7 @@ class DownloadTests(unittest.TestCase):
             ('ls', input_filename))
     ]
     expected_ret_codes = [
-        (1, 'File %s for %s does not exist.' % (
+        (1, 'Failed to fetch file %s for %s. [Err: ]' % (
             input_filename, output_filename))
     ]
     self.assertEqual(list(stdout_queue.queue), expected_output)
@@ -248,13 +247,12 @@ class DownloadTests(unittest.TestCase):
         ('check_call',
             ('ls', input_filename)),
         ('check_call',
-            ('cp', '-q', input_filename, output_filename))
+            ('cp', input_filename, output_filename))
     ]
     if sys.platform != 'win32':
       expected_calls.append(
           ('check_call',
-           ('ls',
-            '-L',
+           ('stat',
             'gs://sometesturl/7871c8e24da15bad8b0be2c36edc9dc77e37727f')))
     self.assertEqual(self.gsutil.history, expected_calls)
     self.assertEqual(code, 101)
@@ -280,12 +278,11 @@ class DownloadTests(unittest.TestCase):
         ('check_call',
             ('ls', input_filename)),
         ('check_call',
-            ('cp', '-q', input_filename, output_filename))]
+            ('cp', input_filename, output_filename))]
     if sys.platform != 'win32':
       expected_calls.append(
           ('check_call',
-           ('ls',
-            '-L',
+           ('stat',
             'gs://sometesturl/7871c8e24da15bad8b0be2c36edc9dc77e37727f')))
     self.assertEqual(self.gsutil.history, expected_calls)
     self.assertEqual(code, 0)

[commit-bot: I haz the power, 2015-05-26 21:09:06]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1091373004/1

[commit-bot: I haz the power, 2015-05-26 21:12:09]

Committed patchset #1 (id:1) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=295416