Add verification to downloaded files

Add verification to downloaded files

This does one last check to see if the file downloaded by
download_from_google_storage.py actually matches its sha1

BUG=

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=296155

[Ryan Tseng, 2015-07-29 20:34:29]

hinoka@google.com changed reviewers:
+ hinoka@google.com, iannucci@chromium.org, vadimsh@chromium.org

[Ryan Tseng, 2015-07-29 20:34:30]

I'm kind of embarrassed this wasn't in there before.  Ptal

New test passes, some old ones are still broken but they're unrelated.

[Vadim Sh., 2015-07-29 20:43:49]

https://codereview.chromium.org/1252313005/diff/1/download_from_google_storag...
File download_from_google_storage.py (right):

https://codereview.chromium.org/1252313005/diff/1/download_from_google_storag...
download_from_google_storage.py:229: ret_codes.put((code, err))
is it ok that there's no "continue" after this line? IIUC, its a failure to
download the file.

[Ryan Tseng, 2015-07-29 20:51:24]

Also open question: Should the file be delete if its known to be corrupted.

Pros: Easy forensics, can easily tell what exactly got downloaded
Cons: File could be a hairy virus

https://codereview.chromium.org/1252313005/diff/1/download_from_google_storag...
File download_from_google_storage.py (right):

https://codereview.chromium.org/1252313005/diff/1/download_from_google_storag...
download_from_google_storage.py:229: ret_codes.put((code, err))
On 2015/07/29 20:43:49, Vadim Sh. wrote:
> is it ok that there's no "continue" after this line? IIUC, its a failure to
> download the file.

Good catch there should be a continue here.  I think the lack thereof is due to
this being the final point in the loop before the executable bit stuff was
added.

[Vadim Sh., 2015-07-29 20:54:46]

lgtm

On 2015/07/29 20:51:24, Ryan Tseng wrote:
> Also open question: Should the file be delete if its known to be corrupted.

IMHO, yes. But not by "download_from_google_storage.py". Imagine there's a
single machine that has corrupted disk. Suddenly the hash doesn't match => it
removes the file => all other 1000 machines fail.

[Ryan Tseng, 2015-07-29 20:56:08]

The CQ bit was checked by hinoka@google.com

[commit-bot: I haz the power, 2015-07-29 20:56:25]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1252313005/20001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1252313005/20001

[commit-bot: I haz the power, 2015-07-29 21:01:18]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-07-29 21:01:18]

Try jobs failed on following builders:
  depot_tools_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/depot_tools_pre...)

[Ryan Tseng, 2015-07-29 21:09:19]

The CQ bit was checked by hinoka@google.com

[Ryan Tseng, 2015-07-29 21:09:20]

The patchset sent to the CQ was uploaded after l-g-t-m from vadimsh@chromium.org
Link to the patchset: https://codereview.chromium.org/1252313005/#ps40001
(title: "lint")

[commit-bot: I haz the power, 2015-07-29 21:09:38]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1252313005/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1252313005/40001

[commit-bot: I haz the power, 2015-07-29 21:11:39]

Committed patchset #3 (id:40001) as
http://src.chromium.org/viewvc/chrome?view=rev&revision=296155