rebaseline.py: try to download images from Google Storage before skia-autogen

rebaseline.py: try to download images from Google Storage before skia-autogen

This will allow us to complete Step 1 of https://goto.google.com/ChecksumTransitionDetail

R=scroggo@google.com

Committed: https://code.google.com/p/skia/source/detail?r=9534

[epoger, 2013-06-12 15:14:22]

https://codereview.chromium.org/16311011/diff/1/tools/tests/run.sh
File tools/tests/run.sh (right):

https://codereview.chromium.org/16311011/diff/1/tools/tests/run.sh#newcode212
tools/tests/run.sh:212: rebaseline_test "--json-base-url
file:$REBASELINE_INPUT/json1 --tests test1 test2 --configs 565 8888 --subdirs
base-android-galaxy-nexus base-shuttle-win7-intel-float"
"$REBASELINE_OUTPUT/subset"
Patchset 1 adds json-base-url specification for all tests, since later on in
this CL we will start trying to read information out of the JSON file even if
the list of tests to rebaseline is specified.

In one test ("subset"), the JSON file will exist, and in the other ("all") it
won't; this will allow us to exercise both code paths.

[epoger, 2013-06-12 15:32:36]

https://codereview.chromium.org/16311011/diff/4001/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16311011/diff/4001/tools/rebaseline.py#newcod...
tools/rebaseline.py:201: files_to_rebaseline.sort()
Patchset 2 sorts the files we will rebaseline, just so the output in --dry-run
mode will be more consistent (in the face of upcoming changes)

https://codereview.chromium.org/16311011/diff/4001/tools/tests/rebaseline/out...
File tools/tests/rebaseline/output/using-json1-add-new/output-expected/stdout
(right):

https://codereview.chromium.org/16311011/diff/4001/tools/tests/rebaseline/out...
tools/tests/rebaseline/output/using-json1-add-new/output-expected/stdout:4: #
... found files_to_rebaseline [u'3x3bitmaprect_565.png',
u'3x3bitmaprect_8888.png', u'imageblur_565.png', u'imageblur_8888.png',
u'shadertext3_8888.png', u'xfermodeimagefilter_pdf.png']
output is reordered, but otherwise unchanged

[epoger, 2013-06-12 15:58:46]

https://codereview.chromium.org/16311011/diff/15001/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16311011/diff/15001/tools/rebaseline.py#newco...
tools/rebaseline.py:172: def _GetActualResults(self, json_url, sections=None):
Patchset 3 extracts a new _GetActualResults() method out of
_GetFilesToRebaseline().  This will help us look up actual-result checksums from
the actual-results.json file later on in the CL.

[epoger, 2013-06-12 17:17:15]

Ready for review at patchset 4...

Leon, I've put you down as official reviewer since you and I have been
discussing the checksum-indexed file downloads lately.  Feel free to reassign if
you think that's better...

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py#newco...
tools/rebaseline.py:287: if not self._DownloadFromGoogleStorage(
Patchset 4 gets to the meat of it: first try to download the actual image result
from Google Storage.  If that fails for any reason, fall back to downloading
from skia-autogen.

https://codereview.chromium.org/16311011/diff/21001/tools/tests/rebaseline/ou...
File tools/tests/rebaseline/output/subset/output-expected/stdout (right):

https://codereview.chromium.org/16311011/diff/21001/tools/tests/rebaseline/ou...
tools/tests/rebaseline/output/subset/output-expected/stdout:5: # unable to find
filename test1_565.png in all_results dict
For this test case, the test names used (test1, test2) are not found in the
actual-results.json files at all (e.g.
https://skia.googlecode.com/svn-history/r9471/trunk/tools/tests/rebaseline/in...
)

So we just fall back to reading them from skia-autogen.

https://codereview.chromium.org/16311011/diff/21001/tools/tests/rebaseline/ou...
File tools/tests/rebaseline/output/using-json1-add-new/output-expected/stdout
(right):

https://codereview.chromium.org/16311011/diff/21001/tools/tests/rebaseline/ou...
tools/tests/rebaseline/output/using-json1-add-new/output-expected/stdout:8: curl
--fail --silent
http://chromium-skia-gm.commondatastorage.googleapis.com/gm/bitmap-64bitMD5/3...
--output base-android-galaxy-nexus/.temp-3x3bitmaprect_565.png
For this test case, we can find the tests in actual-results.json!  So we are
able to construct GoogleStorage URLs based on the bitmap hashes listed in
https://skia.googlecode.com/svn-history/r9471/trunk/tools/tests/rebaseline/in...
.

[scroggo, 2013-06-12 17:40:12]

lgtm

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py#newco...
tools/rebaseline.py:290: if not self._DownloadFromAutogen(
Nit:
I don't see anything against it in the styleguide, but it looks ugly/confusing
to me to have this line line up with the line above it.

[epoger, 2013-06-12 17:43:44]

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16311011/diff/21001/tools/rebaseline.py#newco...
tools/rebaseline.py:290: if not self._DownloadFromAutogen(
On 2013/06/12 17:40:12, scroggo wrote:
> Nit:
> I don't see anything against it in the styleguide, but it looks ugly/confusing
> to me to have this line line up with the line above it.

Agreed.  My space-age IDE (emacs) lined them up this way.  I decided that going
over 80 chars was less bad.

[epoger, 2013-06-12 17:44:18]

Committed patchset #5 manually as r9534 (presubmit successful).