rebaseline.py: split image-based rebaselining, which will go away soon, into its own script

rebaseline.py: split image-based rebaselining, which will go away soon, into its own script
(part of step 3 in https://goto.google.com/ChecksumTransitionDetail )

also adds new --expectations-root option

R=scroggo@google.com, senorblanco@chromium.org

Committed: https://code.google.com/p/skia/source/detail?r=9689

[epoger, 2013-06-18 17:27:40]

https://codereview.chromium.org/17379004/diff/1/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/17379004/diff/1/tools/rebaseline.py#newcode417
tools/rebaseline.py:417: rebaseliner = Rebaseliner(tests=args.tests,
configs=args.configs,
Patchset 1 moves the subdir loop outside of the Rebaseliner object, for the
reason detailed in the new comment:

# Soon, we will be instantiating different Rebaseliner objects depending
# on whether we are rebaselining an expected-results.json file, or
# individual image files.  Different gm-expected subdirectories may move
# from individual image files to JSON-format expectations at different
# times, so we need to make this determination per subdirectory.

[epoger, 2013-06-18 18:08:52]

https://codereview.chromium.org/17379004/diff/8001/tools/rebaseline_imagefile...
File tools/rebaseline_imagefiles.py (right):

https://codereview.chromium.org/17379004/diff/8001/tools/rebaseline_imagefile...
tools/rebaseline_imagefiles.py:49: class ImageRebaseliner(object):
Patchset 2 copies the Rebaseliner object out into a separate file where it can
die in isolation later this week (hopefully), renamed as ImageRebaseliner.

The original version (now renamed JsonRebaseliner) remains in rebaseline.py,
where we will tweak it to store expectations in a JSON summary rather than
individual image files.

[epoger, 2013-06-18 19:11:38]

https://codereview.chromium.org/17379004/diff/8002/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/17379004/diff/8002/tools/rebaseline.py#newcod...
tools/rebaseline.py:395: parser.add_argument('--expectations-root',
The new --expectations-root flag will help in testing, and also in production
(so the caller doesn't always have to cd into ./gm-expected )

https://codereview.chromium.org/17379004/diff/8002/tools/rebaseline.py#newcod...
tools/rebaseline.py:443: rebaseliner = JsonRebaseliner(
As of patchset 3... if we find the expected-results.json file, we instantiate
the new JsonRebaseliner instead of the old ImageRebaseliner.  (Although the
JsonRebaseliner isn't fully implemented yet.)

https://codereview.chromium.org/17379004/diff/8002/tools/tests/rebaseline/inp...
File
tools/tests/rebaseline/input/json1/base-android-galaxy-nexus/expected-results.json
(right):

https://codereview.chromium.org/17379004/diff/8002/tools/tests/rebaseline/inp...
tools/tests/rebaseline/input/json1/base-android-galaxy-nexus/expected-results.json:2:
"expected-results" : {
Added some JSON expectations files to exercise the code.

https://codereview.chromium.org/17379004/diff/8002/tools/tests/rebaseline/out...
File tools/tests/rebaseline/output/all/output-expected/stdout (right):

https://codereview.chromium.org/17379004/diff/8002/tools/tests/rebaseline/out...
tools/tests/rebaseline/output/all/output-expected/stdout:7: curl --fail --silent
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-galaxy-nexus/Te...
--output ./base-android-galaxy-nexus/.temp-test1_565.png
Now that the caller can specify --expectations-root, and the default is '.', you
can see the paths within the default expectations-root.

https://codereview.chromium.org/17379004/diff/8002/tools/tests/run.sh
File tools/tests/run.sh (right):

https://codereview.chromium.org/17379004/diff/8002/tools/tests/run.sh#newcode216
tools/tests/run.sh:216: rebaseline_test "--json-base-url
file:$REBASELINE_INPUT/json1 --subdirs base-android-galaxy-nexus
base-shuttle-win7-intel-float --expectations-root $REBASELINE_INPUT/json1"
"$REBASELINE_OUTPUT/using-json1-expectations"
Added a new test that points at an --expectations-root containing JSON-format
expectations, which causes an error because JsonRebaseliner isn't implemented
yet.

[epoger, 2013-06-18 19:21:32]

Ready for review at patchset 4.

https://codereview.chromium.org/17379004/diff/12003/tools/tests/rebaseline/ou...
File tools/tests/rebaseline/output/subset/output-expected/command_line (right):

https://codereview.chromium.org/17379004/diff/12003/tools/tests/rebaseline/ou...
tools/tests/rebaseline/output/subset/output-expected/command_line:1: python
tools/rebaseline.py --dry-run --expectations-root fake/expectations/path
--json-base-url file:tools/tests/rebaseline/input/json1 --tests test1 test2
--configs 565 8888 --subdirs base-android-galaxy-nexus
base-shuttle-win7-intel-float
In patchset 4, I modify the --expectations-root in one of the test cases, just
to see how it changes the output....

https://codereview.chromium.org/17379004/diff/12003/tools/tests/rebaseline/ou...
File tools/tests/rebaseline/output/subset/output-expected/stdout (right):

https://codereview.chromium.org/17379004/diff/12003/tools/tests/rebaseline/ou...
tools/tests/rebaseline/output/subset/output-expected/stdout:6: curl --fail
--silent
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-galaxy-nexus/Te...
--output fake/expectations/path/base-android-galaxy-nexus/.temp-test1_565.png
... and yes, the output looks as I would expect.

[scroggo, 2013-06-19 15:11:32]

lgtm

[epoger, 2013-06-19 15:12:44]

On 2013/06/19 15:11:32, scroggo wrote:
> lgtm

Thanks, Leon.  Do any of the other rebaseline-heads (Rob/Brian/Stephen) have any
thoughts on it?

[Stephen White, 2013-06-19 17:23:56]

On 2013/06/19 15:12:44, epoger wrote:
> On 2013/06/19 15:11:32, scroggo wrote:
> > lgtm
> 
> Thanks, Leon.  Do any of the other rebaseline-heads (Rob/Brian/Stephen) have
any
> thoughts on it?

Added some comments to the doc (now that I understand it a bit better), but the
patch LGTM.

[Stephen White, 2013-06-19 17:25:05]

https://codereview.chromium.org/17379004/diff/12003/tools/rebaseline_imagefil...
File tools/rebaseline_imagefiles.py (right):

https://codereview.chromium.org/17379004/diff/12003/tools/rebaseline_imagefil...
tools/rebaseline_imagefiles.py:12: (the "old way", before
https://goto.google.com/ChecksumTransitionDetail ).
Nit: we should probably move this doc to somewhere public, so external
contributors can see it too.

[epoger, 2013-06-19 18:33:00]

https://codereview.chromium.org/17379004/diff/12003/tools/rebaseline_imagefil...
File tools/rebaseline_imagefiles.py (right):

https://codereview.chromium.org/17379004/diff/12003/tools/rebaseline_imagefil...
tools/rebaseline_imagefiles.py:12: (the "old way", before
https://goto.google.com/ChecksumTransitionDetail ).
On 2013/06/19 17:25:05, Stephen White wrote:
> Nit: we should probably move this doc to somewhere public, so external
> contributors can see it too.

I think the transition process is "inside baseball" (I think only a small number
of people care about it, all of whom are Googlers), and this file and other
references to it should disappear within a week or so.  So I'm not going to
bother finding a public place for that doc.

On the other hand, any long-lived documentation about the new checksum system
itself should be publicly readable, yes.

[epoger, 2013-06-19 18:57:07]

Committed patchset #5 manually as r9689 (presubmit successful).