rebaseline.py: skip any platforms we don't have actual results for, with warning message

rebaseline.py: skip any platforms we don't have actual results for, with warning message

R=borenet@google.com

Committed: https://code.google.com/p/skia/source/detail?r=10022

[epoger, 2013-07-11 18:08:30]

Ready for review at patchset 1.

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py#newcode112
tools/rebaseline.py:112: raise Exception('unable to read URL %s: %s' %
(filepath, e))
while I was at it, improved error reporting so the caller can see which URL
could not be fetched

[borenet, 2013-07-11 18:11:34]

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py
File tools/rebaseline.py (left):

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py#oldcode66
tools/rebaseline.py:66: 'Test-Android-Xoom-Tegra2-Arm7-Release',
I'm pretty sure we still want this.  The GMs just aren't getting created because
of the crash/reboot problem going on right now.

[epoger, 2013-07-11 18:16:17]

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py
File tools/rebaseline.py (left):

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py#oldcode66
tools/rebaseline.py:66: 'Test-Android-Xoom-Tegra2-Arm7-Release',
On 2013/07/11 18:11:34, borenet wrote:
> I'm pretty sure we still want this.  The GMs just aren't getting created
because
> of the crash/reboot problem going on right now.

The problem is, because we have NO actual results for this platform yet,
rebaseline.py crashes:

$ tools/rebaseline.py --expectations-root $EXPECTATIONS_DIR
unable to read URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-xoom/Test-Andro...:
HTTP Error 404: Not Found
$ echo $?
1

Which is as we want it... if there is a problem, rebaseline.py should fail.

Removing this platform from the list allows the tool to successfully complete.

In my view, until we actually HAVE actual results available for rebaselining on
this platform, we should remove it from the list.

Do you have a counterproposal that will allow rebaselining of other platforms to
continue?

[borenet, 2013-07-11 18:20:08]

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py
File tools/rebaseline.py (left):

https://codereview.chromium.org/19052007/diff/1/tools/rebaseline.py#oldcode66
tools/rebaseline.py:66: 'Test-Android-Xoom-Tegra2-Arm7-Release',
On 2013/07/11 18:16:17, epoger wrote:
> On 2013/07/11 18:11:34, borenet wrote:
> > I'm pretty sure we still want this.  The GMs just aren't getting created
> because
> > of the crash/reboot problem going on right now.
> 
> The problem is, because we have NO actual results for this platform yet,
> rebaseline.py crashes:
> 
> $ tools/rebaseline.py --expectations-root $EXPECTATIONS_DIR
> unable to read URL
>
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-xoom/Test-Andro...:
> HTTP Error 404: Not Found
> $ echo $?
> 1
> 
> Which is as we want it... if there is a problem, rebaseline.py should fail.
> 
> Removing this platform from the list allows the tool to successfully complete.
> 
> In my view, until we actually HAVE actual results available for rebaselining
on
> this platform, we should remove it from the list.
> 
> Do you have a counterproposal that will allow rebaselining of other platforms
to
> continue?

Do we really want it to fail?  My preference would be a warning ("Couldn't find
GM actuals for X"), but ONLY if the user specifically requested that platform. 
Ideally, there shouldn't even be a list here.  We should be able to rebaseline
any *available* platform.  This gets easier if we flatten things out to
"<BuilderName>.json" so that we just have to "svn ls" once and then we know what
platforms are available.

[epoger, 2013-07-11 18:56:30]

On 2013/07/11 18:20:08, borenet wrote:
> Do we really want it to fail?  My preference would be a warning ("Couldn't
find
> GM actuals for X"), but ONLY if the user specifically requested that platform.

> Ideally, there shouldn't even be a list here.  We should be able to rebaseline
> any *available* platform.  This gets easier if we flatten things out to
> "<BuilderName>.json" so that we just have to "svn ls" once and then we know
what
> platforms are available.

Seems fair.  PTAL at patchset 2... I have restored xoom to the list, and here's
the output now:

$ tools/rebaseline.py --expectations-root $EXPECTATIONS_DIR
could not read json_url
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-xoom/Test-Andro...
; skipping this platform.
$ echo $?
0

[borenet, 2013-07-11 19:02:10]

On 2013/07/11 18:56:30, epoger wrote:
> On 2013/07/11 18:20:08, borenet wrote:
> > Do we really want it to fail?  My preference would be a warning ("Couldn't
> find
> > GM actuals for X"), but ONLY if the user specifically requested that
platform.
> 
> > Ideally, there shouldn't even be a list here.  We should be able to
rebaseline
> > any *available* platform.  This gets easier if we flatten things out to
> > "<BuilderName>.json" so that we just have to "svn ls" once and then we know
> what
> > platforms are available.
> 
> Seems fair.  PTAL at patchset 2... I have restored xoom to the list, and
here's
> the output now:
> 
> $ tools/rebaseline.py --expectations-root $EXPECTATIONS_DIR
> could not read json_url
>
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-xoom/Test-Andro...
> ; skipping this platform.
> $ echo $?
> 0

LGTM!

[epoger, 2013-07-11 19:20:32]

Committed patchset #2 manually as r10022 (presubmit successful).