rebaseline.py: if rebaselining all platforms (not listing them explicitly), allow missing platforms

rebaseline.py: if rebaselining all platforms (not listing them explicitly), allow missing platforms

TBR=any-reviewer

Committed: https://code.google.com/p/skia/source/detail?r=9506

[epoger, 2013-06-11 16:15:33]

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py#newcode1
tools/rebaseline.py:1: #!/usr/bin/python
Apologies for the "TBR" commit, but I have a number of small changes which will
interfere with each other's diffs, so this seemed like the cleanest way to go. 
If anyone has concerns I will be happy to create a followup CL.

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py#newcode3
tools/rebaseline.py:3: '''
Before this change:

$ python tools/rebaseline.py --dry-run
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-galaxy-nexus/Te...
...
# ... found files_to_rebaseline []
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-nexus-10/Test-A...
...
# ... found files_to_rebaseline []
[...]
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-angl...
...
Traceback (most recent call last):
  File "tools/rebaseline.py", line 288, in <module>
    rebaseliner.RebaselineAll()
  File "tools/rebaseline.py", line 246, in RebaselineAll
    filenames = self._GetFilesToRebaseline(json_url=json_url)
  File "tools/rebaseline.py", line 156, in _GetFilesToRebaseline
    json_contents = self._GetContentsOfUrl(json_url)
  File "tools/rebaseline.py", line 138, in _GetContentsOfUrl
    return urllib2.urlopen(url).read()
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 406, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found



After this change:

$  python tools/rebaseline.py --dry-run
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-galaxy-nexus/Te...
...
# ... found files_to_rebaseline []
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-android-nexus-10/Test-A...
...
# ... found files_to_rebaseline []
[...]
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-angl...
...
# unable to load JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-angl...
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-dire...
...
# ... found files_to_rebaseline []
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-floa...
...
# ... found files_to_rebaseline []
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle_ubuntu12_ati577...
...
# ... found files_to_rebaseline []

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py#newcode94
tools/rebaseline.py:94: self._missing_json_is_fatal = True
After this change, you still get an exception if the caller specified one or
more subdirs and those could not be found:

$ python tools/rebaseline.py --dry-run --subdirs base-shuttle_ubuntu12_ati5770
base-shuttle-win7-intel-angle
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle_ubuntu12_ati577...
...
# ... found files_to_rebaseline []
# Getting files to rebaseline from JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-angl...
...
Traceback (most recent call last):
  File "tools/rebaseline.py", line 305, in <module>
    rebaseliner.RebaselineAll()
  File "tools/rebaseline.py", line 263, in RebaselineAll
    filenames = self._GetFilesToRebaseline(json_url=json_url)
  File "tools/rebaseline.py", line 169, in _GetFilesToRebaseline
    raise ValueError(message)
ValueError: unable to load JSON summary URL
http://skia-autogen.googlecode.com/svn/gm-actual/base-shuttle-win7-intel-angl...

[epoger, 2013-06-11 16:16:04]

Committed patchset #1 manually as r9506 (presubmit successful).

[Stephen White, 2013-06-11 16:40:47]

LGTM

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py
File tools/rebaseline.py (right):

https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py#newcode140
tools/rebaseline.py:140: return urllib2.urlopen(url).read()
Oooh, I missed the change where we switched to python-native URL fetching. This
is great (something I meant to do for a while).

[epoger, 2013-06-11 16:47:19]

On 2013/06/11 16:40:47, Stephen White wrote:
> LGTM
> 
> https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py
> File tools/rebaseline.py (right):
> 
> https://codereview.chromium.org/16782003/diff/1/tools/rebaseline.py#newcode140
> tools/rebaseline.py:140: return urllib2.urlopen(url).read()
> Oooh, I missed the change where we switched to python-native URL fetching.
This
> is great (something I meant to do for a while).

So far, we only do python-native URL fetching for the JSON file, not the
individual image files. See
https://code.google.com/p/skia/source/browse/trunk/tools/rebaseline.py?r=9507...
:

        # TODO(epoger): Replace calls to "curl"/"mv" (which will only work on
        # Unix) with a Python HTTP library (which should work cross-platform)
        self._Call([ 'curl', '--fail', '--silent', source_url,
                     '--output', temp_filename ])
        self._Call([ 'mv', temp_filename, dest_filename ])