add reformat-json.py script

add reformat-json.py script

R=borenet@google.com

Committed: https://code.google.com/p/skia/source/detail?r=10192

[epoger, 2013-07-18 19:03:31]

I used this to create https://code.google.com/p/skia/source/detail?r=10152
('base-macmini: replace file-based GM expectations with JSON-based ones').  

Any thoughts on whether I should commit it?  I don't foresee needing to run it
again, except maybe when checking in expected-results.json files for new
platforms...

[borenet, 2013-07-18 20:18:06]

On 2013/07/18 19:03:31, epoger wrote:
> I used this to create https://code.google.com/p/skia/source/detail?r=10152
> ('base-macmini: replace file-based GM expectations with JSON-based ones').  
> 
> Any thoughts on whether I should commit it?  I don't foresee needing to run it
> again, except maybe when checking in expected-results.json files for new
> platforms...

The script LGTM.  Wouldn't rebaseline.py take care of new platforms
appropriately?

[epoger, 2013-07-18 20:45:18]

On 2013/07/18 20:18:06, borenet wrote:
> On 2013/07/18 19:03:31, epoger wrote:
> > I used this to create https://code.google.com/p/skia/source/detail?r=10152
> > ('base-macmini: replace file-based GM expectations with JSON-based ones').  
> > 
> > Any thoughts on whether I should commit it?  I don't foresee needing to run
it
> > again, except maybe when checking in expected-results.json files for new
> > platforms...
> 
> The script LGTM.  Wouldn't rebaseline.py take care of new platforms
> appropriately?

No.  Currently, rebaseline.py assumes that the expected-results.json file
already exists for each platform.  When run with --add-new, it can add new TESTS
to an already-existing expected-results.json file, but it cannot add a whole new
platform.

Right now, if I were to add a new platform I would probably copy over the
expected-results.json file from a similar platform and then run rebaseline.py
after a full bot cycle to update any tests that failed.  That should probably
work pretty well.

As far as this particular script goes... I think we should keep it around, in
case we ever need to reformat one of our JSON files.  Maybe if we change the
schema of the JSON file, or some other circumstance...

[borenet, 2013-07-18 20:47:22]

On 2013/07/18 20:45:18, epoger wrote:
> On 2013/07/18 20:18:06, borenet wrote:
> > On 2013/07/18 19:03:31, epoger wrote:
> > > I used this to create https://code.google.com/p/skia/source/detail?r=10152
> > > ('base-macmini: replace file-based GM expectations with JSON-based ones').
 
> > > 
> > > Any thoughts on whether I should commit it?  I don't foresee needing to
run
> it
> > > again, except maybe when checking in expected-results.json files for new
> > > platforms...
> > 
> > The script LGTM.  Wouldn't rebaseline.py take care of new platforms
> > appropriately?
> 
> No.  Currently, rebaseline.py assumes that the expected-results.json file
> already exists for each platform.  When run with --add-new, it can add new
TESTS
> to an already-existing expected-results.json file, but it cannot add a whole
new
> platform.
> 
> Right now, if I were to add a new platform I would probably copy over the
> expected-results.json file from a similar platform and then run rebaseline.py
> after a full bot cycle to update any tests that failed.  That should probably
> work pretty well.
> 

Hmm... I think it might be good to add an "add new platform by copying an
existing platform" option to rebaseline.py so that there's no manual work
needed.

> As far as this particular script goes... I think we should keep it around, in
> case we ever need to reformat one of our JSON files.  Maybe if we change the
> schema of the JSON file, or some other circumstance...

SGTM

[epoger, 2013-07-19 15:45:23]

Committed patchset #2 manually as r10192 (presubmit successful).

[epoger, 2013-07-19 15:52:14]

On 2013/07/18 20:47:22, borenet wrote:
> > Right now, if I were to add a new platform I would probably copy over the
> > expected-results.json file from a similar platform and then run
rebaseline.py
> > after a full bot cycle to update any tests that failed.  That should
probably
> > work pretty well.
> > 
> 
> Hmm... I think it might be good to add an "add new platform by copying an
> existing platform" option to rebaseline.py so that there's no manual work
> needed.

Mmmaybe.  Given that adding a platform is a relatively rare event, I am hesitant
to make rebaseline.py any more confusing to support it.  For now, I have
documented this process suggestion within the comment section of
https://sites.google.com/site/skiadocs/developer-documentation/using-gm-golde...
; if we want to toolify it later, we can do that.

> 
> > As far as this particular script goes... I think we should keep it around,
in
> > case we ever need to reformat one of our JSON files.  Maybe if we change the
> > schema of the JSON file, or some other circumstance...
> 
> SGTM

Committed.