rebaseline_server: allow users to generate SKP diff patches on a shared instance

rebaseline_server: allow users to generate SKP diff patches on a shared instance

This will allow developers to maintain SKP baselines without ever running their
own rebaseline_server instance!

For now, the developer must manually apply the resulting patchset to his local
Skia checkout to actually modify expectations; in the near future, we hope to
make the UI upload the patchset to Rietveld on the user's behalf.

BUG=skia:1918
NOTRY=true

Committed: https://skia.googlesource.com/skia/+/2c4352bb9db02db70b829f96f397faa80fade220

[epoger, 2014-08-20 04:50:29]

The holy grail... it's finally here, and ready for review at patchset 3.


To try it out:

1. patch the CL into your Skia checkout

2. re-run "gclient sync" to get the latest common DEPS

3. run the server:
  gm/rebaseline_server/server.py --export \
    --builders '.*Mac10.6.*' --boto ~/.boto \
    --truncate --threads 4

4. in a browser, bring up the client UI  ( http://127.0.0.1:8888 )

5. click on one of the "expected vs actuals" links

6. show all resultTypes so you have something to rebaseline

7. select some rows (check the checkboxes in the rightmost column)

8. click the "move XX selected tests to Pending Approval tab" button

9. switch to the Pending Approval tab and click on the "Get a patchfile" button

10. wait a little bit, maybe watching the server-side log

11. get a textbox containing your modified JSON expectation summaries, as a
patchfile!

https://codereview.chromium.org/487853004/diff/1/DEPS
File DEPS (right):

https://codereview.chromium.org/487853004/diff/1/DEPS#newcode6
DEPS:6: "common":
"https://skia.googlesource.com/common.git@5dd2918f96b277ebe04eb66469ec3d5c7ba6952a",
needed to pick up latest changes to git_utils.py

https://codereview.chromium.org/487853004/diff/1/gm/rebaseline_server/compare...
File gm/rebaseline_server/compare_rendered_pictures.py (right):

https://codereview.chromium.org/487853004/diff/1/gm/rebaseline_server/compare...
gm/rebaseline_server/compare_rendered_pictures.py:385: source_json_file: string;
relative path of the JSON file where this
we need this on the server when receiving new image expectations... we need to
know which of the many JSON files on the server these expectations belong in.

https://codereview.chromium.org/487853004/diff/1/gm/rebaseline_server/static/...
File gm/rebaseline_server/static/live-view.html (right):

https://codereview.chromium.org/487853004/diff/1/gm/rebaseline_server/static/...
gm/rebaseline_server/static/live-view.html:205: <textarea rows="8"
cols="50">{{diffResults}}</textarea>
For now we display the patch within this textarea.  It would be fabulous if we
could upload this patch to Rietveld on the user's behalf.

[stephana, 2014-08-20 14:27:28]

I added some initial comments and it looks good overall. I am going to try the
patch and give more feedback as need.

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/res...
File gm/rebaseline_server/results.py (right):

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/res...
gm/rebaseline_server/results.py:231: def read_dicts_from_root(root,
pattern='*.json'):
This is a nit, and not necessary to change, but IMHO this should be factored out
into a utility module and be a standalone function (outside of a class). 

So you would do something like:

import util
...
x = util.read_dicts_from_root(...)

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/ser...
File gm/rebaseline_server/server.py (right):

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/ser...
gm/rebaseline_server/server.py:794: self.wfile.write(diffs)
This is another nit. Moving forward we'll have to replace these kind of
functions with a (simple!) web framework.

I added an issue for this: 
https://code.google.com/p/skia/issues/detail?id=2856

Please add a TODO with my name.

[epoger, 2014-08-20 14:33:57]

Thanks, Stephan... I added these as TODOs for you.

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/res...
File gm/rebaseline_server/results.py (right):

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/res...
gm/rebaseline_server/results.py:231: def read_dicts_from_root(root,
pattern='*.json'):
On 2014/08/20 14:27:28, stephana wrote:
> This is a nit, and not necessary to change, but IMHO this should be factored
out
> into a utility module and be a standalone function (outside of a class). 
> 
> So you would do something like:
> 
> import util
> ...
> x = util.read_dicts_from_root(...)
> 

Sounds like a good idea.  Added a TODO.

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/ser...
File gm/rebaseline_server/server.py (right):

https://codereview.chromium.org/487853004/diff/40001/gm/rebaseline_server/ser...
gm/rebaseline_server/server.py:794: self.wfile.write(diffs)
On 2014/08/20 14:27:28, stephana wrote:
> This is another nit. Moving forward we'll have to replace these kind of
> functions with a (simple!) web framework.
> 
> I added an issue for this: 
> https://code.google.com/p/skia/issues/detail?id=2856
> 
> Please add a TODO with my name. 

Done.

[stephana, 2014-08-20 14:56:20]

Great. Worked as described.

[epoger, 2014-08-20 14:57:42]

On 2014/08/20 14:56:20, stephana wrote:
> Great. Worked as described.

Cool.  Right now I am about to use the tool to commit some SKP expectations
within the "experimental" dir; we'll see how it works.

I'll await the magic 4 letters from you before committing this CL... let me know
if you want to discuss any of it live.

[stephana, 2014-08-20 14:57:50]

On 2014/08/20 14:56:20, stephana wrote:
> Great. Worked as described.

LGTM

[epoger, 2014-08-20 14:59:00]

The CQ bit was checked by epoger@google.com

[epoger, 2014-08-20 14:59:05]

The CQ bit was unchecked by epoger@google.com

[epoger, 2014-08-20 14:59:39]

The CQ bit was checked by epoger@google.com

[commit-bot: I haz the power, 2014-08-20 15:00:18]

CQ is trying da patch. Follow status at
 https://skia-tree-status.appspot.com/cq/epoger@google.com/487853004/60001

[commit-bot: I haz the power, 2014-08-20 15:00:40]

Committed patchset #4 (60001) as 2c4352bb9db02db70b829f96f397faa80fade220