Optimize baselines in parallel.

Optimize baselines in parallel.

This is by far the slowest part of doing rebaselines.
We can do everything but the SCM commands in parallel.
Like we do for the other commands, save off the SCM commands
and do them all at the end.

Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=175191

[ojan, 2014-05-30 03:12:55]

This isn't quite ready for review. It needs a bit more testing.

[ojan, 2014-05-30 23:10:25]

Now ready for review.

[Dirk Pranke, 2014-05-30 23:35:11]

this basically looks good. I just have a few minor notes about naming and tests.

I assume optimizing baselines was slow *because* of the scm commands (and not
due to the actual optimization). If so, can you note that in the change
description?

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
File Tools/Scripts/webkitpy/tool/commands/rebaseline.py (right):

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:77: def _add_to_scm(self,
path):
I would probably use different names, like "remember_to_add" or something;
"add_to_scm" sounds like it's actually doing the add, and it isn't.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:390: def _files_to_add(self,
command_results):
since we're now returning the files to delete as well as the files to add, can
you change this name to something else as well?

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:436: # TODO: add test that
shows 'delete' getting populated.
can we actually write this test now, rather than later? it seems unlikely that
someone will come back and write it later (unless you just wanted to do so in a
separate patch you're already working on?)

otherwise, if we have code coverage of everything and pylint isn't complaining,
I'd probably just omit the TODO.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:481: files_to_add,
files_to_delete, lines_to_remove = self._files_to_add(command_results)
see note on the naming of "self._files_to_add()", above.

[ojan, 2014-05-31 00:02:56]

Done. PTAL.

> I assume optimizing baselines was slow *because* of the scm commands (and not
> due to the actual optimization). If so, can you note that in the change
> description?

I didn't actually investigate why it was slow. This was just the easy way to
make it faster and it did indeed seem to.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
File Tools/Scripts/webkitpy/tool/commands/rebaseline.py (right):

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:77: def _add_to_scm(self,
path):
On 2014/05/30 23:35:11, Dirk Pranke wrote:
> I would probably use different names, like "remember_to_add" or something;
> "add_to_scm" sounds like it's actually doing the add, and it isn't.

I went with add_to_scm_later.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:390: def _files_to_add(self,
command_results):
On 2014/05/30 23:35:11, Dirk Pranke wrote:
> since we're now returning the files to delete as well as the files to add, can
> you change this name to something else as well?

I called it _serial_commands.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:436: # TODO: add test that
shows 'delete' getting populated.
On 2014/05/30 23:35:11, Dirk Pranke wrote:
> can we actually write this test now, rather than later? it seems unlikely that
> someone will come back and write it later (unless you just wanted to do so in
a
> separate patch you're already working on?)
> 
> otherwise, if we have code coverage of everything and pylint isn't
complaining,
> I'd probably just omit the TODO.

Whoops. I meant to do this. I use TODO instead of FIXME when I intend to do it
before I upload for review. :)

All I really wanted when I wrote that was the BaselineOptimizer tests I already
added.

https://codereview.chromium.org/308793004/diff/70001/Tools/Scripts/webkitpy/t...
Tools/Scripts/webkitpy/tool/commands/rebaseline.py:481: files_to_add,
files_to_delete, lines_to_remove = self._files_to_add(command_results)
On 2014/05/30 23:35:11, Dirk Pranke wrote:
> see note on the naming of "self._files_to_add()", above.

Done.

[Dirk Pranke, 2014-05-31 00:55:00]

lgtm. thanks!

[ojan, 2014-05-31 01:23:55]

The CQ bit was checked by ojan@chromium.org

[commit-bot: I haz the power, 2014-05-31 01:24:33]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/ojan@chromium.org/308793004/50002

[commit-bot: I haz the power, 2014-05-31 02:45:19]

Change committed as 175191