Implement clustering support in run_test_cases.py.

Implement clustering support in run_test_cases.py.

This change piles up on the 2 previous CLs.

By clustering a few test cases into a single run, it can minimize the one-cost
process startup.

Since run_test_cases.py knows the exact list of test cases it tried, instead of
just giving a shard index, it knows which output to expect so missing test
cases can be retried properly.

Retries are always done individually; they are not clustered.

R=csharp@chromium.org
BUG=


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=187330

[csharp, 2013-03-11 13:33:22]

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py
File run_test_cases.py (right):

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py#newcode686
run_test_cases.py:686: # It needs to be valid utf-8 otherwise it can't be store.
nit: store -> stored

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py#newcode1033
run_test_cases.py:1033: for i in results:
nit: i -> test_case?

https://codereview.chromium.org/12459014/diff/1/tests/gtest_fake/gtest_fake_p...
File tests/gtest_fake/gtest_fake_pass.py (right):

https://codereview.chromium.org/12459014/diff/1/tests/gtest_fake/gtest_fake_p...
tests/gtest_fake/gtest_fake_pass.py:31: parser.add_option('--gtest_print_time',
action='store_true')
Maybe worth creating a gtest_fake_base class to group all the common code (like
the parser setup and handling)

[M-A Ruel, 2013-03-11 17:30:53]

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py
File run_test_cases.py (right):

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py#newcode686
run_test_cases.py:686: # It needs to be valid utf-8 otherwise it can't be store.
On 2013/03/11 13:33:22, csharp wrote:
> nit: store -> stored

Done.

https://codereview.chromium.org/12459014/diff/1/run_test_cases.py#newcode1033
run_test_cases.py:1033: for i in results:
On 2013/03/11 13:33:22, csharp wrote:
> nit: i -> test_case?

Clarified.

https://codereview.chromium.org/12459014/diff/1/tests/gtest_fake/gtest_fake_p...
File tests/gtest_fake/gtest_fake_pass.py (right):

https://codereview.chromium.org/12459014/diff/1/tests/gtest_fake/gtest_fake_p...
tests/gtest_fake/gtest_fake_pass.py:31: parser.add_option('--gtest_print_time',
action='store_true')
On 2013/03/11 13:33:22, csharp wrote:
> Maybe worth creating a gtest_fake_base class to group all the common code
(like
> the parser setup and handling)

I'd prefer not do that until they get >100 lines, which will probably never
happen.

[csharp, 2013-03-11 17:50:29]

lgtm

[M-A Ruel, 2013-03-11 18:43:22]

I'll need to rework map() so I can unit test it; it would be overwhelming to
test this function with an integration test. Will look at this later.

Anyhow, I tested it manually and re-read the code to make sure it does the right
thing; I got confused with need_to_retry using the wrong variable.

You can just compare patchset 2 and 3;
https://codereview.chromium.org/12459014/diff2/5001:11001/run_test_cases.py

https://codereview.chromium.org/12459014/diff/11001/run_test_cases.py
File run_test_cases.py (right):

https://codereview.chromium.org/12459014/diff/11001/run_test_cases.py#newcode716
run_test_cases.py:716: self.decider.got_result(i['returncode'] == 0)
I got caught by the fact that i['returncode'] == None is failure. So I changed
the conditions to be more explicit.

[csharp, 2013-03-11 18:48:39]

still lgtm

[commit-bot: I haz the power, 2013-03-11 18:53:15]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/maruel@chromium.org/12459014/11001

[commit-bot: I haz the power, 2013-03-11 18:54:02]

Change committed as 187330