Speed up angle_end2end_tests by running tests in larger batches.

Speed up angle_end2end_tests by running tests in larger batches.

Similarly speed up angle_unittests and angle_perftests.

BUG=501604

[Ken Russell (switch to Gerrit), 2015-06-20 03:36:41]

kbr@chromium.org changed reviewers:
+ phajdan.jr@chromium.org

[Ken Russell (switch to Gerrit), 2015-06-20 03:36:41]

Pawel: I'd like to set this flag for all of the _main.cc harnesses in src/gpu/
without modifying the isolates. Is there a convenient way to do this? base's
test launcher seems to initialize everything from argc/argv which makes it
tedious to interpose on it and add a default --test-launcher-batch-limit
argument.

[M-A Ruel, 2015-06-21 01:05:41]

On 2015/06/20 03:36:41, Ken Russell wrote:
> Pawel: I'd like to set this flag for all of the _main.cc harnesses in src/gpu/
> without modifying the isolates. Is there a convenient way to do this?

Through recipes?

> base's
> test launcher seems to initialize everything from argc/argv which makes it
> tedious to interpose on it and add a default --test-launcher-batch-limit
> argument.

https://codereview.chromium.org/1201583002/diff/1/chrome/angle_end2end_tests....
File chrome/angle_end2end_tests.isolate (right):

https://codereview.chromium.org/1201583002/diff/1/chrome/angle_end2end_tests....
chrome/angle_end2end_tests.isolate:14: '--test-launcher-batch-limit=500',
please align.

[Paweł Hajdan Jr., 2015-06-22 10:53:38]

Why is it needed? How many tests do you have?

You could modify src/gpu/angle_end2end_tests_main.cc which calls
base::LaunchUnitTestsSerially - to customize the default value used for
batch_limit, or even disable batching completely.

[Ken Russell (switch to Gerrit), 2015-06-23 07:24:54]

On 2015/06/22 10:53:38, Paweł Hajdan Jr. wrote:
> Why is it needed? How many tests do you have?

Several hundred. The default batch limit of 10 slows down serial test execution
dramatically compared to running in the raw GTest harness. I haven't had time to
revisit our cutover to base/test/launcher/ and see whether we can actually use
your test launcher's parallel execution.

> You could modify src/gpu/angle_end2end_tests_main.cc which calls
> base::LaunchUnitTestsSerially - to customize the default value used for
> batch_limit, or even disable batching completely.

Is this what you had in mind? I couldn't find a way to disable batching.

[Paweł Hajdan Jr., 2015-06-23 09:25:09]

On 2015/06/23 at 07:24:54, kbr wrote:
> On 2015/06/22 10:53:38, Paweł Hajdan Jr. wrote:
> > Why is it needed? How many tests do you have?
> 
> Several hundred. The default batch limit of 10 slows down serial test
execution dramatically compared to running in the raw GTest harness. I haven't
had time to revisit our cutover to base/test/launcher/ and see whether we can
actually use your test launcher's parallel execution.

Could you give it a try?

If the tests access the actual GPU, I'd expect they can't run in parallel.
However, if they're just testing internals of ANGLE or other things that don't
touch hardware, it should be fine. Or actually, we run browser_tests in parallel
and I'd guess at least some of them use the GPU. Maybe the OS/drivers can handle
this.

> > You could modify src/gpu/angle_end2end_tests_main.cc which calls
> > base::LaunchUnitTestsSerially - to customize the default value used for
> > batch_limit, or even disable batching completely.
> 
> Is this what you had in mind? I couldn't find a way to disable batching.

There's no out of the box way to do this - you'd need to change more files.

I think the way it's implemented in this patch, with injecting a command-line
flag, would be less maintainable than what I suggested in the long run.

Would you like me to write a patch (based on this one) to do this in a different
way? Would you actually like batching to be disabled completely? It's possible.

[Jamie Madill, 2015-06-23 14:29:24]

On 2015/06/23 09:25:09, Paweł Hajdan Jr. wrote:
> If the tests access the actual GPU, I'd expect they can't run in parallel.

To add some context, the angle_unittests don't use GPU hardware, and should
be fine. The end2end_tests are rendering and GL tests, they do use the GPU.
I'm not sure if they would be okay in parallel.

The reason they've slowed down is because they run an initialization harness
that checks all the possible configs for validity, and filters out configs
that don't make sense (e.g. D3D configs on Linux, or ES3 configs on older
hardware).

The dEQP tests have the same slowness problem, because they load tens of
thousands of tests at startup time from a case list file.

[Ken Russell (switch to Gerrit), 2015-06-23 21:28:05]

On 2015/06/23 09:25:09, Paweł Hajdan Jr. wrote:
> On 2015/06/23 at 07:24:54, kbr wrote:
> > On 2015/06/22 10:53:38, Paweł Hajdan Jr. wrote:
> > > Why is it needed? How many tests do you have?
> > 
> > Several hundred. The default batch limit of 10 slows down serial test
> execution dramatically compared to running in the raw GTest harness. I haven't
> had time to revisit our cutover to base/test/launcher/ and see whether we can
> actually use your test launcher's parallel execution.
> 
> Could you give it a try?
> 
> If the tests access the actual GPU, I'd expect they can't run in parallel.
> However, if they're just testing internals of ANGLE or other things that don't
> touch hardware, it should be fine. Or actually, we run browser_tests in
parallel
> and I'd guess at least some of them use the GPU. Maybe the OS/drivers can
handle
> this.
> 
> > > You could modify src/gpu/angle_end2end_tests_main.cc which calls
> > > base::LaunchUnitTestsSerially - to customize the default value used for
> > > batch_limit, or even disable batching completely.
> > 
> > Is this what you had in mind? I couldn't find a way to disable batching.
> 
> There's no out of the box way to do this - you'd need to change more files.
> 
> I think the way it's implemented in this patch, with injecting a command-line
> flag, would be less maintainable than what I suggested in the long run.
> 
> Would you like me to write a patch (based on this one) to do this in a
different
> way? Would you actually like batching to be disabled completely? It's
possible.

Yes, would you please do that? I'd appreciate it since I clearly didn't
understand your suggestion.

[Paweł Hajdan Jr., 2015-06-26 13:39:31]

On 2015/06/23 at 21:28:05, kbr wrote:
> On 2015/06/23 09:25:09, Paweł Hajdan Jr. wrote:
> > Would you like me to write a patch (based on this one) to do this in a
different
> > way? Would you actually like batching to be disabled completely? It's
possible.
> 
> Yes, would you please do that? I'd appreciate it since I clearly didn't
understand your suggestion.

Sure! :) Sent https://codereview.chromium.org/1209243004 .

[Ken Russell (switch to Gerrit), 2015-06-26 15:41:06]

Thank you very much. That looks much nicer. Closing this in favor of your CL.