Parallelize telemetry unit tests on Android bots

Parallelize telemetry unit tests on Android bots

BUG=379378
Committed: https://crrev.com/ac997d3fdca4aff32e9c80779c87dfd1e1a8e69b
Cr-Commit-Position: refs/heads/master@{#317621}

[Zhen Wang, 2015-02-05 02:19:25]

zhenw@chromium.org changed reviewers:
+ dtu@chromium.org, nednguyen@google.com

[Zhen Wang, 2015-02-05 02:19:25]

ptal.

If you do not specify "--device=ID" and if you have multiple Android devices
connected, you will use all connected Android devices.

Some initial testing with 2 Android devices connected to local machine:

Before - Nexus 5:
- telemetry_unittests: ~18 min 27 sec
- telemetry_perf_unittests: ~ 7 min 43 sec

After - Nexus 5 + Nexus 7:
- telemetry_unittests: ~ 9 min 55 sec
- telemetry_perf_unittests: ~ 3 min 57 sec

[nednguyen, 2015-02-05 17:55:31]

nednguyen@google.com changed reviewers:
+ dpranke@google.com

[nednguyen, 2015-02-05 17:55:32]

I defer to Dirk & Dave for reviewing this patch.

[dtu, 2015-02-05 23:11:22]

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
File tools/telemetry/telemetry/unittest_util/run_tests.py (right):

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
tools/telemetry/telemetry/unittest_util/run_tests.py:109: runner.args.jobs =
len(android_device.FindAllAvailableDevices(args))
Isn't it len(GetSpecifiedDevices) if --device is specified?

Also, per http://crbug.com/435723, having no --device means local device, not
android.

[Dirk Pranke, 2015-02-06 02:20:55]

dpranke@chromium.org changed reviewers:
+ dpranke@chromium.org

[Dirk Pranke, 2015-02-06 02:20:56]

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
File tools/telemetry/telemetry/unittest_util/run_tests.py (right):

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
tools/telemetry/telemetry/unittest_util/run_tests.py:183: args.device =
android_devices[child.worker_num-1].guid
This part looks at least plausible to me :), and I'd second Dave's question, so
I defer to Dave for the rest of the review.

[Zhen Wang, 2015-02-06 17:54:20]

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
File tools/telemetry/telemetry/unittest_util/run_tests.py (right):

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
tools/telemetry/telemetry/unittest_util/run_tests.py:109: runner.args.jobs =
len(android_device.FindAllAvailableDevices(args))
We need to clarify the --device flag usage. I commented on
http://crbug.com/435723 and will update the CL after we have a conclusion there.

https://codereview.chromium.org/902763002/diff/1/tools/telemetry/telemetry/un...
tools/telemetry/telemetry/unittest_util/run_tests.py:183: args.device =
android_devices[child.worker_num-1].guid
We need to clarify the --device flag usage. I commented on
http://crbug.com/435723 and will update the CL after we have a conclusion there.

[Zhen Wang, 2015-02-09 23:47:19]

review fix according to
https://code.google.com/p/chromium/issues/detail?id=435723#c14

ptal

[Zhen Wang, 2015-02-11 21:35:06]

I also updated some unit tests to make them work with the parallel unit testing.
ptal.

[Zhen Wang, 2015-02-13 16:56:40]

Hi Dave, can you take a look?

[Zhen Wang, 2015-02-17 23:44:18]

Hi Dave, I have updated the config for FYI bot "Android Tests (K N5, user
build)(dbg)". ptal.

Link to the affected bot:
http://build.chromium.org/p/chromium.fyi/builders/Android%20Tests%20(K%20N5%2...

[fmeawad, 2015-02-17 23:50:35]

On 2015/02/17 23:44:18, Zhen Wang wrote:
> Hi Dave, I have updated the config for FYI bot "Android Tests (K N5, user
> build)(dbg)". ptal.
> 
> Link to the affected bot:
>
http://build.chromium.org/p/chromium.fyi/builders/Android%20Tests%20(K%20N5%2...

Not sure if you should mixes telemetry changes to build changes. I don't see the
harm tho.

[dtu, 2015-02-18 20:12:09]

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
File
tools/telemetry/telemetry/core/backends/android_command_line_backend_unittest.py
(right):

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/backends/android_command_line_backend_unittest.py:27:
self._device_serial = options.device
I think I'd prefer to move this into the test itself to avoid creating side
effects on the class.

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/core/device_finder.py (right):

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/device_finder.py:40: def
GetSpecifiedDevices(options):
GetDevicesMatchingOptions maybe?

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/device_finder.py:41: """Returns the specified
devices."""
I want to handle the case of options.device == None inside this function instead
of in the caller.

[Zhen Wang, 2015-02-18 23:50:56]

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
File
tools/telemetry/telemetry/core/backends/android_command_line_backend_unittest.py
(right):

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/backends/android_command_line_backend_unittest.py:27:
self._device_serial = options.device
On 2015/02/18 20:12:09, dtu wrote:
> I think I'd prefer to move this into the test itself to avoid creating side
> effects on the class.

Done.

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
File tools/telemetry/telemetry/core/device_finder.py (right):

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/device_finder.py:40: def
GetSpecifiedDevices(options):
On 2015/02/18 20:12:09, dtu wrote:
> GetDevicesMatchingOptions maybe?

Done.

https://codereview.chromium.org/902763002/diff/80001/tools/telemetry/telemetr...
tools/telemetry/telemetry/core/device_finder.py:41: """Returns the specified
devices."""
On 2015/02/18 20:12:09, dtu wrote:
> I want to handle the case of options.device == None inside this function
instead
> of in the caller.

Done.

[dtu, 2015-02-23 19:17:29]

lgtm

[Zhen Wang, 2015-02-23 19:29:13]

The CQ bit was checked by zhenw@chromium.org

[commit-bot: I haz the power, 2015-02-23 19:30:12]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/902763002/120001

[commit-bot: I haz the power, 2015-02-23 19:34:55]

Committed patchset #7 (id:120001)

[commit-bot: I haz the power, 2015-02-23 19:35:24]

Patchset 7 (id:??) landed as
https://crrev.com/ac997d3fdca4aff32e9c80779c87dfd1e1a8e69b
Cr-Commit-Position: refs/heads/master@{#317621}