Switch webgl2_conformance_tests to new harness, and shard them.

Switch webgl2_conformance_tests to new harness, and shard them.

BUG=352807
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_optional_gpu_tests_rel;tryserver.chromium.mac:mac_optional_gpu_tests_rel;tryserver.chromium.win:win_optional_gpu_tests_rel

Committed: https://crrev.com/48321ca0bcdb5af989517d61e2f927501271faa0
Cr-Commit-Position: refs/heads/master@{#401531}

[Ken Russell (switch to Gerrit), 2016-06-16 01:07:40]

Description was changed from

==========
Switch webgl2_conformance_tests to new harness, and shard them.

BUG=352807
==========

to

==========
Switch webgl2_conformance_tests to new harness, and shard them.

BUG=352807
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_optional_gpu_tests_rel;tryserver.chromium.mac:mac_optional_gpu_tests_rel;tryserver.chromium.win:win_optional_gpu_tests_rel
==========

[Ken Russell (switch to Gerrit), 2016-06-16 01:10:07]

kbr@chromium.org changed reviewers:
+ dpranke@chromium.org, zmo@chromium.org

[Ken Russell (switch to Gerrit), 2016-06-16 01:10:07]

Mo, Dirk: please review. Thanks.

[Zhenyao Mo, 2016-06-16 02:54:02]

lgtm
This is awesome

[Dirk Pranke, 2016-06-16 18:41:04]

lgtm

[Ken Russell (switch to Gerrit), 2016-06-16 20:01:47]

It turns out there is a problem in the recipe code which is preventing these
tests from being sharded properly. I'll retry this job after fixing that.

[Ken Russell (switch to Gerrit), 2016-06-17 02:57:00]

The sharding is working. The shards aren't well balanced yet. This needs to be
fixed in the tests themselves. CQ'ing so that we can make more progress on them.

[Ken Russell (switch to Gerrit), 2016-06-17 02:57:03]

The CQ bit was checked by kbr@chromium.org

[Ken Russell (switch to Gerrit), 2016-06-17 02:57:03]

The patchset sent to the CQ was uploaded after l-g-t-m from zmo@chromium.org
Link to the patchset: https://codereview.chromium.org/2077443002/#ps40001
(title: "Added telemetry_gpu_integration_test_run GYP target.")

[commit-bot: I haz the power, 2016-06-17 02:57:40]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2077443002/40001

[Ken Russell (switch to Gerrit), 2016-06-17 03:44:13]

The CQ bit was unchecked by kbr@chromium.org

[Ken Russell (switch to Gerrit), 2016-06-17 04:03:51]

On 2016/06/17 02:57:40, commit-bot: I haz the power wrote:
> CQ is trying da patch. Follow status at
>  https://chromium-cq-status.appspot.com/patch-status/2077443002/40001

I unchecked the CQ box due to the test failures.
https://bugs.chromium.org/p/chromium/issues/detail?id=352807#c35 has some
comments about the failure modes.

[Ken Russell (switch to Gerrit), 2016-06-22 22:16:54]

Dirk, could Ned and I be per-file OWNERS of the Telemetry based scripts in
src/testing/scripts?

[Dirk Pranke, 2016-06-22 22:33:32]

lgtm w/ nits (including OWNERS change).

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
File content/test/gpu/generate_buildbot_json.py (right):

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1125: else 'telemetry_gpu_test'),
nit: shouldn't this be indented 4 spaces?

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1128:
('telemetry_gpu_integration_test_run' if use_gpu_integration_test_harness
ditto ...

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1173: generate_telemetry_tests(name,
config, TELEMETRY_TESTS, is_fyi, False) + \
ditto ...

[Ken Russell (switch to Gerrit), 2016-06-22 23:01:04]

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
File content/test/gpu/generate_buildbot_json.py (right):

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1125: else 'telemetry_gpu_test'),
On 2016/06/22 22:33:32, Dirk Pranke wrote:
> nit: shouldn't this be indented 4 spaces?

I don't know. I'm using emacs' default Python indentation rules.
https://www.chromium.org/developers/coding-style#TOC-Python indicates that 2
spaces is the indentation in Chromium. I spent ~5 minutes searching and didn't
find an easy way to get emacs to indent continuation lines by 4 spaces but leave
the rest 2 spaces, so I'm leaving this as is.

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1128:
('telemetry_gpu_integration_test_run' if use_gpu_integration_test_harness
On 2016/06/22 22:33:32, Dirk Pranke wrote:
> ditto ...

Same here.

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1173: generate_telemetry_tests(name,
config, TELEMETRY_TESTS, is_fyi, False) + \
On 2016/06/22 22:33:32, Dirk Pranke wrote:
> ditto ...

Same here.

[Ken Russell (switch to Gerrit), 2016-06-22 23:01:09]

The CQ bit was checked by kbr@chromium.org

[Ken Russell (switch to Gerrit), 2016-06-22 23:01:10]

The patchset sent to the CQ was uploaded after l-g-t-m from zmo@chromium.org
Link to the patchset: https://codereview.chromium.org/2077443002/#ps80001
(title: "Fix bug in harness. Mark flaky test. Add Ned and me as per-file
OWNERS.")

[commit-bot: I haz the power, 2016-06-22 23:02:24]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2077443002/80001

[Ken Russell (switch to Gerrit), 2016-06-22 23:20:55]

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
File content/test/gpu/generate_buildbot_json.py (right):

https://codereview.chromium.org/2077443002/diff/80001/content/test/gpu/genera...
content/test/gpu/generate_buildbot_json.py:1125: else 'telemetry_gpu_test'),
On 2016/06/22 23:01:04, Ken Russell wrote:
> On 2016/06/22 22:33:32, Dirk Pranke wrote:
> > nit: shouldn't this be indented 4 spaces?
> 
> I don't know. I'm using emacs' default Python indentation rules.
> https://www.chromium.org/developers/coding-style#TOC-Python indicates that 2
> spaces is the indentation in Chromium. I spent ~5 minutes searching and didn't
> find an easy way to get emacs to indent continuation lines by 4 spaces but
leave
> the rest 2 spaces, so I'm leaving this as is.

P.S. Ned suggested adding (require 'google) to my .emacs, which I did, but it
didn't change the indentation behavior here.

[commit-bot: I haz the power, 2016-06-23 00:00:46]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-23 00:00:47]

Try jobs failed on following builders:
  win_optional_gpu_tests_rel on tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_optional_gpu_...)

[Ken Russell (switch to Gerrit), 2016-06-23 01:23:35]

The CQ bit was checked by kbr@chromium.org

[Ken Russell (switch to Gerrit), 2016-06-23 01:23:36]

The patchset sent to the CQ was uploaded after l-g-t-m from
dpranke@chromium.org, zmo@chromium.org
Link to the patchset: https://codereview.chromium.org/2077443002/#ps100001
(title: "Re-fetch current tab after browser restart.")

[commit-bot: I haz the power, 2016-06-23 01:24:03]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2077443002/100001

[commit-bot: I haz the power, 2016-06-23 03:24:26]

Committed patchset #6 (id:100001)

[commit-bot: I haz the power, 2016-06-23 03:27:36]

Description was changed from

==========
Switch webgl2_conformance_tests to new harness, and shard them.

BUG=352807
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_optional_gpu_tests_rel;tryserver.chromium.mac:mac_optional_gpu_tests_rel;tryserver.chromium.win:win_optional_gpu_tests_rel
==========

to

==========
Switch webgl2_conformance_tests to new harness, and shard them.

BUG=352807
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_optional_gpu_tests_rel;tryserver.chromium.mac:mac_optional_gpu_tests_rel;tryserver.chromium.win:win_optional_gpu_tests_rel

Committed: https://crrev.com/48321ca0bcdb5af989517d61e2f927501271faa0
Cr-Commit-Position: refs/heads/master@{#401531}
==========

[commit-bot: I haz the power, 2016-06-23 03:27:37]

Patchset 6 (id:??) landed as
https://crrev.com/48321ca0bcdb5af989517d61e2f927501271faa0
Cr-Commit-Position: refs/heads/master@{#401531}

[qiankun, 2016-06-23 14:04:09]

qiankun.miao@intel.com changed reviewers:
+ qiankun.miao@intel.com

[qiankun, 2016-06-23 14:04:10]

Hi Ken,
I found sharding didn't work sometimes. Do you know is it expected or a bug?
Please take a look at the log:
https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_optional_gpu....

[Ken Russell (switch to Gerrit), 2016-06-23 17:59:41]

On 2016/06/23 14:04:10, qiankun wrote:
> Hi Ken,
> I found sharding didn't work sometimes. Do you know is it expected or a bug?
> Please take a look at the log:
>
https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_optional_gpu....

Hi Qiankun,

Thanks for pointing this out. There was a problem with the updates to
chromium.gpu.fyi.json, being fixed in
https://codereview.chromium.org/2090143003/ .