Creating webkit_layout_tests target.

Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json"
```

BUG=524758
Committed: https://crrev.com/88df4259334db0710fdd3a096b38af7405e87c1c
Cr-Commit-Position: refs/heads/master@{#432140}

[mithro, 2016-10-27 10:21:13]

Patchset #1 (id:1) has been deleted

[mithro, 2016-10-27 11:08:43]

Description was changed from

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

BUG=524758
==========

to

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

BUG=524758
==========

[mithro, 2016-10-27 11:08:43]

tansell@chromium.org changed reviewers:
+ dpranke@chromium.org, maruel@chromium.org, mcgreevy@golang.org

[mithro, 2016-10-27 11:08:56]

Patchset #2 (id:40001) has been deleted

[mithro, 2016-11-01 06:26:41]

tansell@chromium.org changed reviewers:
+ djd@chromium.org, erikchen@chromium.org

[mithro, 2016-11-01 06:26:42]

Hi Dirk,

This change should enable us to start attempting to upload layout tests to
isolate. It is based on your patch from 6 months ago but cleans it up a bit.

I would like to land this *before* isolate is able to actually upload the layout
tests (in a reasonable amount of time) as we now have multiple people who are
working on this and it is helpful to have a baseline for everyone to work from.
(This might be a temporary solution for http://crbug.com/659270 that Erik is
looking into.)

I'm using the name "webkit_layout_tests" rather than "webkit_tests" to try and
be clearer about what this target actually does.

I still might tweak this patch based on the outcome of
https://codereview.chromium.org/2456793003/ (telemetry: Create
telemetry_gtest_wrapper build group.) but wanted to make sure you thought it was
going in the right direction.

Thanks for your review!

Tim 'mithro' Ansell

[M-A Ruel, 2016-11-01 13:37:22]

lgtm, webkit_layout_tests is a good idea.

[Dirk Pranke, 2016-11-01 16:51:00]

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn
File BUILD.gn (right):

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn#newcode922
BUILD.gn:922: "//chrome/test:telemetry_gtest_wrapper",
This should not depend on anything in //chrome. Can we move the telemetry
wrapper to a more generic place (like in //testing)?

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn#newcode960
BUILD.gn:960: "$root_build_dir/resources/inspector/",
This seems like it should actually be a data dependency of something in
//third_party/WebKit/Source/devtools ?

https://codereview.chromium.org/2452313003/diff/60001/testing/buildbot/manage.py
File testing/buildbot/manage.py (right):

https://codereview.chromium.org/2452313003/diff/60001/testing/buildbot/manage...
testing/buildbot/manage.py:201: # isolate is currently to slow for this target.
s/to/too

[mithro, 2016-11-03 04:19:52]

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn
File BUILD.gn (right):

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn#newcode922
BUILD.gn:922: "//chrome/test:telemetry_gtest_wrapper",
On 2016/11/01 16:51:00, Dirk Pranke wrote:
> This should not depend on anything in //chrome. Can we move the telemetry
> wrapper to a more generic place (like in //testing)?

It's unclear to me why all the telemetry targets seem to be in //chrome/test
rather than something else.

However, that is a question for the people maintaining telemetry rather than
this CL.

https://codereview.chromium.org/2452313003/diff/60001/BUILD.gn#newcode960
BUILD.gn:960: "$root_build_dir/resources/inspector/",
On 2016/11/01 16:51:00, Dirk Pranke wrote:
> This seems like it should actually be a data dependency of something in
> //third_party/WebKit/Source/devtools ?

From what I can tell, these resources are actually only needed for the layout
tests which interact with the inspector. devtools/inspector itself gets the
resources via other means.

This means that it should be only a dependency for the layout tests, right?

https://codereview.chromium.org/2452313003/diff/60001/testing/buildbot/manage.py
File testing/buildbot/manage.py (right):

https://codereview.chromium.org/2452313003/diff/60001/testing/buildbot/manage...
testing/buildbot/manage.py:201: # isolate is currently to slow for this target.
On 2016/11/01 16:51:00, Dirk Pranke wrote:
> s/to/too

Done.

[mithro, 2016-11-03 05:31:05]

Description was changed from

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

BUG=524758
==========

to

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

[mithro, 2016-11-03 05:39:36]

Description was changed from

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

to

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

[Dirk Pranke, 2016-11-03 18:41:11]

lgtm, thanks.

[M-A Ruel, 2016-11-03 18:55:45]

The CQ bit was checked by maruel@chromium.org

[M-A Ruel, 2016-11-03 18:55:45]

The patchset sent to the CQ was uploaded after l-g-t-m from maruel@chromium.org
Link to the patchset: https://codereview.chromium.org/2452313003/#ps100001
(title: "More updates.")

[commit-bot: I haz the power, 2016-11-03 18:56:14]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[M-A Ruel, 2016-11-03 19:00:41]

On 2016/11/03 18:56:14, commit-bot: I haz the power wrote:
> CQ is trying da patch. Follow status at
>  
>
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...


https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.linux...

---
chromium.linux.json: Linux Tests /
browser_side_navigation_components_browsertests is listed multiple times.
---

[commit-bot: I haz the power, 2016-11-03 19:06:45]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-11-03 19:06:46]

Try jobs failed on following builders:
  chromium_presubmit on master.tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[mithro, 2016-11-04 02:29:21]

The CQ bit was checked by tansell@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-11-04 02:30:03]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-11-04 05:46:19]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-11-04 05:46:20]

Dry run: This issue passed the CQ dry run.

[mithro, 2016-11-11 04:36:46]

The CQ bit was checked by tansell@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-11-11 04:36:55]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-11-11 05:27:00]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-11-11 05:27:01]

Dry run: Try jobs failed on following builders:
  win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_...)

[mithro, 2016-11-14 02:45:58]

tansell@chromium.org changed reviewers:
+ ddoman@chromium.org

[mithro, 2016-11-14 02:45:59]

I'm going to land this patch now.

With mine and djd's changes (to go isolate) you can run the webkit layout tests!
I've tested this on the dev swarming cluster, see
https://chromium-swarm-dev.appspot.com/task?id=326aa397db42bb10&refresh=10&sh...

Next steps are;
 * Figure out the issue in https://codereview.chromium.org/2474003002/
 * Land djd's update to the isolate.
 * Combining this with Jeff Carpenter's random ordering fixes.

We are rapidly getting close to actually having this work!

Tim 'mithro' Ansell

[mithro, 2016-11-15 04:50:50]

The CQ bit was checked by tansell@chromium.org

[mithro, 2016-11-15 04:50:50]

The patchset sent to the CQ was uploaded after l-g-t-m from
dpranke@chromium.org, maruel@chromium.org
Link to the patchset: https://codereview.chromium.org/2452313003/#ps140001
(title: "Rebase onto master")

[commit-bot: I haz the power, 2016-11-15 05:01:39]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-11-15 07:23:25]

Description was changed from

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

to

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

[commit-bot: I haz the power, 2016-11-15 07:23:26]

Committed patchset #6 (id:140001)

[commit-bot: I haz the power, 2016-11-15 07:25:36]

Description was changed from

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json" 
```

BUG=524758
==========

to

==========
Creating webkit_layout_tests target.

This provides a webkit_layout_tests target which can be isolated and run on
swarming.

Doing this however is currently unusably slow.

Use with the following;
```bash
gn gen out/Release
gn args out/Release
# Put "is_debug = false" into the editor
ninja -C out/Release webkit_layout_tests

python tools/mb/mb.py isolate //out/Release webkit_layout_tests

# Obsolete Python tool
# python tools/swarming_client/isolate.py archive \
#  -s out/Release/webkit_layout_tests.isolated \
#  -I isolateserver.appspot.com

# Current supported Go tool
tools/luci-go/linux64/isolate archive \
 -isolate=out/Release/webkit_layout_tests.isolate \
 -isolated=out/Release/webkit_layout_tests.isolated \
 -isolate-server=isolateserver.appspot.com

python tools/swarming_client/swarming.py run \
  -v \
  -d pool Chrome \
  -d os Ubuntu-14.04 \
  -d gpu none \
  -S https://chromium-swarm.appspot.com \
  -I https://isolateserver.appspot.com \
  out/Release/webkit_layout_tests.isolated \
  -- \
  --isolated-script-test-output "\${ISOLATED_OUTDIR}/out.json"
```

BUG=524758

Committed: https://crrev.com/88df4259334db0710fdd3a096b38af7405e87c1c
Cr-Commit-Position: refs/heads/master@{#432140}
==========

[commit-bot: I haz the power, 2016-11-15 07:25:37]

Patchset 6 (id:??) landed as
https://crrev.com/88df4259334db0710fdd3a096b38af7405e87c1c
Cr-Commit-Position: refs/heads/master@{#432140}