[test262] Use standalone Test262 harness project

[test262] Use standalone Test262 harness project

TC-39 recently decided to remove the Python-based testing harness from the
Test262 project [1]. The code has been duplicated in a standalone project;
update V8's dependencies to fetch from that new location. This is based on
an earlier patch by Mike Pennisi.

[1] https://github.com/tc39/tc39-notes/blob/2b9722db9b90011d6083a5f1c8ff1559cbe01c0b/es7/2016-05/may-25.md

BUG=v8:5078
Committed: https://crrev.com/9f45c188ddfb9b68878e525a925e2d532df76833
Cr-Commit-Position: refs/heads/master@{#37985}

[Dan Ehrenberg, 2016-07-07 23:41:53]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-07 23:42:03]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-07 23:44:33]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-07 23:44:36]

Dry run: Try jobs failed on following builders:
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8646)

[Dan Ehrenberg, 2016-07-08 19:19:57]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-08 19:20:02]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-08 19:22:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-08 19:22:53]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8759)
  v8_linux_arm64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng/build...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8697)

[Dan Ehrenberg, 2016-07-08 19:28:39]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-08 19:28:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-08 19:42:11]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-08 19:42:12]

Dry run: Try jobs failed on following builders:
  v8_linux_nodcheck_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng/bu...)
  v8_linux_nodcheck_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng_tr...)

[Dan Ehrenberg, 2016-07-11 23:18:20]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-11 23:18:29]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-07-11 23:19:20]

Patchset #4 (id:60001) has been deleted

[commit-bot: I haz the power, 2016-07-11 23:19:59]

Dry run: Try jobs failed on following builders:
  v8_android_arm_compile_rel on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_android_arm_compile_rel/...)
  v8_linux64_avx2_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_avx2_rel_ng/buil...)
  v8_linux64_gyp_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_gyp_rel_ng/build...)
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8880)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/8849)
  v8_linux_mips64el_compile_rel on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_mips64el_compile_r...)
  v8_linux_mipsel_compile_rel on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_mipsel_compile_rel...)
  v8_linux_nodcheck_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng/bu...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8820)
  v8_presubmit on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_presubmit/builds/19186)
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[commit-bot: I haz the power, 2016-07-11 23:23:39]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-11 23:23:40]

Dry run: Try jobs failed on following builders:
  v8_linux64_asan_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_asan_rel_ng/buil...)
  v8_linux64_asan_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_asan_rel_ng_trig...)
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8881)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)
  v8_linux_arm64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng/build...)
  v8_linux_arm64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng_trigg...)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_arm_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng_trigger...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/8850)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)
  v8_linux_nodcheck_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng/bu...)
  v8_linux_nodcheck_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng_tr...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8821)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Dan Ehrenberg, 2016-07-11 23:49:35]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-11 23:49:42]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-11 23:53:51]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-11 23:53:52]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8884)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)
  v8_linux_arm64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng/build...)
  v8_linux_arm64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng_trigg...)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_arm_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng_trigger...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/8853)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8824)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Dan Ehrenberg, 2016-07-12 00:16:07]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-12 00:16:09]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-12 00:19:39]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-12 00:19:40]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8886)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)

[Dan Ehrenberg, 2016-07-12 00:28:58]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-12 00:28:59]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-12 00:32:44]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-12 00:32:45]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/8889)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/8858)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/8829)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Dan Ehrenberg, 2016-07-19 23:43:17]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-19 23:43:27]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-19 23:47:20]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-19 23:47:21]

Dry run: Try jobs failed on following builders:
  v8_linux_arm64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng/build...)
  v8_linux_arm64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng_trigg...)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_arm_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng_trigger...)
  v8_linux_nodcheck_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng/bu...)
  v8_linux_nodcheck_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_nodcheck_rel_ng_tr...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/9298)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Dan Ehrenberg, 2016-07-20 00:55:45]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-20 00:55:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-20 00:59:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-20 00:59:27]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/9358)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_arm_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng_trigger...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/9329)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/9299)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Dan Ehrenberg, 2016-07-20 01:11:20]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-20 01:11:23]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-07-20 01:28:29]

littledan@chromium.org changed reviewers:
+ machenbach@chromium.org, tandrii@chromium.org

[Dan Ehrenberg, 2016-07-20 01:28:30]

Finally got this to work. Andrii, could you take a look as Michael is OOO?

[commit-bot: I haz the power, 2016-07-20 01:34:49]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-20 01:34:50]

Dry run: This issue passed the CQ dry run.

[tandrii(chromium), 2016-07-20 10:42:02]

https://codereview.chromium.org/2131743002/diff/180001/test/test262/archive.py
File test/test262/archive.py (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/archive.p...
test/test262/archive.py:22: if tar_info.name.startswith(os.path.join('data',
'.git')) or \
nit: wrap the whole condition with () and don't use \

https://codereview.chromium.org/2131743002/diff/180001/test/test262/list.py
File test/test262/list.py (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/list.py#n...
test/test262/list.py:11: for parent in ["data", "harness"]:
s/"/'

https://codereview.chromium.org/2131743002/diff/180001/test/test262/test262.gyp
File test/test262/test262.gyp (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/test262.g...
test/test262/test262.gyp:26: 'outputs': ['data.tar', 'harness.tar'],
Michael has to approve this, bcz:

1. this does increase the size of things to be downloaded on bot,   do you know
by how much? Is untarring done always, or only on bots that run harness?
2. Is there any reason to ship it separately in harness.tar and not update
existing data.tar?
3. You probably have to prepare test set (~ a bunch of isolate hashes) for
Michael's end-to-end test.

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.py
File test/test262/testcfg.py (left):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.p...
test/test262/testcfg.py:155: "gclient sync for test262")
I would keep user friendly "ImportError", unless you are certain it never
happens (and I am not)

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.py
File test/test262/testcfg.py (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.p...
test/test262/testcfg.py:48: from harness import parseTestRecord
IMO, the previous version of lazy import is much better. Is there any reason you
had to switch to this?

[Michael Achenbach, 2016-07-20 11:22:50]

As Andrii commented, I strongly suggest to archive the data in the existing
data.tar. It would simplify this CL.

I hope the implementation that moved to the harness repository is not assuming
that yaml is installed on the bots? I guess monkeyyaml is a pure python version.

https://codereview.chromium.org/2131743002/diff/180001/test/test262/archive.py
File test/test262/archive.py (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/archive.p...
test/test262/archive.py:30: tar.add('data', filter=filter_git)
Why not add harness folder to data.tar?

https://codereview.chromium.org/2131743002/diff/180001/test/test262/test262.gyp
File test/test262/test262.gyp (right):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/test262.g...
test/test262/test262.gyp:26: 'outputs': ['data.tar', 'harness.tar'],
On 2016/07/20 10:42:01, tandrii(chromium) wrote:
> Michael has to approve this, bcz:
> 
> 1. this does increase the size of things to be downloaded on bot,   do you
know
> by how much? Is untarring done always, or only on bots that run harness?
> 2. Is there any reason to ship it separately in harness.tar and not update
> existing data.tar?
> 3. You probably have to prepare test set (~ a bunch of isolate hashes) for
> Michael's end-to-end test.

1. Insignificant. As described, it was part of the test262 data before. It
probably is right now until test262 is rolled.
2. +1, should just be packed into data tar IMO.
3. Test262 is not part of the end-to-end test -> ignore.

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.py
File test/test262/testcfg.py (left):

https://codereview.chromium.org/2131743002/diff/180001/test/test262/testcfg.p...
test/test262/testcfg.py:155: "gclient sync for test262")
On 2016/07/20 10:42:01, tandrii(chromium) wrote:
> I would keep user friendly "ImportError", unless you are certain it never
> happens (and I am not)

You could also just move this code to toplevel at the beginning of this script.
But this looks like a refactoring and wouldn't be really needed for the main
change in this CL. Could you send it in a separate CL?

[Dan Ehrenberg, 2016-07-20 23:53:52]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-20 23:53:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-07-20 23:54:50]

OK, the new version is much simpler, DEPSing into a subdirectory and just
tweaking how the import works. WDYT?

[commit-bot: I haz the power, 2016-07-21 00:18:46]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-21 00:18:47]

Dry run: This issue passed the CQ dry run.

[Michael Achenbach, 2016-07-21 06:53:00]

Tentative lgtm. I like the shortness, though I hope it's not too short now.

Can't there now be problems due to the nested directory not having a .gitignores
entry in the test262 project? I'm not sure if gclient sync handles this
correctly. Worries:
- gclient sync is syncing different deps in parallel
- maybe syncing test262 might try to clean up the subdir?
- this could lead to the sync sometimes doing too much or, if
done in reverse order, incompleteness

Please only commit like this if you're really sure these things can't happen
(tandrii any thoughts?). If you're not sure, it's probably not worth spending
the time for finding out. Rather make a slightly bigger patch with data and
harness side-by-side, but in the same tar.

[Dan Ehrenberg, 2016-07-21 22:56:08]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-21 22:56:20]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-07-21 22:57:20]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-21 22:57:31]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-21 22:58:59]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-21 22:59:00]

Dry run: Try jobs failed on following builders:
  v8_android_arm_compile_rel on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_android_arm_compile_rel/...)
  v8_linux64_asan_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_asan_rel_ng/buil...)
  v8_linux64_avx2_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_avx2_rel_ng/buil...)
  v8_linux64_gyp_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_gyp_rel_ng/build...)
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/9513)
  v8_linux_arm64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm64_rel_ng/build...)
  v8_linux_arm_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_arm_rel_ng/builds/...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/9493)
  v8_linux_mipsel_compile_rel on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_mipsel_compile_rel...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/9459)
  v8_mac_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_mac_rel_ng/builds/5473)

[Dan Ehrenberg, 2016-07-21 23:02:07]

The CQ bit was checked by littledan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-07-21 23:02:16]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Dan Ehrenberg, 2016-07-21 23:05:58]

Left harness in a separate directory and made them share a tar file. WDYT?

[commit-bot: I haz the power, 2016-07-21 23:26:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-07-21 23:26:28]

Dry run: This issue passed the CQ dry run.

[Michael Achenbach, 2016-07-22 07:07:47]

lgtm

https://codereview.chromium.org/2131743002/diff/260001/test/test262/testcfg.py
File test/test262/testcfg.py (right):

https://codereview.chromium.org/2131743002/diff/260001/test/test262/testcfg.p...
test/test262/testcfg.py:50: TEST_262_TOOLS_PATH = ["harness", "src"]
Refactoring suggestion, maybe for follow up. I wonder why we couldn't put
abspaths here already?

TEST_ROOT = os.path.dirname(os.path.abspath(__file__))

Then join all other paths to this. I don't see how self.root below would ever
point to something else.

Even in network mode when executed on a different workstation, within this file,
the test root should be static.

[Dan Ehrenberg, 2016-07-22 15:23:23]

The CQ bit was checked by littledan@chromium.org

[commit-bot: I haz the power, 2016-07-22 15:23:32]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-07-22 15:25:14]

Committed patchset #13 (id:260001)

[commit-bot: I haz the power, 2016-07-22 15:25:46]

Description was changed from

==========
[test262] Use standalone Test262 harness project

TC-39 recently decided to remove the Python-based testing harness from the
Test262 project [1]. The code has been duplicated in a standalone project;
update V8's dependencies to fetch from that new location. This is based on
an earlier patch by Mike Pennisi.

[1]
https://github.com/tc39/tc39-notes/blob/2b9722db9b90011d6083a5f1c8ff1559cbe01...

BUG=v8:5078
==========

to

==========
[test262] Use standalone Test262 harness project

TC-39 recently decided to remove the Python-based testing harness from the
Test262 project [1]. The code has been duplicated in a standalone project;
update V8's dependencies to fetch from that new location. This is based on
an earlier patch by Mike Pennisi.

[1]
https://github.com/tc39/tc39-notes/blob/2b9722db9b90011d6083a5f1c8ff1559cbe01...

BUG=v8:5078

Committed: https://crrev.com/9f45c188ddfb9b68878e525a925e2d532df76833
Cr-Commit-Position: refs/heads/master@{#37985}
==========

[commit-bot: I haz the power, 2016-07-22 15:25:48]

Patchset 13 (id:??) landed as
https://crrev.com/9f45c188ddfb9b68878e525a925e2d532df76833
Cr-Commit-Position: refs/heads/master@{#37985}