[SAB] Test262 Agent harness

test/test262/testcfg.py

Index: test/test262/testcfg.py
diff --git a/test/test262/testcfg.py b/test/test262/testcfg.py
index e54b58993a586f5a2c46f697efa396f506883de9..56ad342f769ed5c9c216ac1ec81a68adb29da431 100644
--- a/test/test262/testcfg.py
+++ b/test/test262/testcfg.py
@@ -115,7 +115,8 @@ class Test262TestSuite(testsuite.TestSuite):
     self.harnesspath = os.path.join(self.root, *TEST_262_HARNESS_PATH)
     self.harness = [os.path.join(self.harnesspath, f)
                     for f in TEST_262_HARNESS_FILES]
-    self.harness += [os.path.join(self.root, "harness-adapt.js")]
+    self.harness += [os.path.join(self.root, "harness-adapt.js"),
+                     os.path.join(self.root, "harness-agent.js")]

[Dan Ehrenberg, 2017/01/28 03:41:06]

It would be nice if you could avoid including this file for tests that don't
need the SharedArrayBuffer test infrastructure, especially if it allocates a
SharedArrayBuffer unconditionally, but even if it doesn't, as test262 is already
relatively slow to run and loading another script could make it slower. This is
what is done below for detachArrayBuffer.js. It sounds like the logic could be,
"if it's in built-ins/Atomics, built-ins/SharedArrayBuffer, or has the
SharedArrayBuffer feature tag, you need harness-agent.js".

[binji, 2017/02/18 01:07:59]

Done.

     self.localtestroot = os.path.join(self.root, *TEST_262_LOCAL_TESTS_PATH)
     self.ParseTestRecord = None
 
@@ -151,6 +152,7 @@ class Test262TestSuite(testsuite.TestSuite):
              if "detachArrayBuffer.js" in
                 self.GetTestRecord(testcase).get("includes", [])
              else []) +
+            (["--harmony-sharedarraybuffer"]) +

[Dan Ehrenberg, 2017/01/28 03:41:06]

It would be nice to avoid passing this flag for tests that don't use
SharedArrayBuffer. It's better to be running the test262 tests in something
close to the production configuration, generally, except for in-progress
features. Could you instead put the references to "--harmony-sharedarraybuffer"
in test262.status for the tests that should have the flag passed?

To make this work, you'd actually either have to pull in the test262 tests
locally or wait until they are committed upstream and do a test262 roll. It'd be
ideal if the infrastructure is checked in together with running the tests, so we
actually get coverage of both. For more information on local test262 tests and
per-test flags, see
https://docs.google.com/document/d/16bj7AIDgZLv4WOsUEzQ5NzcEN9_xo095e88Pz8FC5...
. Note that, in the status file, you can list a whole subdirectory of tests as
needing a flag by doing something like "built-ins/Atomics/*":
["--harmony-sharedarraybuffer"]

[binji, 2017/02/18 01:07:59]

Done.

             ([flag for flag in testcase.outcomes if flag.startswith("--")]))
 
   def _VariantGeneratorFactory(self):