Add perf json for simd.js benchmarks.

test/simdjs/SimdJs.json

Index: test/simdjs/SimdJs.json
diff --git a/test/simdjs/SimdJs.json b/test/simdjs/SimdJs.json
new file mode 100644
index 0000000000000000000000000000000000000000..aac428bc0445e1303ec2032369800fcab19026da
--- /dev/null
+++ b/test/simdjs/SimdJs.json
@@ -0,0 +1,235 @@
+{
+  "name": "SIMDJS",
+  "run_count": 5,

[Michael Achenbach, 2015/05/20 10:33:16]

Is 5 ok? What's the total time of running this?

[bradn, 2015/05/20 11:30:04]

It seems to be ok (I'm assuming this is because the 4 variants run for testings
aren't done here and the default for the slow test is reasonably fast).

+  "units": "ms",
+  "total": true,
+  "resources": [
+    "test/simdjs/data/src/benchmarks/kernel-template.js",
+    "test/simdjs/data/src/benchmarks/averageFloat32x4.js",
+    "test/simdjs/data/src/benchmarks/averageFloat32x4LoadFromInt8Array.js",
+    "test/simdjs/data/src/benchmarks/averageFloat32x4LoadX.js",
+    "test/simdjs/data/src/benchmarks/averageFloat32x4LoadXY.js",
+    "test/simdjs/data/src/benchmarks/averageFloat32x4LoadXYZ.js",
+    "test/simdjs/data/src/benchmarks/averageFloat64x2.js",
+    "test/simdjs/data/src/benchmarks/averageFloat64x2Load.js",
+    "test/simdjs/data/src/benchmarks/mandelbrot.js",
+    "test/simdjs/data/src/benchmarks/matrix-multiplication.js",
+    "test/simdjs/data/src/benchmarks/transform.js",
+    "test/simdjs/data/src/benchmarks/shiftrows.js",
+    "test/simdjs/data/src/benchmarks/transpose4x4.js",
+    "test/simdjs/data/src/benchmarks/inverse4x4.js",
+    "test/simdjs/data/src/benchmarks/sinx4.js",
+    "test/simdjs/data/src/benchmarks/memset.js",
+    "test/simdjs/data/src/benchmarks/memcpy.js",
+    "test/simdjs/data/src/benchmarks/base.js",
+    "test/simdjs/data/src/ecmascript_simd.js"
+  ],
+  "flags": ["--harmony-object", "test/simdjs/harness-adapt.js"],

[Michael Achenbach, 2015/05/20 10:33:17]

Should this be benchmark-adapt?

[bradn, 2015/05/20 11:30:05]

Actually this is right, but that file was from an attempt to approach this
differently (invoke run.js once and collect everything), that proved more
verbose.

[Michael Achenbach, 2015/05/20 12:20:02]

ok

+  "path": ["../../"],

[Michael Achenbach, 2015/05/20 10:33:16]

Can the workdir not be "." and all paths be relative to it? Or are there
hardcoded loads in the runner relative to the v8 base dir?

[bradn, 2015/05/20 11:30:04]

This lets me share the harness-adapt/finish with the testing version of this.
I could make another one for this case that's rooted here, but would need
slightly different versions of those two.
Which do you prefer

[Michael Achenbach, 2015/05/20 12:20:02]

I'm fine if you keep it like that for now. With harness-adapt/finish, do you
mean the file test/simdjs/harness-finish.js? If yes, couldn't the relative path
be left out there as well?

+  "tests": [
+    {
+      "name": "kernel-template",
+      "main": "test/simdjs/harness-finish.js",

[Michael Achenbach, 2015/05/20 10:33:17]

It will make a new call to d8 for each "main" file. I guess that's what you
want. The benchmarks will appear in a hierarchy like this:
SIMDJS/kernel-template/SIMD
SIMDJS/kernel-template/Non-SIMD
etc

[bradn, 2015/05/20 11:30:04]

I assume this is preferrable, as then then run_perf knows about each?
I had tried it the other way round (run run.js once), but then it seemed more
verbose and then run_perf doesn't know about all the sub -tests.

[Michael Achenbach, 2015/05/20 12:20:02]

If the behavior of the benchmark is what you want then this is preferred. There
are some benchmarks that behave differently when one runner is called, e.g. the
different line items influence each other and optimizations from one line item
might survive into the methods of the next.

Even though this sounds bad, we sometimes keep this kind of behavior if it's
required to be as close as possible to some publicly existing benchmark. If the
benchmark is run by other parties in a different way, you might sometimes not
see the same results as they do. 

+      "flags": ["test/simdjs/data/src/benchmarks/kernel-template.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat32x4",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4.js"],

[Michael Achenbach, 2015/05/20 10:33:17]

Does this work as expected? Flags are passed to d8 before the main file I think.
So you'd have:
d8 --harmony-object test/simdjs/harness-adapt.js
test/simdjs/data/src/benchmarks/averageFloat32x4.js
test/simdjs/harness-finish.js

[bradn, 2015/05/20 11:30:05]

Yes this is the intended set of args.

+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat32x4LoadFromInt8Array",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadFromInt8Array.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat32x4LoadX",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadX.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat32x4LoadXY",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadXY.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat32x4LoadXYZ",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadXYZ.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat64x2",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat64x2.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "averageFloat64x2Load",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/averageFloat64x2Load.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "mandelbrot",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/mandelbrot.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "matrix-multiplication",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/matrix-multiplication.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "transform",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/transform.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "shiftrows",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/shiftrows.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "transpose4x4",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/transpose4x4.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "inverse4x4",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/inverse4x4.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "sinx4",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/sinx4.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "memset",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/memset.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    },
+    {
+      "name": "memcpy",
+      "main": "test/simdjs/harness-finish.js",
+      "flags": ["test/simdjs/data/src/benchmarks/memcpy.js"],
+      "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",
+      "tests": [
+        {"name": "SIMD"},
+        {"name": "Non-SIMD"},
+        {"name": "Speedup", "units": "score"},
+        {"name": "Iterations", "units": "count"}
+      ]
+    }
+  ]
+}