test/simdjs/SimdJs.json - Issue 1146073002: Add perf json for simd.js benchmarks.

Side by Side Diff: test/simdjs/SimdJs.json

Issue 1146073002: Add perf json for simd.js benchmarks. (Closed) Base URL: https://chromium.googlesource.com/v8/v8.git@master

Patch Set: Created 5 years, 7 months ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View unified diff | Download patch

OLD	NEW
(Empty)
	1 {

	2 "name": "SIMDJS",

	3 "run_count": 5,
	Michael Achenbach 2015/05/20 10:33:16 Is 5 ok? What's the total time of running this? Is 5 ok? What's the total time of running this? bradn 2015/05/20 11:30:04 It seems to be ok (I'm assuming this is because th Show quoted text On 2015/05/20 10:33:16, Michael Achenbach wrote: > Is 5 ok? What's the total time of running this? It seems to be ok (I'm assuming this is because the 4 variants run for testings aren't done here and the default for the slow test is reasonably fast).
	4 "units": "ms",

	5 "total": true,

	6 "resources": [

	7 "test/simdjs/data/src/benchmarks/kernel-template.js",

	8 "test/simdjs/data/src/benchmarks/averageFloat32x4.js",

	9 "test/simdjs/data/src/benchmarks/averageFloat32x4LoadFromInt8Array.js",

	10 "test/simdjs/data/src/benchmarks/averageFloat32x4LoadX.js",

	11 "test/simdjs/data/src/benchmarks/averageFloat32x4LoadXY.js",

	12 "test/simdjs/data/src/benchmarks/averageFloat32x4LoadXYZ.js",

	13 "test/simdjs/data/src/benchmarks/averageFloat64x2.js",

	14 "test/simdjs/data/src/benchmarks/averageFloat64x2Load.js",

	15 "test/simdjs/data/src/benchmarks/mandelbrot.js",

	16 "test/simdjs/data/src/benchmarks/matrix-multiplication.js",

	17 "test/simdjs/data/src/benchmarks/transform.js",

	18 "test/simdjs/data/src/benchmarks/shiftrows.js",

	19 "test/simdjs/data/src/benchmarks/transpose4x4.js",

	20 "test/simdjs/data/src/benchmarks/inverse4x4.js",

	21 "test/simdjs/data/src/benchmarks/sinx4.js",

	22 "test/simdjs/data/src/benchmarks/memset.js",

	23 "test/simdjs/data/src/benchmarks/memcpy.js",

	24 "test/simdjs/data/src/benchmarks/base.js",

	25 "test/simdjs/data/src/ecmascript_simd.js"

	26 ],

	27 "flags": ["--harmony-object", "test/simdjs/harness-adapt.js"],
	Michael Achenbach 2015/05/20 10:33:17 Should this be benchmark-adapt? Should this be benchmark-adapt? bradn 2015/05/20 11:30:05 Actually this is right, but that file was from an Show quoted text On 2015/05/20 10:33:17, Michael Achenbach wrote: > Should this be benchmark-adapt? Actually this is right, but that file was from an attempt to approach this differently (invoke run.js once and collect everything), that proved more verbose. Michael Achenbach 2015/05/20 12:20:02 ok Show quoted text On 2015/05/20 11:30:05, bradn wrote: > On 2015/05/20 10:33:17, Michael Achenbach wrote: > > Should this be benchmark-adapt? > > Actually this is right, but that file was from an attempt to approach this > differently (invoke run.js once and collect everything), that proved more > verbose. ok
	28 "path": ["../../"],
	Michael Achenbach 2015/05/20 10:33:16 Can the workdir not be "." and all paths be relati Can the workdir not be "." and all paths be relative to it? Or are there hardcoded loads in the runner relative to the v8 base dir? bradn 2015/05/20 11:30:04 This lets me share the harness-adapt/finish with t Show quoted text On 2015/05/20 10:33:16, Michael Achenbach wrote: > Can the workdir not be "." and all paths be relative to it? Or are there > hardcoded loads in the runner relative to the v8 base dir? This lets me share the harness-adapt/finish with the testing version of this. I could make another one for this case that's rooted here, but would need slightly different versions of those two. Which do you prefer Michael Achenbach 2015/05/20 12:20:02 I'm fine if you keep it like that for now. With ha Show quoted text On 2015/05/20 11:30:04, bradn wrote: > On 2015/05/20 10:33:16, Michael Achenbach wrote: > > Can the workdir not be "." and all paths be relative to it? Or are there > > hardcoded loads in the runner relative to the v8 base dir? > > This lets me share the harness-adapt/finish with the testing version of this. > I could make another one for this case that's rooted here, but would need > slightly different versions of those two. > Which do you prefer I'm fine if you keep it like that for now. With harness-adapt/finish, do you mean the file test/simdjs/harness-finish.js? If yes, couldn't the relative path be left out there as well?
	29 "tests": [

	30 {

	31 "name": "kernel-template",

	32 "main": "test/simdjs/harness-finish.js",
	Michael Achenbach 2015/05/20 10:33:17 It will make a new call to d8 for each "main" file It will make a new call to d8 for each "main" file. I guess that's what you want. The benchmarks will appear in a hierarchy like this: SIMDJS/kernel-template/SIMD SIMDJS/kernel-template/Non-SIMD etc bradn 2015/05/20 11:30:04 I assume this is preferrable, as then then run_per Show quoted text On 2015/05/20 10:33:17, Michael Achenbach wrote: > It will make a new call to d8 for each "main" file. I guess that's what you > want. The benchmarks will appear in a hierarchy like this: > SIMDJS/kernel-template/SIMD > SIMDJS/kernel-template/Non-SIMD > etc I assume this is preferrable, as then then run_perf knows about each? I had tried it the other way round (run run.js once), but then it seemed more verbose and then run_perf doesn't know about all the sub -tests. Michael Achenbach 2015/05/20 12:20:02 If the behavior of the benchmark is what you want Show quoted text On 2015/05/20 11:30:04, bradn wrote: > On 2015/05/20 10:33:17, Michael Achenbach wrote: > > It will make a new call to d8 for each "main" file. I guess that's what you > > want. The benchmarks will appear in a hierarchy like this: > > SIMDJS/kernel-template/SIMD > > SIMDJS/kernel-template/Non-SIMD > > etc > > I assume this is preferrable, as then then run_perf knows about each? > I had tried it the other way round (run run.js once), but then it seemed more > verbose and then run_perf doesn't know about all the sub -tests. If the behavior of the benchmark is what you want then this is preferred. There are some benchmarks that behave differently when one runner is called, e.g. the different line items influence each other and optimizations from one line item might survive into the methods of the next. Even though this sounds bad, we sometimes keep this kind of behavior if it's required to be as close as possible to some publicly existing benchmark. If the benchmark is run by other parties in a different way, you might sometimes not see the same results as they do.
	33 "flags": ["test/simdjs/data/src/benchmarks/kernel-template.js"],

	34 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	35 "tests": [

	36 {"name": "SIMD"},

	37 {"name": "Non-SIMD"},

	38 {"name": "Speedup", "units": "score"},

	39 {"name": "Iterations", "units": "count"}

	40 ]

	41 },

	42 {

	43 "name": "averageFloat32x4",

	44 "main": "test/simdjs/harness-finish.js",

	45 "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4.js"],
	Michael Achenbach 2015/05/20 10:33:17 Does this work as expected? Flags are passed to d8 Does this work as expected? Flags are passed to d8 before the main file I think. So you'd have: d8 --harmony-object test/simdjs/harness-adapt.js test/simdjs/data/src/benchmarks/averageFloat32x4.js test/simdjs/harness-finish.js bradn 2015/05/20 11:30:05 Yes this is the intended set of args. Show quoted text On 2015/05/20 10:33:17, Michael Achenbach wrote: > Does this work as expected? Flags are passed to d8 before the main file I think. > So you'd have: > d8 --harmony-object test/simdjs/harness-adapt.js > test/simdjs/data/src/benchmarks/averageFloat32x4.js > test/simdjs/harness-finish.js Yes this is the intended set of args.
	46 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	47 "tests": [

	48 {"name": "SIMD"},

	49 {"name": "Non-SIMD"},

	50 {"name": "Speedup", "units": "score"},

	51 {"name": "Iterations", "units": "count"}

	52 ]

	53 },

	54 {

	55 "name": "averageFloat32x4LoadFromInt8Array",

	56 "main": "test/simdjs/harness-finish.js",

	57 "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadFromInt8Arr ay.js"],

	58 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	59 "tests": [

	60 {"name": "SIMD"},

	61 {"name": "Non-SIMD"},

	62 {"name": "Speedup", "units": "score"},

	63 {"name": "Iterations", "units": "count"}

	64 ]

	65 },

	66 {

	67 "name": "averageFloat32x4LoadX",

	68 "main": "test/simdjs/harness-finish.js",

	69 "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadX.js"],

	70 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	71 "tests": [

	72 {"name": "SIMD"},

	73 {"name": "Non-SIMD"},

	74 {"name": "Speedup", "units": "score"},

	75 {"name": "Iterations", "units": "count"}

	76 ]

	77 },

	78 {

	79 "name": "averageFloat32x4LoadXY",

	80 "main": "test/simdjs/harness-finish.js",

	81 "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadXY.js"],

	82 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	83 "tests": [

	84 {"name": "SIMD"},

	85 {"name": "Non-SIMD"},

	86 {"name": "Speedup", "units": "score"},

	87 {"name": "Iterations", "units": "count"}

	88 ]

	89 },

	90 {

	91 "name": "averageFloat32x4LoadXYZ",

	92 "main": "test/simdjs/harness-finish.js",

	93 "flags": ["test/simdjs/data/src/benchmarks/averageFloat32x4LoadXYZ.js"],

	94 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	95 "tests": [

	96 {"name": "SIMD"},

	97 {"name": "Non-SIMD"},

	98 {"name": "Speedup", "units": "score"},

	99 {"name": "Iterations", "units": "count"}

	100 ]

	101 },

	102 {

	103 "name": "averageFloat64x2",

	104 "main": "test/simdjs/harness-finish.js",

	105 "flags": ["test/simdjs/data/src/benchmarks/averageFloat64x2.js"],

	106 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	107 "tests": [

	108 {"name": "SIMD"},

	109 {"name": "Non-SIMD"},

	110 {"name": "Speedup", "units": "score"},

	111 {"name": "Iterations", "units": "count"}

	112 ]

	113 },

	114 {

	115 "name": "averageFloat64x2Load",

	116 "main": "test/simdjs/harness-finish.js",

	117 "flags": ["test/simdjs/data/src/benchmarks/averageFloat64x2Load.js"],

	118 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	119 "tests": [

	120 {"name": "SIMD"},

	121 {"name": "Non-SIMD"},

	122 {"name": "Speedup", "units": "score"},

	123 {"name": "Iterations", "units": "count"}

	124 ]

	125 },

	126 {

	127 "name": "mandelbrot",

	128 "main": "test/simdjs/harness-finish.js",

	129 "flags": ["test/simdjs/data/src/benchmarks/mandelbrot.js"],

	130 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	131 "tests": [

	132 {"name": "SIMD"},

	133 {"name": "Non-SIMD"},

	134 {"name": "Speedup", "units": "score"},

	135 {"name": "Iterations", "units": "count"}

	136 ]

	137 },

	138 {

	139 "name": "matrix-multiplication",

	140 "main": "test/simdjs/harness-finish.js",

	141 "flags": ["test/simdjs/data/src/benchmarks/matrix-multiplication.js"],

	142 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	143 "tests": [

	144 {"name": "SIMD"},

	145 {"name": "Non-SIMD"},

	146 {"name": "Speedup", "units": "score"},

	147 {"name": "Iterations", "units": "count"}

	148 ]

	149 },

	150 {

	151 "name": "transform",

	152 "main": "test/simdjs/harness-finish.js",

	153 "flags": ["test/simdjs/data/src/benchmarks/transform.js"],

	154 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	155 "tests": [

	156 {"name": "SIMD"},

	157 {"name": "Non-SIMD"},

	158 {"name": "Speedup", "units": "score"},

	159 {"name": "Iterations", "units": "count"}

	160 ]

	161 },

	162 {

	163 "name": "shiftrows",

	164 "main": "test/simdjs/harness-finish.js",

	165 "flags": ["test/simdjs/data/src/benchmarks/shiftrows.js"],

	166 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	167 "tests": [

	168 {"name": "SIMD"},

	169 {"name": "Non-SIMD"},

	170 {"name": "Speedup", "units": "score"},

	171 {"name": "Iterations", "units": "count"}

	172 ]

	173 },

	174 {

	175 "name": "transpose4x4",

	176 "main": "test/simdjs/harness-finish.js",

	177 "flags": ["test/simdjs/data/src/benchmarks/transpose4x4.js"],

	178 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	179 "tests": [

	180 {"name": "SIMD"},

	181 {"name": "Non-SIMD"},

	182 {"name": "Speedup", "units": "score"},

	183 {"name": "Iterations", "units": "count"}

	184 ]

	185 },

	186 {

	187 "name": "inverse4x4",

	188 "main": "test/simdjs/harness-finish.js",

	189 "flags": ["test/simdjs/data/src/benchmarks/inverse4x4.js"],

	190 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	191 "tests": [

	192 {"name": "SIMD"},

	193 {"name": "Non-SIMD"},

	194 {"name": "Speedup", "units": "score"},

	195 {"name": "Iterations", "units": "count"}

	196 ]

	197 },

	198 {

	199 "name": "sinx4",

	200 "main": "test/simdjs/harness-finish.js",

	201 "flags": ["test/simdjs/data/src/benchmarks/sinx4.js"],

	202 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	203 "tests": [

	204 {"name": "SIMD"},

	205 {"name": "Non-SIMD"},

	206 {"name": "Speedup", "units": "score"},

	207 {"name": "Iterations", "units": "count"}

	208 ]

	209 },

	210 {

	211 "name": "memset",

	212 "main": "test/simdjs/harness-finish.js",

	213 "flags": ["test/simdjs/data/src/benchmarks/memset.js"],

	214 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	215 "tests": [

	216 {"name": "SIMD"},

	217 {"name": "Non-SIMD"},

	218 {"name": "Speedup", "units": "score"},

	219 {"name": "Iterations", "units": "count"}

	220 ]

	221 },

	222 {

	223 "name": "memcpy",

	224 "main": "test/simdjs/harness-finish.js",

	225 "flags": ["test/simdjs/data/src/benchmarks/memcpy.js"],

	226 "results_regexp": "%s\\([ ]*([0-9.]+)(ms)?\\)",

	227 "tests": [

	228 {"name": "SIMD"},

	229 {"name": "Non-SIMD"},

	230 {"name": "Speedup", "units": "score"},

	231 {"name": "Iterations", "units": "count"}

	232 ]

	233 }

	234 ]

	235 }

OLD	NEW

« no previous file with comments | « no previous file | test/simdjs/benchmarks-adapt.js » ('j') | test/simdjs/benchmarks-adapt.js » ('J')