[js-perf-test] Add RegExp microbenchmarks

[js-perf-test] Add RegExp microbenchmarks

This adds microbenchmarks for:

* the RegExp constructor,
* flag accessors,
* and RegExp functions: exec, @@match, @@search, test, @@split, @@replace.

Each benchmark is further split to measure fast and slow paths.

BUG=v8:5339
Committed: https://crrev.com/b034b715687fa869e1eade1aa6e1343b6ba4e0ee
Cr-Commit-Position: refs/heads/master@{#41285}

[jgruber, 2016-11-23 13:14:36]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-11-23 13:14:47]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[jgruber, 2016-11-23 13:29:19]

jgruber@chromium.org changed reviewers:
+ machenbach@chromium.org

[jgruber, 2016-11-23 13:29:20]

Some initial results comparing pre-migration to current ToT:

Name                 Delta (in %, positive means we've improved)
----------------------------------------------------------------

Ctor-RegExp(Score):	-11
Exec-RegExp(Score):	10
Flags-RegExp(Score):	63
Match-RegExp(Score):	-70
Replace-RegExp(Score):	10
Search-RegExp(Score):	14
Split-RegExp(Score):	-25
Test-RegExp(Score):	12

This first bench batch only measures fast paths. Ctor, Match, and Split
are the three remaining functions implemented as C++ builtins.

Unfortunately, the slow paths have regressed quite a bit: 

SlowExec-RegExp(Score):	-46
SlowFlags-RegExp(Score):	-78
SlowMatch-RegExp(Score):	-74
SlowReplace-RegExp(Score):	-70
SlowSearch-RegExp(Score):	-36
SlowSplit-RegExp(Score):	-65
SlowTest-RegExp(Score):	-2

We need to make sure that fast path checks are as permissive as possible -
see crbug.com/v8/5577. Perhaps we should also look into making the slow
paths faster.

[commit-bot: I haz the power, 2016-11-23 13:40:06]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-11-23 13:40:07]

Dry run: This issue passed the CQ dry run.

[jgruber, 2016-11-23 13:45:31]

The CQ bit was checked by jgruber@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-11-23 13:45:36]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-11-23 14:11:47]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-11-23 14:11:48]

Dry run: This issue passed the CQ dry run.

[Michael Achenbach, 2016-11-24 08:25:47]

What's the expected total runtime? If it's too slow we might need to add this as
separate benchmark.

Please also make sure that single line items are not too fast? Is the benchmark
runner making sure a certain minimum time is reached or is it using fixed
iterations?

You might need to refer to somebody else for review of the meaningfulness of the
JS regexp cases.

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp/RegExpTests.json (right):

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp/RegExpTests.json:3: "run_count": 5,
This is just for convenience I assume. Note that the retry numbers in js-tests
got lowered a while ago, because they were too slow.

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp/base.js (right):

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp/base.js:24: let s = "abCdefgz";
Will it make a difference if you just return abCdefgzabCdefgzabCdefgzabCdefgz?

[jgruber, 2016-11-24 12:23:01]

On 2016/11/24 08:25:47, machenbach (slow) wrote:
> What's the expected total runtime? If it's too slow we might need to add this
as
> separate benchmark.

Locally, the benchmark runs in 1m13s. Not sure how that translates to time on
the bots and what the acceptable upper limit would be. Thoughts?

> Please also make sure that single line items are not too fast? Is the
benchmark
> runner making sure a certain minimum time is reached or is it using fixed
> iterations?

Looks like we ensure each benchmark runs for at least one second:

https://cs.chromium.org/chromium/src/v8/test/js-perf-test/base.js?q=RunSingle...

[jgruber, 2016-11-24 12:26:19]

jgruber@chromium.org changed reviewers:
+ yangguo@chromium.org

[jgruber, 2016-11-24 12:26:20]

+yang for benchmark contents.

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp/RegExpTests.json (right):

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp/RegExpTests.json:3: "run_count": 5,
On 2016/11/24 08:25:47, machenbach (slow) wrote:
> This is just for convenience I assume. Note that the retry numbers in js-tests
> got lowered a while ago, because they were too slow.

This was copied from Object/ tests. We may need to adjust run counts, I'm open
for changes here.

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp/base.js (right):

https://codereview.chromium.org/2521263003/diff/20001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp/base.js:24: let s = "abCdefgz";
On 2016/11/24 08:25:47, machenbach (slow) wrote:
> Will it make a difference if you just return abCdefgzabCdefgzabCdefgzabCdefgz?

No, we could just return
"abCdefgzabCdefgzabCdefgzabCdefgzabCdefgzabCdefgzabCdefgzabCdefgz".

I don't think it matters though since setup time is not measured.

[jgruber, 2016-11-24 15:04:21]

Moved to separate 'RegExp' test suite as discussed offline. machenbach, PTAL.

[Yang, 2016-11-25 08:53:16]

On 2016/11/24 15:04:21, jgruber wrote:
> Moved to separate 'RegExp' test suite as discussed offline. machenbach, PTAL.

lgtm

[jgruber, 2016-11-25 08:56:18]

The CQ bit was checked by jgruber@chromium.org

[commit-bot: I haz the power, 2016-11-25 08:56:30]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-11-25 09:25:12]

CQ is committing da patch.

Bot data: {"patchset_id": 40001, "attempt_start_ts": 1480064178845080,
"parent_rev": "be5d76a5c6d8bd9f96db2cd9ae03aa790fe84a2e", "commit_rev":
"f4eb82b9e7bba0eaa0dd0b3171a1eba510c8f59b"}

[commit-bot: I haz the power, 2016-11-25 09:25:15]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2016-11-25 09:25:29]

Description was changed from

==========
[js-perf-test] Add RegExp microbenchmarks

This adds microbenchmarks for:

* the RegExp constructor,
* flag accessors,
* and RegExp functions: exec, @@match, @@search, test, @@split, @@replace.

Each benchmark is further split to measure fast and slow paths.

BUG=v8:5339
==========

to

==========
[js-perf-test] Add RegExp microbenchmarks

This adds microbenchmarks for:

* the RegExp constructor,
* flag accessors,
* and RegExp functions: exec, @@match, @@search, test, @@split, @@replace.

Each benchmark is further split to measure fast and slow paths.

BUG=v8:5339

Committed: https://crrev.com/b034b715687fa869e1eade1aa6e1343b6ba4e0ee
Cr-Commit-Position: refs/heads/master@{#41285}
==========

[commit-bot: I haz the power, 2016-11-25 09:25:30]

Patchset 3 (id:??) landed as
https://crrev.com/b034b715687fa869e1eade1aa6e1343b6ba4e0ee
Cr-Commit-Position: refs/heads/master@{#41285}

[Michael Achenbach, 2016-11-28 08:33:37]

lgtm

[Michael Achenbach, 2016-11-28 11:02:08]

https://codereview.chromium.org/2521263003/diff/40001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp.json (right):

https://codereview.chromium.org/2521263003/diff/40001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp.json:39: "slow_test.js",
incorrect json. remove trailing , also below and in the end

[jgruber, 2016-11-28 12:31:42]

https://codereview.chromium.org/2521263003/diff/40001/test/js-perf-test/RegEx...
File test/js-perf-test/RegExp.json (right):

https://codereview.chromium.org/2521263003/diff/40001/test/js-perf-test/RegEx...
test/js-perf-test/RegExp.json:39: "slow_test.js",
On 2016/11/28 11:02:08, Michael Achenbach wrote:
> incorrect json. remove trailing , also below and in the end

https://codereview.chromium.org/2532013003