Refactor mjsunit/fuzz-natives-* into a separate test suite.

Refactor mjsunit/fuzz-natives-* into a separate test suite.

R=machenbach@chromium.org

Committed: https://code.google.com/p/v8/source/detail?r=21190

[Jakob Kummerow, 2014-04-29 12:12:54]

PTAL.

[Michael Achenbach, 2014-04-29 15:51:53]

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.js
File test/fuzz-natives/base.js (right):

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:32: return new Function(args.join(", "),
args.join(", ") == argsStr

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:37: for (var i = 0; i < 10; i++) {
Should we not better just loop until argc + 1?

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:45: throw "unexpected exception";
Could we not already "continue" in all other cases where func === null?

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:64: // When we have 5 or more arguments we lower the
amount of tests cases
...which is always the case.

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:65: // by randomly removing kOnManyArgumentsRemove
entries
Wouldn't it be better to always choose, 5 instead of removing 5 of 10? In case
somebody extends the list at some point...

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:67: kArgObjects : kArgObjects-kOnManyArgumentsRemove;
nit: space around -

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:70: var rand = Math.floor(Math.random() * (kArgObjects
- i));
Might be easier readable when calling repeatedly argPool.length instead of
(kArgObjects - i)

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:75: var hasMore = false;
var hasMore exists in the outer scope. Remove var?

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:77: for (var i = 0; i < argc; i++) {
Maybe add a TODO about improving the complexity, so that it doesn't depend
anymore on argc or does some other kind of combination like all-pairs testing...

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/testcf...
File test/fuzz-natives/testcfg.py (right):

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/testcf...
test/fuzz-natives/testcfg.py:28: if output.exit_code != 0:
If the command fails, the test driver would rather quietly pass with no tests...
But no need to do something about that now... I assume in release mode, the
command returns nothing?

[Michael Achenbach, 2014-04-30 09:50:43]

LGTM as discussed offline, keeping the c/p issues and improvement suggestions
from the original test case for future CLs.

[Jakob Kummerow, 2014-04-30 13:46:11]

Addressed the nits. 

Since the code in base.js is taken from the original mjsunit/fuzz-natives*
tests, I'd prefer to just keep the logic we have for now, unless there's a
concrete reason to change something. I've added a header comment to sum up our
feelings :-)

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.js
File test/fuzz-natives/base.js (right):

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:32: return new Function(args.join(", "),
On 2014/04/29 15:51:53, Michael Achenbach wrote:
> args.join(", ") == argsStr

Done.

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:67: kArgObjects : kArgObjects-kOnManyArgumentsRemove;
On 2014/04/29 15:51:53, Michael Achenbach wrote:
> nit: space around -

Done.

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/base.j...
test/fuzz-natives/base.js:75: var hasMore = false;
On 2014/04/29 15:51:53, Michael Achenbach wrote:
> var hasMore exists in the outer scope. Remove var?

Done.

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/testcf...
File test/fuzz-natives/testcfg.py (right):

https://codereview.chromium.org/252143002/diff/20001/test/fuzz-natives/testcf...
test/fuzz-natives/testcfg.py:28: if output.exit_code != 0:
On 2014/04/29 15:51:53, Michael Achenbach wrote:
> If the command fails, the test driver would rather quietly pass with no
tests...
> But no need to do something about that now... I assume in release mode, the
> command returns nothing?

%ListNatives() isn't available in Release; the try..catch ensures that the
command silently terminates with empty output.

You can have an assert instead of the empty result, it most likely won't ever
make a difference in practice.

[Michael Achenbach, 2014-05-02 12:23:21]

lgtm

[Jakob Kummerow, 2014-05-07 12:31:44]

Committed patchset #3 manually as r21190 (tree was closed).