Add test case generator for runtime functions

Add test case generator for runtime functions

R=dslomov@chromium.org, machenbach@chromium.org

Committed: https://code.google.com/p/v8/source/detail?r=21199

[Jakob Kummerow, 2014-04-25 11:35:12]

PTAL.

The notable changes are in tools/.

There's no need to review all the tests in test/mjsunit/runtime-gen/; since
they're generated it's sufficient to look at one or two samples.

[Dmitry Lomov (no reviews), 2014-04-25 12:54:33]

lgtm! Very cool.

Just a couple of suggestions.

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests.py
File tools/generate-runtime-tests.py (right):

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests...
tools/generate-runtime-tests.py:257: return "%s.js" % self.name.lower()
Suggestion: why lowercase names for tests? It's ok to have mixed-case filenames,
and easier to map function name to test case.

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests...
tools/generate-runtime-tests.py:475: print("Missing testcase for %s, please run
'%s generate'" %
Suggestion: would also be awesome to check if file is added to git.

[Jakob Kummerow, 2014-05-07 13:07:53]

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests.py
File tools/generate-runtime-tests.py (right):

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests...
tools/generate-runtime-tests.py:257: return "%s.js" % self.name.lower()
On 2014/04/25 12:54:34, Dmitry Lomov (chromium) wrote:
> Suggestion: why lowercase names for tests? It's ok to have mixed-case
filenames,
> and easier to map function name to test case.

I don't care much either way, but
http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml?showone=File_N...
states a clear preference.

https://codereview.chromium.org/250923002/diff/1/tools/generate-runtime-tests...
tools/generate-runtime-tests.py:475: print("Missing testcase for %s, please run
'%s generate'" %
On 2014/04/25 12:54:34, Dmitry Lomov (chromium) wrote:
> Suggestion: would also be awesome to check if file is added to git.

Yeah, that's a possible follow-up change.

[Michael Achenbach, 2014-05-08 08:03:11]

LGTM! Nice work! The degree of hackiness in this script could be a bit lower,
but I think for the scope of this script it's ok. If the scope gets widened at
some point (e.g. to auto-generate more kinds of tests) then we should clean up a
bit more first.

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
File tools/generate-runtime-tests.py (right):

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:20: BASEPATH = os.path.join(WORKSPACE, "test",
"mjsunit", "runtime-gen")
Wouldn't this fit better into the fuzz-natives test suite?

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:193: self.ArgCtor = ctor
nit: lower case?

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:408: if i > 0: call += ", "
how about collecting the names in a list and then defining call afterwards as:
call = "%%%s%s(%s)" % (f.inline, f.name, ", ".join(arg_names))

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:450: functions = FindRuntimeFunctions()
Would it be useful if it found also functions defined in extensions (like free
buffer)?

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:487: files = os.listdir(BASEPATH)
Wonder if there appears a .svn folder on the bots here, once this is committed?

[Jakob Kummerow, 2014-05-08 09:45:36]

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
File tools/generate-runtime-tests.py (right):

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:20: BASEPATH = os.path.join(WORKSPACE, "test",
"mjsunit", "runtime-gen")
On 2014/05/08 08:03:12, Michael Achenbach wrote:
> Wouldn't this fit better into the fuzz-natives test suite?

Semantically yes, but the distinction between mjsunit and fuzz-natives is based
on what testcfg.py needs to do (how test cases are found and executed), and from
that point of view these generated tests fall squarely into mjsunit territory.

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:193: self.ArgCtor = ctor
On 2014/05/08 08:03:12, Michael Achenbach wrote:
> nit: lower case?

Chose upper case since this isn't just a field, it's a dynamically bound
non-featherweight function. (See the call in line 2260).

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:408: if i > 0: call += ", "
On 2014/05/08 08:03:12, Michael Achenbach wrote:
> how about collecting the names in a list and then defining call afterwards as:
> call = "%%%s%s(%s)" % (f.inline, f.name, ", ".join(arg_names))

I've already refactored this in a follow-up CL that's sitting on my hard disk.

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:450: functions = FindRuntimeFunctions()
On 2014/05/08 08:03:12, Michael Achenbach wrote:
> Would it be useful if it found also functions defined in extensions (like free
> buffer)?

Potentially. This is just a first step. There is plenty of room for future
improvement.

https://codereview.chromium.org/250923002/diff/20001/tools/generate-runtime-t...
tools/generate-runtime-tests.py:487: files = os.listdir(BASEPATH)
On 2014/05/08 08:03:12, Michael Achenbach wrote:
> Wonder if there appears a .svn folder on the bots here, once this is
committed?

Good point. Let's filter out dotfiles.

[Jakob Kummerow, 2014-05-08 13:15:15]

Committed patchset #4 manually as r21199 (tree was closed).