Add new test suite for migrated blink tests.

Migrate blink tests that are not relevant to blink into a new V8 test suite called 'blink'.

This initial CL contains the new test suite code and two tests for demonstration.

Other tests will follow in a separate CL.

R=jkummerow@chromium.org

Committed: https://code.google.com/p/v8/source/detail?r=15339

[Michael Achenbach, 2013-06-17 12:09:43]

PTAL

The .js and .txt files are all copied from
<chromium_root>/src/third_party/WebKit/LayoutTests/fast/js(/script-tests). The
only modification is a new license header.

Is 'blink' actually a good name for the test suite? It does not describe its
origin very well. Maybe webkit or jsc is better?

[Jakob Kummerow, 2013-06-17 14:22:52]

"webkit" would definitely be a better name for this test suite than "blink".
After all, these are tests that the Blink project inherited from WebKit and is
now trying to get rid of.

https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status
File test/blink/blink.status (right):

https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status#newco...
test/blink/blink.status:28: prefix blink
This line is not needed.

https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status2
File test/blink/blink.status2 (right):

https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status2#newc...
test/blink/blink.status2:28: []
This entire file is not needed. It is auto-generated on demand from the .status
equivalent.

https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py
File test/blink/testcfg.py (right):

https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py#newcode51
test/blink/testcfg.py:51: dirs.sort()
I think you want to filter the "resources" directory here.

https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py#newcode109
test/blink/testcfg.py:109: if not line.strip(): continue
To make this test suite pass on Android and NaCl, you'll also have to filter the
actual output lines we ignore for the "messages" test suite.

[Michael Achenbach, 2013-06-17 20:22:47]

PTAL

On 2013/06/17 14:22:52, Jakob wrote:
> "webkit" would definitely be a better name for this test suite than "blink".
> After all, these are tests that the Blink project inherited from WebKit and is
> now trying to get rid of.
Done

> 
> https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status
> File test/blink/blink.status (right):
> 
>
https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status#newco...
> test/blink/blink.status:28: prefix blink
> This line is not needed.
Done

> 
> https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status2
> File test/blink/blink.status2 (right):
> 
>
https://codereview.chromium.org/17260002/diff/1/test/blink/blink.status2#newc...
> test/blink/blink.status2:28: []
> This entire file is not needed. It is auto-generated on demand from the
.status
> equivalent.
Done

> 
> https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py
> File test/blink/testcfg.py (right):
> 
>
https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py#newcode51
> test/blink/testcfg.py:51: dirs.sort()
> I think you want to filter the "resources" directory here.
Done

> 
>
https://codereview.chromium.org/17260002/diff/1/test/blink/testcfg.py#newcode109
> test/blink/testcfg.py:109: if not line.strip(): continue
> To make this test suite pass on Android and NaCl, you'll also have to filter
the
> actual output lines we ignore for the "messages" test suite.
Done

[Jakob Kummerow, 2013-06-18 08:43:40]

LGTM with comments.

https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
File test/webkit/testcfg.py (right):

https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
test/webkit/testcfg.py:52: if dirname == exclude:
I guess this works, but it doesn't fit with the exclusion scheme used right
above. How about:

exclude = 'resources'
if exclude in dirs:
  dirs.remove(exclude)

That way we won't even visit the 'resources' directory.
Or you could modify/extend the existing filter:

for excluded in [x for x in dirs if x.startswith('.') or x == 'resources']:
  dirs.remove(excluded)

https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
test/webkit/testcfg.py:122: if line.startswith("#") or not line.strip():
continue
Don't you need to have this line in ActIterator too to filter out the copyright
headers?

[Michael Achenbach, 2013-06-18 09:47:54]

>
https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
> test/webkit/testcfg.py:52: if dirname == exclude:
> I guess this works, but it doesn't fit with the exclusion scheme used right
> above. How about:
> 
> exclude = 'resources'
> if exclude in dirs:
>   dirs.remove(exclude)
> 
> That way we won't even visit the 'resources' directory.
> Or you could modify/extend the existing filter:
> 
> for excluded in [x for x in dirs if x.startswith('.') or x == 'resources']:
>   dirs.remove(excluded)
With both suggested ways, we would ignore _every_ subdir called 'resources'. If
the test suite grows, there could be a subdir for 'resource' tests somewhere
called special_tests/resources, which we would also ignore. But we only want to
ignore the exact dir <test root>/resources and its subdirs. I agree that this
kind of filter looks awkward - I'd rather prefer a first-class filter object
that is assigned to the tree-walker instead of these conditions. But I could not
find a nice python way of doing that...

> 
>
https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
> test/webkit/testcfg.py:122: if line.startswith("#") or not line.strip():
> continue
> Don't you need to have this line in ActIterator too to filter out the
copyright
> headers?
No - the test cases don't print out the license header.

[Jakob Kummerow, 2013-06-18 10:03:52]

On 2013/06/18 09:47:54, machenbach wrote:
> >
>
https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
> > test/webkit/testcfg.py:52: if dirname == exclude:
> > I guess this works, but it doesn't fit with the exclusion scheme used right
> > above. How about:
> > 
> > exclude = 'resources'
> > if exclude in dirs:
> >   dirs.remove(exclude)
> > 
> > That way we won't even visit the 'resources' directory.
> > Or you could modify/extend the existing filter:
> > 
> > for excluded in [x for x in dirs if x.startswith('.') or x == 'resources']:
> >   dirs.remove(excluded)
> With both suggested ways, we would ignore _every_ subdir called 'resources'.

Yes, that was actually my intention.

> If
> the test suite grows, there could be a subdir for 'resource' tests somewhere
> called special_tests/resources, which we would also ignore. 

I don't think that's a problem. It should be OK to give special meaning to the
directory name 'resources', at least within the webkit test suite.

>
https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
> > test/webkit/testcfg.py:122: if line.startswith("#") or not line.strip():
> > continue
> > Don't you need to have this line in ActIterator too to filter out the
> copyright
> > headers?
> No - the test cases don't print out the license header.

Right. I got that backwards.

[Michael Achenbach, 2013-06-18 12:30:34]

>
https://chromiumcodereview.appspot.com/17260002/diff/15001/test/webkit/testcf...
> > > test/webkit/testcfg.py:52: if dirname == exclude:
> > > I guess this works, but it doesn't fit with the exclusion scheme used
right
> > > above. How about:
> > > 
> > > exclude = 'resources'
> > > if exclude in dirs:
> > >   dirs.remove(exclude)
> > > 
> > > That way we won't even visit the 'resources' directory.
> > > Or you could modify/extend the existing filter:
> > > 
> > > for excluded in [x for x in dirs if x.startswith('.') or x ==
'resources']:
> > >   dirs.remove(excluded)
> > With both suggested ways, we would ignore _every_ subdir called 'resources'.
> 
> Yes, that was actually my intention.
Done. It probably doesn't matter, since the directory structure here might stay
rather flat anyway...

[Jakob Kummerow, 2013-06-18 13:13:55]

lgtm

[Michael Achenbach, 2013-06-19 15:14:09]

PTAL at patch 4.

The izip function omits elements from the longer iterator. If the SUT produces
more or less output then expected, this is not detected as an error.

[Michael Achenbach, 2013-06-19 15:44:39]

Patch 5 includes better formatting and fixes the iterator problem also for the
message test suite.

[Jakob Kummerow, 2013-06-19 17:03:54]

LGTM (pending approval of the copyright headers).

[Michael Achenbach, 2013-06-26 14:23:45]

Committed patchset #6 manually as r15339 (presubmit successful).