Split unittest tests into separate test files.

Split unittest tests into separate test files.
For now we replicate the status for all the tests in some cases (e.g.
FireFox flakiness), but after this is committed we can re-enable these 
tests to try to isolate flaky test cases more narrowly.

R=sigmund@google.com

Committed: https://code.google.com/p/dart/source/detail?r=25670

[Siggi Cherem (dart-lang), 2013-07-30 23:37:04]

lgtm

(I assumed you just moved code around, but that there is no "new" logic for me
to review in detail, correct?)

[gram, 2013-07-31 17:03:20]

Committed patchset #1 manually as r25670 (presubmit successful).

[kustermann, 2013-08-02 12:41:01]

For this specific CL: Thank you for splitting up unittest_test.dart into smaller
tests.


In general: I'm not too happy with the approach taken. Here are a few thoughts:

A lot of tests in the dart repository use the unittest framework. Which means
that unittest is a crucial piece of software. If it contained a bug, we can't
rely on our tests/green-buildbot-state/.... We get wrong confidence about the
state of the dart platform.

unittest_test_utils.dart changes the semantics of how unittest works (you're
subclassing Configuration and overriding methods).
If we want to make sure that the unittest framework is doing what it is supposed
to do, we should not change it's behaviour.

Blackbox tests could help there.

One example: By just briefly looking, I cannot find a test which ensures that if
a test (which uses pkg/unittest) fails, that the exitCode is nonzero. The
testing driver is relying on that fact.

https://codereview.chromium.org/21220004/diff/1/pkg/unittest/test/unittest_te...
File pkg/unittest/test/unittest_test_utils.dart (right):

https://codereview.chromium.org/21220004/diff/1/pkg/unittest/test/unittest_te...
pkg/unittest/test/unittest_test_utils.dart:57: _port.send(_result);
You are overriding a method here!

'Configuration.onInit()' does:
- create a new ReceivePort
- print "unittest-suite-wait-for-done"

'Configuration.onDone()' does:
- close the ReceivePort opened by 'Configuration.onInit()'
- print "unittest-suite-success"

There are two issues here:

a) By overriding 'Configuration.onDone()' and not calling the 'super.onDone()'
you leak the ReceivePort, which means that the isolate will stay alive. I'm
highly confident that this was  the reason why we got an out-of-memory crashes
of unittest_test.dart on DRT/ContentShell.

b) Even though you run some parts of the unittest framework in an isolate, you
are still printing the "unittest-suite-wait-for-done" messages. These have a
semantic meaning: These messages are used to communicate with the testing
driver.

In particular b) is very crucial: please make sure that the tests you run inside
an isolate have no side-effects / don't communicate with the testing driver.

https://codereview.chromium.org/21220004/diff/1/pkg/unittest/test/unittest_te...
pkg/unittest/test/unittest_test_utils.dart:94: });
How do you ensure that this closure will be called in the future?