Support for running a limited amount of browser tests in parallel

tools/test.dart

 #!/usr/bin/env dart
 // Copyright (c) 2012, the Dart project authors.  Please see the AUTHORS file
 // for details. All rights reserved. Use of this source code is governed by a
 // BSD-style license that can be found in the LICENSE file.
 
 /**
  * This file is the entrypoint of the dart test suite.  This suite is used
  * to test:
  *
  *     1. the dart vm
@@ skipping 138 matching lines @@
             serverList: TestingServerRunner.serverList));
       }
     }
   }
 
   void allTestsFinished() {
     TestingServerRunner.terminateHttpServers();
     DebugLogger.close();
   }
 
+  // We can't run multiple IE processes in parallel.
+  var maxBrowserProcesses =
+      firstConf['runtime'].contains('ie') ? 1 : maxProcesses;

[ahe, 2013/01/29 09:18:42]

This is probably good enough for now, but I think it means that you'll throttle
all browsers. So if I run test.py like this:

test.py -mrelease -cdart2js -rff,chrome,ie

Both Firefox and Chrome will be throttled.

[Mads Ager (google), 2013/01/29 10:47:48]

Or none of them will be throttled because we only look at the first
configuration which could be runtime ff in this case. We should avoid passing it
in as an argument to the process queue constructor since this is a configuration
specific option. Can we have a helper method that looks at the configuration for
a given test instead of having a global setting?

[kustermann, 2013/01/29 12:10:49]

Thanks for catching this peter. Since I never use the "cross product"-feature, I
haven't thought about this case.

[kustermann, 2013/01/29 12:10:49]

IMHO, we should pass it to the process queue because it is not a configuration
specific setting -- the number of parallel IE processes is fixed for an instance
of ProcessQueue. (As mentioned in my other comment, we could make this a map
(maxBrowserProcesses = {'ie9' : 1, ...}) -- but the code would become a bit more
complicated).

I discussed this offline a bit with rico, and the simplest way is to loop
through all configurations and check if we have an IE or not. If we have one we
pass "maxBrowserProcesses=1".

In general we don't slow test.py down a lot because:
- we do use --runtime=a,b,c very seldom on test.py invocations (personally, I've
never used it)
- even if we do, it will only throttle the browser runs if we run test.py on
windows (if IE was included)
- the majority of the time is spent in dart2js compilations

+
   // Start process queue.
   new ProcessQueue(maxProcesses,
+                   maxBrowserProcesses,
                    progressIndicator,
                    startTime,
                    printTiming,
                    testSuites,
                    allTestsFinished,
                    verbose,
                    listTests);
 }