bots: include webdriver tests in bots.

bots: include webdriver tests in bots.

Committed: https://code.google.com/p/dart/source/detail?r=3206

[Siggi Cherem (dart-lang), 2012-01-11 18:37:01]

Bill - could you take a look at the changes in test.dart. I basically added a
flag to pass the specific browser used by the webdriver tests. We need this so
we can refer to specific browsers in the status files.

Thanks!

[Emily Fortuna, 2012-01-11 18:59:45]

http://codereview.chromium.org/9188012/diff/1011/frog/scripts/buildbot_annota...
File frog/scripts/buildbot_annotated_steps.py (right):

http://codereview.chromium.org/9188012/diff/1011/frog/scripts/buildbot_annota...
frog/scripts/buildbot_annotated_steps.py:124: for browser in browsers:
browsers never gets initialized to [], so in the case of DumpRenderTree, this
would crash. Add a line before system == 'linux'?

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
File tools/testing/dart/test_suite.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
tools/testing/dart/test_suite.dart:466: args = ['--out', htmlPath, '--browser',
configuration['browser']];
What if the browser flag isn't specified? Previously, we defaulted to chrome if
we specified webdriver without a flag. But throwing an error is also valid.

[Siggi Cherem (dart-lang), 2012-01-11 19:55:22]

http://codereview.chromium.org/9188012/diff/1011/frog/scripts/buildbot_annota...
File frog/scripts/buildbot_annotated_steps.py (right):

http://codereview.chromium.org/9188012/diff/1011/frog/scripts/buildbot_annota...
frog/scripts/buildbot_annotated_steps.py:124: for browser in browsers:
On 2012/01/11 18:59:45, Emily Fortuna wrote:
> browsers never gets initialized to [], so in the case of DumpRenderTree, this
> would crash. Add a line before system == 'linux'?


In this case it's ok, we are running webdriver tests in addition to DRT
('browsers' is always initialized to some value here)

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
File tools/testing/dart/test_suite.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
tools/testing/dart/test_suite.dart:466: args = ['--out', htmlPath, '--browser',
configuration['browser']];
On 2012/01/11 18:59:45, Emily Fortuna wrote:
> What if the browser flag isn't specified? Previously, we defaulted to chrome
if
> we specified webdriver without a flag. But throwing an error is also valid. 

Turns out I made the 'browser' flag in test_options.dart have 'chrome' as the
default.

[Emily Fortuna, 2012-01-11 20:29:06]

lgtm!

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
File tools/testing/dart/test_suite.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_suit...
tools/testing/dart/test_suite.dart:466: args = ['--out', htmlPath, '--browser',
configuration['browser']];
Gotcha! I forgot about that. Carry on.

On 2012/01/11 19:55:22, sigmund wrote:
> On 2012/01/11 18:59:45, Emily Fortuna wrote:
> > What if the browser flag isn't specified? Previously, we defaulted to chrome
> if
> > we specified webdriver without a flag. But throwing an error is also valid. 
> 
> Turns out I made the 'browser' flag in test_options.dart have 'chrome' as the
> default.

[Bill Hesse, 2012-01-11 20:50:36]

LGTM

http://codereview.chromium.org/9188012/diff/1011/tests/language/language.status
File tests/language/language.status (right):

http://codereview.chromium.org/9188012/diff/1011/tests/language/language.stat...
tests/language/language.status:301: [ $component == webdriver && $browser == ff
]
Aren't you adding the (ie, windows) and (safari, mac) cases with no exclusions
(status lines) by doing this?  This is OK because they are not run
automatically, and we want to run all tests on them to see what fails?

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
File tools/testing/dart/test_options.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
tools/testing/dart/test_options.dart:160: new _TestOptionSpecification(
Nothing is using --flag anymore, then, and nothing is implemented that looks at
this flag, so it should be removed.  If we need it for frogium= and
frogium-lib=, we can add those as separate flags.  So remove the --flag option
if you can.

[Emily Fortuna, 2012-01-11 21:07:02]

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
File tools/testing/dart/test_options.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
tools/testing/dart/test_options.dart:160: new _TestOptionSpecification(
Yes, we still need it for frog/frog-lib.

On 2012/01/11 20:50:37, Bill Hesse wrote:
> Nothing is using --flag anymore, then, and nothing is implemented that looks
at
> this flag, so it should be removed.  If we need it for frogium= and
> frogium-lib=, we can add those as separate flags.  So remove the --flag option
> if you can.

[Emily Fortuna, 2012-01-11 23:05:00]

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
File tools/testing/dart/test_options.dart (right):

http://codereview.chromium.org/9188012/diff/1011/tools/testing/dart/test_opti...
tools/testing/dart/test_options.dart:160: new _TestOptionSpecification(
That's fine. Just keep me posted on what you decide/when it goes into effect.

On 2012/01/11 21:07:03, Emily Fortuna wrote:
> Yes, we still need it for frog/frog-lib.
> 
> On 2012/01/11 20:50:37, Bill Hesse wrote:
> > Nothing is using --flag anymore, then, and nothing is implemented that looks
> at
> > this flag, so it should be removed.  If we need it for frogium= and
> > frogium-lib=, we can add those as separate flags.  So remove the --flag
option
> > if you can.
>