Adding in-browser correctness testing via selenium.

Adding in-browser correctness testing via selenium. 
Do not, I repeat, DO NOT enable these to run by default on the buildbot yet
(Currently they do not run by default). I am checking this in in its raw state
so that my fellow frog developers can start making fixes to frog for running in
other browsers. You can run these test by running: 

Install WebDriver stuff by following these instructions:
http://code.google.com/p/selenium/wiki/PythonBindings

Then start the server:
java -jar selenium-server-standalone-2.11.0.jar -browserSessionReuse

../tools/test.py --component=webdriver --report --timeout=10 --progress=color
--mode=release language

Frog generates code that fails in Firefox, but passes in Chrome (for example). We
need to fix this eventually, but it shouldn't cause our entire tree to turn red.
Note also that to run selenium, you will need to install the selenium python
library, and ChromeDriver.
Committed: https://code.google.com/p/dart/source/detail?r=1436

[Siggi Cherem (dart-lang), 2011-11-10 23:04:37]

very cool stuff :)!

http://codereview.chromium.org/8469016/diff/5001/frog/gen.dart
File frog/gen.dart (right):

http://codereview.chromium.org/8469016/diff/5001/frog/gen.dart#newcode335
frog/gen.dart:335: writer.writeln(endOfLine);
(nit): for this short logic, I prefer doing this in a single line, for instance:

// IE9 fails if there is a trailing comma
writer.writeln(property.setter == null ? '' : ',');

http://codereview.chromium.org/8469016/diff/5001/tools/test.py
File tools/test.py (right):

http://codereview.chromium.org/8469016/diff/5001/tools/test.py#newcode1223
tools/test.py:1223: 'frogbrowsers)',
how about: frogbrowsers -> webdriver

http://codereview.chromium.org/8469016/diff/5001/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/5001/tools/testing/architecture.p...
tools/testing/architecture.py:397: class
RealBrowserArchitecture(ChromiumArchitecture):
RealBrowserArchitecture -> WebDriverArchitecture?

http://codereview.chromium.org/8469016/diff/5001/tools/testing/runSelenium.py
File tools/testing/runSelenium.py (right):

http://codereview.chromium.org/8469016/diff/5001/tools/testing/runSelenium.py...
tools/testing/runSelenium.py:1: #!/usr/bin/python
a few nits:
- add license
- rename the file to fit the python style (run_selenium.py)
- add a top level comment (using """ ... """) explaining this script.

[Emily Fortuna, 2011-11-10 23:49:37]

Take 2!

[Jennifer Messerly, 2011-11-10 23:57:08]

lgtm

http://codereview.chromium.org/8469016/diff/7010/frog/gen.dart
File frog/gen.dart (right):

http://codereview.chromium.org/8469016/diff/7010/frog/gen.dart#newcode330
frog/gen.dart:330: // trailing comma on the last element in a list.
interesting. Is it only object literals, or does it affect list literals too?

I think we might to this in a few more places. e.g. var_member.dart,
VarMethodSet.generate. Perhaps worth fixing that location too?

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:410: frog =
os.path.abspath(os.path.join(build_root, 'frog', 'bin',
TODO: maybe this should run "frogc"?

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py
File tools/testing/test_case.py (left):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py#o...
tools/testing/test_case.py:106: command =
self.run_arch.GetCompileCommand(self.fatal_static_type_errors)
as long as this looks good to Siggi it works for me

[Siggi Cherem (dart-lang), 2011-11-11 00:06:02]

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:411: 'frogsh'))
a more compact way to write the 2 lines above:

frog = os.path.abspath(utils.GetDartRunner(self.mode, self.arch, 'frogsh'))

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:412: cmd = [frog, '--compile-only', '--out=%s' %
self.GetScriptPath()]
you might need to pass --libdir here

http://codereview.chromium.org/8469016/diff/7010/tools/testing/run_selenium.py
File tools/testing/run_selenium.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/run_selenium.p...
tools/testing/run_selenium.py:53: Main()
it's common to return a value in main, and do here: sys.exit(Main())

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py
File tools/testing/test_case.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py#n...
tools/testing/test_case.py:63: def BeforeRunHelper(self, command):
I'm not sure why this refactoring needed?

[Emily Fortuna, 2011-11-11 00:58:41]

Okay, this time, for real it should be ready!

http://codereview.chromium.org/8469016/diff/7010/frog/gen.dart
File frog/gen.dart (right):

http://codereview.chromium.org/8469016/diff/7010/frog/gen.dart#newcode330
frog/gen.dart:330: // trailing comma on the last element in a list.
I hadn't run into that case yet since I don't have the IE code fully running.
Good catch. Fixed!


On 2011/11/10 23:57:08, John Messerly wrote:
> interesting. Is it only object literals, or does it affect list literals too?
> 
> I think we might to this in a few more places. e.g. var_member.dart,
> VarMethodSet.generate. Perhaps worth fixing that location too?

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:410: frog =
os.path.abspath(os.path.join(build_root, 'frog', 'bin',
What's frogc? Add this as a TODO?

On 2011/11/10 23:57:08, John Messerly wrote:
> TODO: maybe this should run "frogc"?

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:412: cmd = [frog, '--compile-only', '--out=%s' %
self.GetScriptPath()]
On 2011/11/11 00:06:02, sigmund wrote:
> you might need to pass --libdir here

Done.

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py
File tools/testing/test_case.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/test_case.py#n...
tools/testing/test_case.py:63: def BeforeRunHelper(self, command):
Because I wanted to be able to compile tests, but not copy code. At any rate,
this has gone away because of other changes I made. 

On 2011/11/11 00:06:02, sigmund wrote:
> I'm not sure why this refactoring needed?

[Jennifer Messerly, 2011-11-11 01:02:32]

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/7010/tools/testing/architecture.p...
tools/testing/architecture.py:410: frog =
os.path.abspath(os.path.join(build_root, 'frog', 'bin',
On 2011/11/11 00:58:41, Emily Fortuna wrote:
> What's frogc? Add this as a TODO?
> 
> On 2011/11/10 23:57:08, John Messerly wrote:
> > TODO: maybe this should run "frogc"?
> 

I think the idea is it's another entry point that doesn't need --compile-only.
It'll eventually replace frog.py.

But I just noticed you're using frogsh, so nevermind :)

[Siggi Cherem (dart-lang), 2011-11-11 01:06:01]

lgtm with the changes below.

http://codereview.chromium.org/8469016/diff/40/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/40/tools/testing/architecture.py#...
tools/testing/architecture.py:402: super(WebDriverArchiecture,
self).__init__(root_path, arch, mode, \
remove the trailing '\' (here and below)

http://codereview.chromium.org/8469016/diff/40/tools/testing/architecture.py#...
tools/testing/architecture.py:412: '..','..','frog', 'lib')), '--compile-only',
\
replace (build_root, '..', '..', 'frog', 'lib')
with (self.root_path, 'frog', 'lib')

http://codereview.chromium.org/8469016/diff/40/tools/utils.py
File tools/utils.py (right):

http://codereview.chromium.org/8469016/diff/40/tools/utils.py#newcode117
tools/utils.py:117: elif component == 'frogsh' or component == 'webdriver':
is this needed?

[Siggi Cherem (dart-lang), 2011-11-11 01:08:25]

+ngeoffray

Feel free to proceed - I'm adding Nicolas just to make sure I didn't miss
anything, in which case we can fix later or rollback if necessary.

[ngeoffray, 2011-11-11 09:20:50]

Very nice. I'm looking forward to using it.

http://codereview.chromium.org/8469016/diff/6021/tools/testing/architecture.py
File tools/testing/architecture.py (right):

http://codereview.chromium.org/8469016/diff/6021/tools/testing/architecture.p...
tools/testing/architecture.py:409: frog =
os.path.abspath(utils.GetDartRunner(self.mode, self.arch, 'frogsh'))
Could that be frog instead? frogsh requires node, and people not working on frog
may not have it. If you turn that into 'frog', you'll get more people able to
test webdriver :)

BTW, that's what I did for leg.

http://codereview.chromium.org/8469016/diff/6021/tools/testing/run_selenium.py
File tools/testing/run_selenium.py (right):

http://codereview.chromium.org/8469016/diff/6021/tools/testing/run_selenium.p...
tools/testing/run_selenium.py:30: # Note: you need ChromeDriver in your path to
run chrome, in addition to
nit: chrome -> Chrome