Add a new variable environment for testing, to replace the misusage of 'arch'.

Add a new variable environment for testing, to replace the misusage of 'arch'.
Committed: https://code.google.com/p/dart/source/detail?r=993

[ngeoffray, 2011-10-27 14:30:13]

Now that we're getting more and more 'runtimes' (vm, dartc, chromium, dartium,
frog, ...), it's silly to override the term 'arch' in test.py and use it for all
these runtimes.

Therefore I suggest giving a new command line argument to test.py, that will
define which runtime to test. The argument is called FOOBAR right now because
I'm sure you have strong opinions on the name, and FOOBAR is easy to replace.
Suggestions so far:

1) 'runtime' 
2) 'component'
3) 'project'

Personally, I'm in favor of 'runtime', but may be confusing because of the
runtime directory.

So, for example, to run the tests for the VM, you would write (note that the VM
is the default 'FOOBAR').

./tools/test.py --runtime=vm OR ./tools/test.py

To test dartc:

./tools/test.py --runtime=dartc

I think you get the idea. I have also added a 'most' runtime, that will run most
of our runtimes:

./tools/test.py --runtime=most

What do you guys think? Of course, I will change all the status files to match
this new argument, and make sure the build bots, if needed, are updated.

http://codereview.chromium.org/8408002/diff/2001/dart/tools/test.py
File dart/tools/test.py (right):

http://codereview.chromium.org/8408002/diff/2001/dart/tools/test.py#newcode1453
dart/tools/test.py:1453: 'foobar': FOOBAR,
Our tokenizer lower case expressions, so $FOOBAR in a status file becomes
foobar.

[ahe, 2011-10-27 14:38:12]

My take is that neither dartc nor frog are runtimes :-)

[zundel, 2011-10-27 14:53:33]

What I don't like about adding a third parameter is that now we have a matrix
possible configurations, but only a few combinations make sense unless compiling
the vm.  

We had this problem before in that the 'mode' argument sticks out like a sore
thumb.   Only the 'vm' code needs to be compiled separately for a debug vs.
release build, but every other subproject suffers from the general structure as
a consequence having to build a duplicate copy just to test in mode 'release',
which will just be a configuration difference for the dart --> js compilers.

Now that you've separated out build architecture from run architecture, only the
vm needs the distinction, as any dart or java code is going to generate machine
independent executable.

What I'm getting at is that for frog and dartc, only the --mode option is
useful.  If you build with --arch dartc then --runtime will also be dartc.  I
suspect the same will be true of frog.  But really, the runtime is 'd8' so what
about:

--arch dartc --mode release --runtime d8 
--arch frog --mode release --runtime d8

(this is currently called standalone, lumped together with the standalone vm)

This does make sense for the 'chromium' arch. 

--arch dartc --mode release --runtime chromium
--arch frog --mode release --runtime chromium

The *nix automake tools define a single string for architecture and could list
out valid configurations:

e.g.

dartc-release-chromium
frog-debug-d8
ia32-debug-vm
ia32-debug-dartium

You could specify wildcards like:

frog-*-d8

to run all the frog tests for debug and release in standalone mode.
On 2011/10/27 14:30:13, ngeoffray wrote:
> Now that we're getting more and more 'runtimes' (vm, dartc, chromium, dartium,
> frog, ...), it's silly to override the term 'arch' in test.py and use it for
all
> these runtimes.
> 
> Therefore I suggest giving a new command line argument to test.py, that will
> define which runtime to test. The argument is called FOOBAR right now because
> I'm sure you have strong opinions on the name, and FOOBAR is easy to replace.
> Suggestions so far:
> 
> 1) 'runtime' 
> 2) 'component'
> 3) 'project'
> 
> Personally, I'm in favor of 'runtime', but may be confusing because of the
> runtime directory.
> 
> So, for example, to run the tests for the VM, you would write (note that the
VM
> is the default 'FOOBAR').
> 
> ./tools/test.py --runtime=vm OR ./tools/test.py
> 
> To test dartc:
> 
> ./tools/test.py --runtime=dartc
> 
> I think you get the idea. I have also added a 'most' runtime, that will run
most
> of our runtimes:
> 
> ./tools/test.py --runtime=most
> 
> What do you guys think? Of course, I will change all the status files to match
> this new argument, and make sure the build bots, if needed, are updated.
> 
> http://codereview.chromium.org/8408002/diff/2001/dart/tools/test.py
> File dart/tools/test.py (right):
> 
>
http://codereview.chromium.org/8408002/diff/2001/dart/tools/test.py#newcode1453
> dart/tools/test.py:1453: 'foobar': FOOBAR,
> Our tokenizer lower case expressions, so $FOOBAR in a status file becomes
> foobar.

[ngeoffray, 2011-10-31 17:10:57]

PTAL, I have updated all our files with the new scheme.

I plan on submitting tomorrow (Tuesday) and will take care of updating the bots
and the wiki.

And we have a winner for the name. It is 'component'.

[Siggi Cherem (dart-lang), 2011-10-31 17:18:48]

lgtm (assuming FOOBAR --> chosen name :))

http://codereview.chromium.org/8408002/diff/8020/dart/tools/testing/architect...
File dart/tools/testing/architecture.py (right):

http://codereview.chromium.org/8408002/diff/8020/dart/tools/testing/architect...
dart/tools/testing/architecture.py:136: class Architecture(object):
(probably for a follow up cl) Architecture --> TestComponent?

[ngeoffray, 2011-11-01 08:30:50]

Thanks!

Replacing FOOBAR with component and submitting.

http://codereview.chromium.org/8408002/diff/8020/dart/tools/testing/architect...
File dart/tools/testing/architecture.py (right):

http://codereview.chromium.org/8408002/diff/8020/dart/tools/testing/architect...
dart/tools/testing/architecture.py:136: class Architecture(object):
On 2011/10/31 17:18:48, sigmund wrote:
> (probably for a follow up cl) Architecture --> TestComponent? 

Yes, naming them Architecture is really confusing now. I will rename the classes
in that file in a follow up CL. Thanks Siggi.