Remove 'pkg' from the default list of test suites

Remove 'pkg' from the default list of test suites

Allow 'exclude-suite' option of test.dart to include a suite not on
the default list, so existing '--exclude-suite=pkg' options are
ignored. This change does not need to be reversed later.

BUG=https://github.com/dart-lang/sdk/issues/28237
R=kustermann@google.com

Committed: https://github.com/dart-lang/sdk/commit/3f7a85e0684bcd4b37b54dced7130552e7eecb91

[kustermann, 2017-01-04 17:33:40]

https://codereview.chromium.org/2610943002/diff/1/tools/testing/dart/test_opt...
File tools/testing/dart/test_options.dart (left):

https://codereview.chromium.org/2610943002/diff/1/tools/testing/dart/test_opt...
tools/testing/dart/test_options.dart:26: 'pkg',
We need to be careful here. I'm asking for pkg to be removed when we run vm (or
precompilation) tests. This is fine, because our buildbot explicitly passes
--exclude-suite=pkg.

BUT: There are other (non-vm) bots which actually do run 'pkg' (don't pass
--exclude-suite=pkg). For example:


python -u
/b/build/slave/dart2js-linux-d8-hostchecked-1-5-be/build/sdk/tools/test.py
'--mode=release' '--arch=ia32' '--compiler=dart2js' --dart2js-batch
'--runtime=d8' '--progress=buildbot' -v --reset-browser-configuration --report
--time --failure-summary --write-debug-log --write-test-outcome-log
--copy-coredumps --host-checked '--shards=5' '--shard=1'
'--exclude-suite=observatory_ui,co19'

[Bill Hesse, 2017-01-04 17:49:51]

On 2017/01/04 17:33:40, kustermann wrote:
>
https://codereview.chromium.org/2610943002/diff/1/tools/testing/dart/test_opt...
> File tools/testing/dart/test_options.dart (left):
> 
>
https://codereview.chromium.org/2610943002/diff/1/tools/testing/dart/test_opt...
> tools/testing/dart/test_options.dart:26: 'pkg',
> We need to be careful here. I'm asking for pkg to be removed when we run vm
(or
> precompilation) tests. This is fine, because our buildbot explicitly passes
> --exclude-suite=pkg.
> 
> BUT: There are other (non-vm) bots which actually do run 'pkg' (don't pass
> --exclude-suite=pkg). For example:
> 
> 
> python -u
> /b/build/slave/dart2js-linux-d8-hostchecked-1-5-be/build/sdk/tools/test.py
> '--mode=release' '--arch=ia32' '--compiler=dart2js' --dart2js-batch
> '--runtime=d8' '--progress=buildbot' -v --reset-browser-configuration --report
> --time --failure-summary --write-debug-log --write-test-outcome-log
> --copy-coredumps --host-checked '--shards=5' '--shard=1'
> '--exclude-suite=observatory_ui,co19'

OK, I missed those.  But we still need to land this before we remove the
exclude-suite=pkg from the other bots, like the vm bots, or we will end up
running it on them.  We will also need to add the pkg tests as a separate step
on those bots, since they can't be in the same run of test.dart as the default
ones.

So we will be missing the runs of the package tests on those bots until it is
explicitly added, in the second CL that updates the recipes.

[Bill Hesse, 2017-01-10 15:27:28]

On 2017/01/10 15:24:24, Bill Hesse wrote:

Somehow my message got lost:

I have added a step running the pkg tests to all the buildbot builders that
should keep running them,
https://chromium-review.googlesource.com/#/c/425798/

so we can now remove the pkg suite from the list of default suites.
PTAL.

[kustermann, 2017-01-10 15:37:29]

lgtm

[kustermann, 2017-01-10 15:37:36]

YAY!

[Bill Hesse, 2017-01-10 15:51:28]

Description was changed from

==========
Remove 'pkg' from the default list of test suites

Allow 'exclude-suite' option of test.dart to include a suite not on
the default list, so existing '--exclude-suite=pkg' options are
ignored. This change does not need to be reversed later.

BUG=https://github.com/dart-lang/sdk/issues/28237
R=kustermann@google.com
==========

to

==========
Remove 'pkg' from the default list of test suites

Allow 'exclude-suite' option of test.dart to include a suite not on
the default list, so existing '--exclude-suite=pkg' options are
ignored. This change does not need to be reversed later.

BUG=https://github.com/dart-lang/sdk/issues/28237
R=kustermann@google.com

Committed:
https://github.com/dart-lang/sdk/commit/3f7a85e0684bcd4b37b54dced7130552e7eecb91
==========

[Bill Hesse, 2017-01-10 15:51:29]

Committed patchset #1 (id:1) manually as
3f7a85e0684bcd4b37b54dced7130552e7eecb91 (presubmit successful).