Move many tests on the FYI waterfall to swarming.

Move many tests on the FYI waterfall to swarming.

I ran

   ./testing/buildbot/manage.py --convert foo_test

for all tests that were swarmed on some bot in chromium.memory.json.

This updated lots of FYI bots, but also some other bots.

Also add four blink test binaries to the Mac ASan bot. These binaries
run without issues on the ClangToTMacASan bot.

BUG=98637

[Nico, 2015-12-31 16:25:05]

Description was changed from

==========
Move many tests on the FYI waterfall to swarming.

I ran

   ./testing/buildbot/manage.py --convert foo_test

for all tests that were swarmed on some bot in chromium.memory.json.

This updated lots of FYI bots, but also some other bots.

BUG=98637
==========

to

==========
Move many tests on the FYI waterfall to swarming.

I ran

   ./testing/buildbot/manage.py --convert foo_test

for all tests that were swarmed on some bot in chromium.memory.json.

This updated lots of FYI bots, but also some other bots.

Also add four blink test binaries to the Mac ASan bot. These binaries
run without issues on the ClangToTMacASan bot.

BUG=98637
==========

[Nico, 2015-12-31 16:27:40]

thakis@chromium.org changed reviewers:
+ dpranke@chromium.org, hans@chromium.org, martiniss@chromium.org

[Nico, 2015-12-31 16:27:41]

martiniss to check if the chromium.android.json (and chromium.fyi.json) are ok
or if these bots shouldn't use swarming (in that case, I'll add them to the
blacklist in testing/buildbot/manage.py).

dpranke to check if the changes to the GN bots in chromium.linux.json are ok.

hans for all the clang tot bots in chromium.fyi.json

[hans, 2016-01-04 11:10:49]

On 2015/12/31 16:27:41, Nico wrote:
> hans for all the clang tot bots in chromium.fyi.json

lgtm

[Nico, 2016-01-14 19:01:08]

martiniss, dpranke: ping

[Nico, 2016-01-19 17:46:53]

martiniss, dpranke: ping^2

[Nico, 2016-01-19 17:46:55]

martiniss, dpranke: ping^2

[Dirk Pranke, 2016-01-19 17:51:44]

dpranke@chromium.org changed reviewers:
+ stip@chromium.org

[Dirk Pranke, 2016-01-19 17:51:44]

The code changes look mechanically fine, but I don't know if we have the
capacity or are capable of running some of these tests under swarming on
Android.

This is a lot of tests and bots to change at once. Maybe split this up into
smaller CLs?

Also checking w/ stip@ in case martiniss@ is not available.

[martiniss, 2016-01-19 20:05:57]

On 2016/01/19 at 17:51:44, dpranke wrote:
> The code changes look mechanically fine, but I don't know if we have the
capacity or are capable of running some of these tests under swarming on
Android.
> 
> This is a lot of tests and bots to change at once. Maybe split this up into
smaller CLs?
> 
> Also checking w/ stip@ in case martiniss@ is not available.

I would second the splitting this up. Maybe canary one bot, and then move the
rest.

stip@ would know if we are able to run these tests on swarming, not me. 

You were too over zealous on converting these tests to swarming though. At least
Android Cloud Tests should not be run on swarming, and probably others as well.

[ghost stip (do not use), 2016-01-19 20:20:33]

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
File testing/buildbot/chromium.android.json (right):

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
testing/buildbot/chromium.android.json:160: "can_use_on_swarming_builders": true
this will "work" in that it will launch tests on the public swarming instance,
but there is nothing restricting the execution to jelly bean. we'd need to plumb
the OS dimensions down to the recipe, set them here, and put jelly bean devices
in the pool

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
File testing/buildbot/chromium.linux.json (right):

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
testing/buildbot/chromium.linux.json:21: "can_use_on_swarming_builders": true
GN doesn't make isolate files yet, so GN swarming won't actually test what you
want

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
File testing/buildbot/chromium.memory.json (right):

https://chromiumcodereview.appspot.com/1551183002/diff/1/testing/buildbot/chr...
testing/buildbot/chromium.memory.json:531: "can_use_on_swarming_builders": true
for mac asan you'd have to talk to smut@

[martiniss, 2016-01-20 01:07:00]

martiniss@chromium.org changed reviewers:
+ bpastene@chromium.org

[martiniss, 2016-01-20 01:07:01]

Also, you probably don't want to swarm the Android Cloud builder. Talk to
bpastene

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.f...
File testing/buildbot/chromium.fyi.json (right):

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.f...
testing/buildbot/chromium.fyi.json:2: "Android Cloud Tests": {
These should not be swarmed.

[Nico, 2016-01-20 02:13:18]

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.f...
File testing/buildbot/chromium.fyi.json (right):

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.f...
testing/buildbot/chromium.fyi.json:2: "Android Cloud Tests": {
On 2016/01/20 01:07:01, martiniss wrote:
> These should not be swarmed.

If things shouldn't be swarmed on some bot, you need to update the blacklist in
testing/buildbot/manage.py to contain that bot. That's a script that
automatically manages these json files.

[Dirk Pranke, 2016-01-20 18:53:23]

On 2016/01/20 02:13:18, Nico wrote:
> If things shouldn't be swarmed on some bot, you need to update the blacklist
in
> testing/buildbot/manage.py to contain that bot. That's a script that
> automatically manages these json files.

The script isn't perfect nor a source of truth, it's more of a helper utility
(at least that's
my understanding; maruel may correct me). You still need to be aware of what
you're
doing.

[Nico, 2016-01-28 20:20:55]

(Kind of forgot about this; will break it up.)

On 2016/01/20 18:53:23, Dirk Pranke wrote:
> On 2016/01/20 02:13:18, Nico wrote:
> > If things shouldn't be swarmed on some bot, you need to update the blacklist
> in
> > testing/buildbot/manage.py to contain that bot. That's a script that
> > automatically manages these json files.
> 
> The script isn't perfect nor a source of truth, it's more of a helper utility
> (at least that's
> my understanding; maruel may correct me). You still need to be aware of what
> you're
> doing.

If it isn't the source of truth, I think it really should become it: We have so
many bots that I think there is no single person who knows about all of them.
Which means "you need to be aware of what you're doing" can only be best-effort.
On the other hand, if everyone keeps the bits of manage.py they know about
up-to-date in manage.py, then anyone on the team can make changes to bot
configs.

[Nico, 2016-01-28 20:21:33]

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.m...
File testing/buildbot/chromium.memory.json (right):

https://codereview.chromium.org/1551183002/diff/1/testing/buildbot/chromium.m...
testing/buildbot/chromium.memory.json:531: "can_use_on_swarming_builders": true
On 2016/01/19 20:20:33, stip wrote:
> for mac asan you'd have to talk to smut@

This bot already has other binaries that are swarmed.

[Dirk Pranke, 2016-01-28 23:39:21]

On 2016/01/28 20:20:55, Nico wrote:
> (Kind of forgot about this; will break it up.)
> 
> On 2016/01/20 18:53:23, Dirk Pranke wrote:
> > On 2016/01/20 02:13:18, Nico wrote:
> > > If things shouldn't be swarmed on some bot, you need to update the
blacklist
> > in
> > > testing/buildbot/manage.py to contain that bot. That's a script that
> > > automatically manages these json files.
> > 
> > The script isn't perfect nor a source of truth, it's more of a helper
utility
> > (at least that's
> > my understanding; maruel may correct me). You still need to be aware of what
> > you're
> > doing.
> 
> If it isn't the source of truth, I think it really should become it: We have
so
> many bots that I think there is no single person who knows about all of them.
> Which means "you need to be aware of what you're doing" can only be
best-effort.
> On the other hand, if everyone keeps the bits of manage.py they know about
> up-to-date in manage.py, then anyone on the team can make changes to bot
> configs.

I agree that if manage.py is meant to have long-term value and not just be a
helper
utility that should eventually be deleted, then we should make it work
consistently
with all of the json files so that it doesn't potentially corrupt things.

[Dirk Pranke, 2016-08-31 23:12:22]

closing ... I think we ended up migrating everything in other CLs?

[Nico, 2016-08-31 23:14:03]

Yes, maruel++ did this a few weeks ago.