Update isolate files for content tests and enable swarming for them.

Update isolate files for content tests and enable swarming for them.

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=284736

[jam, 2014-07-20 21:14:22]

content_unittests seems unable to start on Windows bots. How can I debug or try
to repro this?

It would be helpful if the swarming shards display a command(s) that I can run
on a VM to reproduce how they run. Or maybe they already do and I didn't see it?

[M-A Ruel, 2014-07-21 15:47:47]

On 2014/07/20 21:14:22, jam wrote:
> content_unittests seems unable to start on Windows bots. How can I debug or
try
> to repro this?

ninja -C out/Release content_unittests
python tools/swarming_client/isolate.py run -s
out/Release/content_unittests.isolated

> It would be helpful if the swarming shards display a command(s) that I can run
> on a VM to reproduce how they run. Or maybe they already do and I didn't see
it?

They do; for example taking base_unittests on build 202 on
win_chromium_rel_swarming; 
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...

The stdout:
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...

[==========] Running: python run_isolated.zip --hash
2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
--isolate-server https://isolateserver.appspot.com --
--test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json

You can replace this with:

python tools/swarming_client/run_isolated.py --hash
2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
--isolate-server https://isolateserver.appspot.com

You can add the "--
--test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json" part if you want
but it's not strictly necessary.

If you look at the [trigger] base_unittests step and click on "shard #0", you'll
get
https://chromium-swarm.appspot.com/user/task/14758d4ba542400
which gives a bit of details about the environment (which bot, exactly when, how
long, what's the spec of the bot;
Clicking on the bot link, you can try to find the task back so you can see what
ran before and after;
https://chromium-swarm.appspot.com/restricted/bot/vm15-m4

[jam, 2014-07-21 16:22:39]

On 2014/07/21 15:47:47, M-A Ruel wrote:
> On 2014/07/20 21:14:22, jam wrote:
> > content_unittests seems unable to start on Windows bots. How can I debug or
> try
> > to repro this?
> 
> ninja -C out/Release content_unittests
> python tools/swarming_client/isolate.py run -s
> out/Release/content_unittests.isolated
> 
> > It would be helpful if the swarming shards display a command(s) that I can
run
> > on a VM to reproduce how they run. Or maybe they already do and I didn't see
> it?
> 
> They do; for example taking base_unittests on build 202 on
> win_chromium_rel_swarming; 
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> 
> The stdout:
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> 
> [==========] Running: python run_isolated.zip --hash
> 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> --isolate-server https://isolateserver.appspot.com --
> --test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json
> 
> You can replace this with:
> 
> python tools/swarming_client/run_isolated.py --hash
> 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> --isolate-server https://isolateserver.appspot.com

This mapping is magical to someone not familiar with swarming. It should be
printed in the output so that when someone sees a failure, they can run the
command locally immediately (i.e. without having to ask you what command to
run).

I tried doing this and I got a bunch of

F:\src\chromegit2\src>python tools/swarming_client/run_isolated.py --hash 
2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --na
espace default-gzip --isolate-server https://isolateserver.appspot.com
RROR             net(550): Unable to authenticate to
https://isolateserver.appspot.com (403 Client Error: Forbidden). U
e auth.py to login: python auth.py login
--service=https://isolateserver.appspot.com

I then ran 
python tools/swarming_client/auth.py login
--service=https://isolateserver.appspot.com
which opened my browser (happened to be the profile I'm signed in with my
@chromium account)
it then said

Authentication successful.
Logged in as 'user:jam@chromium.org'.

However running run_isolated.py gave me the same authentication errors as
before.

> 
> You can add the "--
> --test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json" part if you
want
> but it's not strictly necessary.
> 
> If you look at the [trigger] base_unittests step and click on "shard #0",
you'll
> get
> https://chromium-swarm.appspot.com/user/task/14758d4ba542400
> which gives a bit of details about the environment (which bot, exactly when,
how
> long, what's the spec of the bot;
> Clicking on the bot link, you can try to find the task back so you can see
what
> ran before and after;
> https://chromium-swarm.appspot.com/restricted/bot/vm15-m4

[M-A Ruel, 2014-07-21 17:14:22]

On 2014/07/21 16:22:39, jam wrote:
> On 2014/07/21 15:47:47, M-A Ruel wrote:
> > On 2014/07/20 21:14:22, jam wrote:
> > > content_unittests seems unable to start on Windows bots. How can I debug
or
> > try
> > > to repro this?
> > 
> > ninja -C out/Release content_unittests
> > python tools/swarming_client/isolate.py run -s
> > out/Release/content_unittests.isolated
> > 
> > > It would be helpful if the swarming shards display a command(s) that I can
> run
> > > on a VM to reproduce how they run. Or maybe they already do and I didn't
see
> > it?
> > 
> > They do; for example taking base_unittests on build 202 on
> > win_chromium_rel_swarming; 
> >
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> > 
> > The stdout:
> >
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> > 
> > [==========] Running: python run_isolated.zip --hash
> > 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> > --isolate-server https://isolateserver.appspot.com --
> > --test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json
> > 
> > You can replace this with:
> > 
> > python tools/swarming_client/run_isolated.py --hash
> > 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> > --isolate-server https://isolateserver.appspot.com
> 
> This mapping is magical to someone not familiar with swarming. It should be
> printed in the output so that when someone sees a failure, they can run the
> command locally immediately (i.e. without having to ask you what command to
> run).
> 
> I tried doing this and I got a bunch of
> 
> F:\src\chromegit2\src>python tools/swarming_client/run_isolated.py --hash 
> 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --na
> espace default-gzip --isolate-server https://isolateserver.appspot.com
> RROR             net(550): Unable to authenticate to
> https://isolateserver.appspot.com (403 Client Error: Forbidden). U
> e auth.py to login: python auth.py login
> --service=https://isolateserver.appspot.com
> 
> I then ran 
> python tools/swarming_client/auth.py login
> --service=https://isolateserver.appspot.com
> which opened my browser (happened to be the profile I'm signed in with my
> @chromium account)
> it then said
> 
> Authentication successful.
> Logged in as 'user:jam@chromium.org'.
> 
> However running run_isolated.py gave me the same authentication errors as
> before.

Sorry, only @google accounts can use it for now. So redo the authentication with
@google.com. E.g.

python tools/swarming_client/auth.py login
--service=https://isolateserver.appspot.com
python tools/swarming_client/auth.py login
--service=https://chromium-swarm.appspot.com

both with @google.

[jam, 2014-07-21 18:20:08]

ah, looks like the bug was that I wrote os_mesa.dll instead of osmesa.dll in the
isolate file. I filed https://code.google.com/p/chromium/issues/detail?id=395700
to track this.

ptal, this should now work

On 2014/07/21 17:14:22, M-A Ruel wrote:
> On 2014/07/21 16:22:39, jam wrote:
> > On 2014/07/21 15:47:47, M-A Ruel wrote:
> > > On 2014/07/20 21:14:22, jam wrote:
> > > > content_unittests seems unable to start on Windows bots. How can I debug
> or
> > > try
> > > > to repro this?
> > > 
> > > ninja -C out/Release content_unittests
> > > python tools/swarming_client/isolate.py run -s
> > > out/Release/content_unittests.isolated
> > > 
> > > > It would be helpful if the swarming shards display a command(s) that I
can
> > run
> > > > on a VM to reproduce how they run. Or maybe they already do and I didn't
> see
> > > it?
> > > 
> > > They do; for example taking base_unittests on build 202 on
> > > win_chromium_rel_swarming; 
> > >
> >
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> > > 
> > > The stdout:
> > >
> >
>
http://build.chromium.org/p/tryserver.chromium/builders/win_chromium_rel_swar...
> > > 
> > > [==========] Running: python run_isolated.zip --hash
> > > 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> > > --isolate-server https://isolateserver.appspot.com --
> > > --test-launcher-summary-output=${ISOLATED_OUTDIR}/output.json
> > > 
> > > You can replace this with:
> > > 
> > > python tools/swarming_client/run_isolated.py --hash
> > > 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --namespace default-gzip
> > > --isolate-server https://isolateserver.appspot.com
> > 
> > This mapping is magical to someone not familiar with swarming. It should be
> > printed in the output so that when someone sees a failure, they can run the
> > command locally immediately (i.e. without having to ask you what command to
> > run).

btw I filed https://code.google.com/p/chromium/issues/detail?id=395696 to track
this
> > 
> > I tried doing this and I got a bunch of
> > 
> > F:\src\chromegit2\src>python tools/swarming_client/run_isolated.py --hash 
> > 2bcd61c1e80cbd4342a9fbbcb1828cd9e0c687d0 --na
> > espace default-gzip --isolate-server https://isolateserver.appspot.com
> > RROR             net(550): Unable to authenticate to
> > https://isolateserver.appspot.com (403 Client Error: Forbidden). U
> > e auth.py to login: python auth.py login
> > --service=https://isolateserver.appspot.com
> > 
> > I then ran 
> > python tools/swarming_client/auth.py login
> > --service=https://isolateserver.appspot.com
> > which opened my browser (happened to be the profile I'm signed in with my
> > @chromium account)
> > it then said
> > 
> > Authentication successful.
> > Logged in as 'user:jam@chromium.org'.
> > 
> > However running run_isolated.py gave me the same authentication errors as
> > before.
> 
> Sorry, only @google accounts can use it for now. So redo the authentication
with
> @google.com. E.g.

oh, this seems worrisome as we've switched to this system but outsiders can't
repro failures. filed https://code.google.com/p/chromium/issues/detail?id=395694
to track this.

> 
> python tools/swarming_client/auth.py login
> --service=https://isolateserver.appspot.com
> python tools/swarming_client/auth.py login
> --service=https://chromium-swarm.appspot.com
> 
> both with @google.

[M-A Ruel, 2014-07-22 16:18:58]

lgtm

[jam, 2014-07-22 16:30:46]

The CQ bit was checked by jam@chromium.org

[commit-bot: I haz the power, 2014-07-22 16:31:48]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/jam@chromium.org/401923004/140001

[jam, 2014-07-22 17:05:05]

+sky for testing/buildbot owners

[sky, 2014-07-22 17:22:27]

LGTM

[commit-bot: I haz the power, 2014-07-22 18:25:51]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  chromium_presubmit on tryserver.chromium
(http://build.chromium.org/p/tryserver.chromium/builders/chromium_presubmit/bu...)

[commit-bot: I haz the power, 2014-07-22 18:29:58]

Change committed as 284736