[Swarming] Let test runner exit gracefully after test failures.

[Swarming] Let test runner exit gracefully after test failures.

The flake detection is done on the infra-side according to
the contents of the json test results. We don't want the
runner to fail after flakes.

This was controlled on the infra side by accepting any exit
codes so far. After the swarming switch, this is more
difficult, because the runner is wrapped by the swarming
collect script. There, failing exit codes can mean many
things, including network failures. Therefore, we now
force exit code 0 with test failures if those failures
are reported in the formal test results json.

The infrastructure will take care of reporting the flakes
and failures accordingly.

BUG=chromium:535160
LOG=n

Committed: https://crrev.com/efcc7fb2bd88393450ca03e5a427fd8dc4e09f1d
Cr-Commit-Position: refs/heads/master@{#31740}

[Michael Achenbach, 2015-11-03 10:06:49]

Description was changed from

==========
[Swarming] Let test runner exit gracefully after test failures.

BUG=
==========

to

==========
[Swarming] Let test runner exit gracefully after test failures.

The flake detection is done on the infra-side according to
the contents of the json test results. We don't want the
runner to fail after flakes.

This was controlled on the infra side by accepting any exit
codes so far. After the swarming switch, this is more
difficult, because the runner is wrapped by the swarming
collect script. There, failing exit codes can mean many
things, including network failures. Therefore, we now
force exit code 0 with test failures if those failures
are reported in the formal test results json.

The infrastructure will take care of reporting the flakes
and failures accordingly.

BUG=chromium:535160
LOG=n
==========

[Michael Achenbach, 2015-11-03 10:06:49]

machenbach@chromium.org changed reviewers:
+ jkummerow@chromium.org, tandrii@chromium.org

[Michael Achenbach, 2015-11-03 10:07:20]

PTAL

[Michael Achenbach, 2015-11-03 10:07:58]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2015-11-03 10:08:11]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1416373005/1
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1416373005/1

[commit-bot: I haz the power, 2015-11-03 10:24:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2015-11-03 10:24:36]

Dry run: This issue passed the CQ dry run.

[tandrii_google, 2015-11-03 10:31:33]

tandrii@google.com changed reviewers:
+ tandrii@google.com

[tandrii_google, 2015-11-03 10:31:34]

lgtm from chrome-infra perspective

[Jakob Kummerow, 2015-11-03 10:38:25]

lgtm

[Michael Achenbach, 2015-11-03 10:53:29]

The CQ bit was checked by machenbach@chromium.org

[commit-bot: I haz the power, 2015-11-03 10:53:42]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1416373005/1
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1416373005/1

[commit-bot: I haz the power, 2015-11-03 10:55:10]

Committed patchset #1 (id:1)

[commit-bot: I haz the power, 2015-11-03 10:55:27]

Patchset 1 (id:??) landed as
https://crrev.com/efcc7fb2bd88393450ca03e5a427fd8dc4e09f1d
Cr-Commit-Position: refs/heads/master@{#31740}