Relanding updates to swarming api to add chartjson output.

Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in output directory with a file named chart-output.json.

BUG=chromium:633253
TBR=dpranke@chromium.org

patch from issue 2336293002 at patchset 180001 (http://crrev.com/2336293002#ps180001)

Committed: https://chromium.googlesource.com/chromium/tools/build/+/5762aa84bb564e0c091aafeff5fef30ae3406cf9

[eyaich1, 2016-09-21 14:35:13]

eyaich@chromium.org changed reviewers:
+ kbr@chromium.org

[eyaich1, 2016-09-21 14:35:13]

Ok so this CL is dependent on https://codereview.chromium.org/2358013002/.

My theory is that when we include this flag that is always generated by the
isolated script in the known list of args, a chartjson-results.json file is
always generated even though we never write anything to it.

Therefore, the json I copy and pasted in the reverted CL is actually invalid:

{
"0/chartjson-output.json": "", 
"0/output.json": "{\"failures\": [], \"valid\": true}", 
}

"" is invalid json and we are trying to parse it every time because we see the
file is there and think it is valid results.  You can see we are getting 

"error decoding JSON results from shard #0" on the android tryserver:

https://build.chromium.org/p/tryserver.chromium.android/builders/linux_androi...

I suspect this is coming from line 761 of api.py, which of course is the same
error message as line 798 so I have updated it for confirmation.  

We can solve this one of two ways:

1) Update api.py to check that the results are valid from the file, but we want
to distinguish between not present and valid (the current case) and present and
invalid (could happen when chartjson results are written).  

2) Update the src side scripts to stop creating the file when we don't write out
results so this recipe should correctly execute on line 756.

I went with path 2, so this CL doesn't change much except updating some logging
so it is clear on failure cases.

[Ken Russell (switch to Gerrit), 2016-09-21 18:57:04]

Description was changed from

==========
Relanding udpates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253

Committed:
https://chromium.googlesource.com/chromium/tools/build/+/99542aa3517f274e8f44...

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

to

==========
Relanding udpates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

[Ken Russell (switch to Gerrit), 2016-09-21 23:08:43]

Description was changed from

==========
Relanding udpates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

to

==========
Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

[Ken Russell (switch to Gerrit), 2016-09-21 23:09:45]

LGTM

Good analysis. Agree with the direction you took.

[eyaich1, 2016-09-22 14:45:31]

The CQ bit was checked by eyaich@chromium.org

[commit-bot: I haz the power, 2016-09-22 14:45:42]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 14:53:51]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-09-22 14:53:52]

Try jobs failed on following builders:
  Build Presubmit on luci.infra.try (JOB_FAILED,
https://luci-milo.appspot.com/swarming/task/316b315330e23210)

[Ken Russell (switch to Gerrit), 2016-09-22 14:57:08]

On 2016/09/22 14:53:52, commit-bot: I haz the power wrote:
> Try jobs failed on following builders:
>   Build Presubmit on luci.infra.try (JOB_FAILED,
> https://luci-milo.appspot.com/swarming/task/316b315330e23210)

It might be appropriate to TBR= the reviewers of the original CL rather than
waiting for re-reviews since I think this is basically a straight reland.

[eyaich1, 2016-09-22 14:57:40]

eyaich@chromium.org changed reviewers:
+ dpranke@chromium.org

[eyaich1, 2016-09-22 14:57:40]

+dpranke for directory ownership

[eyaich1, 2016-09-22 14:58:34]

On 2016/09/22 14:57:08, Ken Russell wrote:
> On 2016/09/22 14:53:52, commit-bot: I haz the power wrote:
> > Try jobs failed on following builders:
> >   Build Presubmit on luci.infra.try (JOB_FAILED,
> > https://luci-milo.appspot.com/swarming/task/316b315330e23210)
> 
> It might be appropriate to TBR= the reviewers of the original CL rather than
> waiting for re-reviews since I think this is basically a straight reland.

I am not a chromium committer so I don't have that ability.

[Ken Russell (switch to Gerrit), 2016-09-22 15:11:15]

Description was changed from

==========
Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

to

==========
Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253
TBR=dpranke@chromium.org

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

[Ken Russell (switch to Gerrit), 2016-09-22 15:11:29]

On 2016/09/22 14:58:34, eyaich1 wrote:
> On 2016/09/22 14:57:08, Ken Russell wrote:
> > On 2016/09/22 14:53:52, commit-bot: I haz the power wrote:
> > > Try jobs failed on following builders:
> > >   Build Presubmit on luci.infra.try (JOB_FAILED,
> > > https://luci-milo.appspot.com/swarming/task/316b315330e23210)
> > 
> > It might be appropriate to TBR= the reviewers of the original CL rather than
> > waiting for re-reviews since I think this is basically a straight reland.
> 
> I am not a chromium committer so I don't have that ability.

Let me try for you. I am 99% sure Dirk won't mind.

[Ken Russell (switch to Gerrit), 2016-09-22 15:11:34]

The CQ bit was checked by kbr@chromium.org

[commit-bot: I haz the power, 2016-09-22 15:11:42]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-09-22 15:16:03]

Description was changed from

==========
Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253
TBR=dpranke@chromium.org

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)
==========

to

==========
Relanding updates to swarming api to add chartjson output.

This updates the api to check for chartjson results from a finished task in
output directory with a file named chart-output.json.

BUG=chromium:633253
TBR=dpranke@chromium.org

patch from issue 2336293002 at patchset 180001
(http://crrev.com/2336293002#ps180001)

Committed:
https://chromium.googlesource.com/chromium/tools/build/+/5762aa84bb564e0c091a...
==========

[commit-bot: I haz the power, 2016-09-22 15:16:05]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/chromium/tools/build/+/5762aa84bb564e0c091a...

[eyaich1, 2016-09-22 16:59:01]

A revert of this CL (patchset #3 id:40001) has been created in
https://codereview.chromium.org/2361873002/ by eyaich@chromium.org.

The reason for reverting is: GPU tests failing on chromium tryservers
(https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium...)

A fix is in the works, but will revert this until fix is in..

[Dirk Pranke, 2016-09-22 17:10:22]

lgtm ;)