[Findit] Adding additional fields to try job metadata

[Findit] Adding additional fields to try job metadata

BUG=605363
Committed: https://chromium.googlesource.com/infra/infra/+/fb4a38fd292f43d4c7e3b70fdaf039b2b45cdd4c

[lijeffrey, 2016-04-22 00:11:45]

Description was changed from

==========
[Findit] Adding additional fields to try job metadata

BUG=
==========

to

==========
[Findit] Adding additional fields to try job metadata

BUG=605363
==========

[lijeffrey, 2016-04-22 00:11:45]

lijeffrey@chromium.org changed reviewers:
+ chanli@chromium.org, stgao@chromium.org

[lijeffrey, 2016-04-22 00:15:31]

This change consolidates the method for updating try job metadata and adds build
number for easier retracing what the original build failure was that triggered
this try job should a url be unavailable, and a new field that captures the last
buildbucket build response to use in a follow up CL that may determine other
errors.

PTAL

[stgao, 2016-04-22 00:37:53]

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/common...
File appengine/findit/common/waterfall/buildbucket_client.py (right):

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/common...
appengine/findit/common/waterfall/buildbucket_client.py:87:
self.buildbucket_response = raw_json_data
Why we save two copies of the same data?

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
File appengine/findit/model/wf_try_job_data.py (right):

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
appengine/findit/model/wf_try_job_data.py:35: last_buildbucket_response =
ndb.JsonProperty(indexed=False)
This should be rarely used, let's compress it like others.

[lijeffrey, 2016-04-22 00:50:53]

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/common...
File appengine/findit/common/waterfall/buildbucket_client.py (right):

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/common...
appengine/findit/common/waterfall/buildbucket_client.py:87:
self.buildbucket_response = raw_json_data
On 2016/04/22 00:37:53, stgao wrote:
> Why we save two copies of the same data?

Oops, removed.

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
File appengine/findit/model/wf_try_job_data.py (right):

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
appengine/findit/model/wf_try_job_data.py:35: last_buildbucket_response =
ndb.JsonProperty(indexed=False)
On 2016/04/22 00:37:53, stgao wrote:
> This should be rarely used, let's compress it like others.

I'm actually thinking of getting rid of culprits, number of commits analyzed,
regression range size, etc. in a later CL since this field will contain the full
report too which can be used to derive the other information?

[stgao, 2016-04-22 01:06:22]

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
File appengine/findit/model/wf_try_job_data.py (right):

https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
appengine/findit/model/wf_try_job_data.py:35: last_buildbucket_response =
ndb.JsonProperty(indexed=False)
On 2016/04/22 00:50:53, lijeffrey wrote:
> On 2016/04/22 00:37:53, stgao wrote:
> > This should be rarely used, let's compress it like others.
> 
> I'm actually thinking of getting rid of culprits, number of commits analyzed,
> regression range size, etc. in a later CL since this field will contain the
full
> report too which can be used to derive the other information?

We could do that for new data. But how do you plan to deal with legacy data?
Drop them?

[lijeffrey, 2016-04-22 01:14:41]

On 2016/04/22 01:06:22, stgao wrote:
>
https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
> File appengine/findit/model/wf_try_job_data.py (right):
> 
>
https://codereview.chromium.org/1906293002/diff/20001/appengine/findit/model/...
> appengine/findit/model/wf_try_job_data.py:35: last_buildbucket_response =
> ndb.JsonProperty(indexed=False)
> On 2016/04/22 00:50:53, lijeffrey wrote:
> > On 2016/04/22 00:37:53, stgao wrote:
> > > This should be rarely used, let's compress it like others.
> > 
> > I'm actually thinking of getting rid of culprits, number of commits
analyzed,
> > regression range size, etc. in a later CL since this field will contain the
> full
> > report too which can be used to derive the other information?
> 
> We could do that for new data. But how do you plan to deal with legacy data?
> Drop them?

Good point, on second thought it may be better to derive the data once and store
it than to have to derive it every time when we want to monitor try jobs over
time.

[chanli, 2016-04-22 02:09:41]

On 2016/04/22 01:15:23, lijeffrey wrote:

lgtm.

[lijeffrey, 2016-04-22 18:17:49]

The CQ bit was checked by lijeffrey@chromium.org

[commit-bot: I haz the power, 2016-04-22 18:17:59]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1906293002/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1906293002/60001

[commit-bot: I haz the power, 2016-04-22 18:22:28]

Description was changed from

==========
[Findit] Adding additional fields to try job metadata

BUG=605363
==========

to

==========
[Findit] Adding additional fields to try job metadata

BUG=605363

Committed:
https://chromium.googlesource.com/infra/infra/+/fb4a38fd292f43d4c7e3b70fdaf03...
==========

[commit-bot: I haz the power, 2016-04-22 18:22:29]

Committed patchset #4 (id:60001) as
https://chromium.googlesource.com/infra/infra/+/fb4a38fd292f43d4c7e3b70fdaf03...