[Findit] Capture and store latest buildbucket response with each query

[Findit] Capture and store latest buildbucket response with each query

BUG=639170
Committed: https://chromium.googlesource.com/infra/infra/+/afbb53a8e212103db5ff956d9debb50f08eec8b6

[lijeffrey, 2016-08-19 21:17:23]

Description was changed from

==========
[Findit] Capture and store latest buildbucket response with each query

BUG=
==========

to

==========
[Findit] Capture and store latest buildbucket response with each query

BUG=639170
==========

[lijeffrey, 2016-08-19 21:17:23]

lijeffrey@chromium.org changed reviewers:
+ chanli@chromium.org, stgao@chromium.org

[lijeffrey, 2016-08-19 21:17:36]

ptal

[stgao, 2016-08-19 21:44:37]

lgtm with nit.

https://codereview.chromium.org/2260313003/diff/1/appengine/findit/waterfall/...
File appengine/findit/waterfall/monitor_try_job_pipeline.py (right):

https://codereview.chromium.org/2260313003/diff/1/appengine/findit/waterfall/...
appengine/findit/waterfall/monitor_try_job_pipeline.py:260:
try_job_data.last_buildbucket_response = build.response
nit: not update datastore if the current response is the same as last one,
because we want to avoid too many write if possible.

[lijeffrey, 2016-08-19 23:42:59]

It seems each time we get a buildbucket response, the respone dict will be
slightly different due to "utcnow_ts" being included as a field which will
change each time. Thus simply by checking last_buildbucket_response !=
build.response isn't sufficient.

This latest iteration adds a function to check dict equality while ignoring
certain keys, but this complicates the code slightly to avoid unnecessary
writes.

WDYT?

[stgao, 2016-08-20 00:03:56]

On 2016/08/19 23:42:59, lijeffrey wrote:
> It seems each time we get a buildbucket response, the respone dict will be
> slightly different due to "utcnow_ts" being included as a field which will
> change each time. Thus simply by checking last_buildbucket_response !=
> build.response isn't sufficient.
> 
> This latest iteration adds a function to check dict equality while ignoring
> certain keys, but this complicates the code slightly to avoid unnecessary
> writes.
> 
> WDYT?

hm, that's a bit unexpected to me ...
Still lgtm

[lijeffrey, 2016-08-20 00:06:17]

The CQ bit was checked by lijeffrey@chromium.org

[commit-bot: I haz the power, 2016-08-20 00:06:22]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-08-20 00:20:38]

Description was changed from

==========
[Findit] Capture and store latest buildbucket response with each query

BUG=639170
==========

to

==========
[Findit] Capture and store latest buildbucket response with each query

BUG=639170

Committed:
https://chromium.googlesource.com/infra/infra/+/afbb53a8e212103db5ff956d9debb...
==========

[commit-bot: I haz the power, 2016-08-20 00:20:39]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/infra/infra/+/afbb53a8e212103db5ff956d9debb...