Add a ref_test key to Anomaly entities and surface it in alerts JSON.

Add a ref_test key to Anomaly entities and surface it in alerts JSON.

This will be added to group-report UI when
https://codereview.chromium.org/2633513003/ has landed.

This is part of step 3 of this doc:
https://docs.google.com/document/d/1HCoC__FPYwnuZRmnA-cmi6kKoWGq8YlFU9y3KRFbFnU/edit#

BUG=catapult:#2887

Review-Url: https://codereview.chromium.org/2627073006
Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/a0f0498a69bd83eff14ef703cd22548bbdbe028e

[sullivan, 2017-01-13 18:03:26]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-01-13 18:03:36]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[sullivan, 2017-01-13 18:03:38]

sullivan@chromium.org changed reviewers:
+ eakuefner@chromium.org, jessimb@chromium.org

[commit-bot: I haz the power, 2017-01-13 18:24:45]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-01-13 18:24:45]

Dry run: This issue passed the CQ dry run.

[eakuefner, 2017-01-13 19:45:53]

lgtm

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/alerts.py
File dashboard/dashboard/alerts.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/alerts....
dashboard/dashboard/alerts.py:168: 'ref_test': anomaly_entity.GetRefTestPath(),
Just want to say that I had thought of this in my head as I was reading the
background section of the doc, so it's gratifying to know that this is a correct
fix.

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/find_an...
File dashboard/dashboard/find_anomalies.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/find_an...
dashboard/dashboard/find_anomalies.py:225: def _GetRefBuildKeyForTest(test):
So, I'm curious if this is something you could see us subsuming with a histogram
relationship. Rather than ever querying like this, it would be nice to see if we
could set something up that records these relationships more explicitly, but I
think we could also write an MR to do the migration once that's implemented.

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/models/...
File dashboard/dashboard/models/anomaly.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/models/...
dashboard/dashboard/models/anomaly.py:59: # If the TestMetaData alerted upon has
a ref build, store the ref build.
nit: TestMetadata?

[sullivan, 2017-01-13 19:57:50]

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/alerts.py
File dashboard/dashboard/alerts.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/alerts....
dashboard/dashboard/alerts.py:168: 'ref_test': anomaly_entity.GetRefTestPath(),
On 2017/01/13 19:45:53, eakuefner wrote:
> Just want to say that I had thought of this in my head as I was reading the
> background section of the doc, so it's gratifying to know that this is a
correct
> fix.

\o/

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/find_an...
File dashboard/dashboard/find_anomalies.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/find_an...
dashboard/dashboard/find_anomalies.py:225: def _GetRefBuildKeyForTest(test):
On 2017/01/13 19:45:53, eakuefner wrote:
> So, I'm curious if this is something you could see us subsuming with a
histogram
> relationship. Rather than ever querying like this, it would be nice to see if
we
> could set something up that records these relationships more explicitly, but I
> think we could also write an MR to do the migration once that's implemented.

Yeah, the big thing we want to do is make sure we can store it efficiently (I
think a key on the TestMetadata entity makes sense here).

The mapreduce (or maybe task queue job, since those have been more reliable in
my experience) would need to run over 4 million TestMetadata entities, which
would probably take a weekend. (Running over Row takes months by contrast)

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/models/...
File dashboard/dashboard/models/anomaly.py (right):

https://codereview.chromium.org/2627073006/diff/1/dashboard/dashboard/models/...
dashboard/dashboard/models/anomaly.py:59: # If the TestMetaData alerted upon has
a ref build, store the ref build.
On 2017/01/13 19:45:53, eakuefner wrote:
> nit: TestMetadata?

Done.

[sullivan, 2017-01-13 19:58:57]

The CQ bit was checked by sullivan@chromium.org

[sullivan, 2017-01-13 19:58:58]

The patchset sent to the CQ was uploaded after l-g-t-m from
eakuefner@chromium.org
Link to the patchset: https://codereview.chromium.org/2627073006/#ps20001
(title: "Address review comments")

[commit-bot: I haz the power, 2017-01-13 19:59:08]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-01-13 20:19:43]

CQ is committing da patch.

Bot data: {"patchset_id": 20001, "attempt_start_ts": 1484337537668950,
"parent_rev": "5314d7f867e596b396a7ba1352362c534aa81dfd", "commit_rev":
"a0f0498a69bd83eff14ef703cd22548bbdbe028e"}

[commit-bot: I haz the power, 2017-01-13 20:19:46]

Description was changed from

==========
Add a ref_test key to Anomaly entities and surface it in alerts JSON.

This will be added to group-report UI when
https://codereview.chromium.org/2633513003/ has landed.

This is part of step 3 of this doc:
https://docs.google.com/document/d/1HCoC__FPYwnuZRmnA-cmi6kKoWGq8YlFU9y3KRFbF...

BUG=catapult:#2887
==========

to

==========
Add a ref_test key to Anomaly entities and surface it in alerts JSON.

This will be added to group-report UI when
https://codereview.chromium.org/2633513003/ has landed.

This is part of step 3 of this doc:
https://docs.google.com/document/d/1HCoC__FPYwnuZRmnA-cmi6kKoWGq8YlFU9y3KRFbF...

BUG=catapult:#2887

Review-Url: https://codereview.chromium.org/2627073006
Committed:
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...
==========

[commit-bot: I haz the power, 2017-01-13 20:19:47]

Committed patchset #2 (id:20001) as
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...