Chromium Code Reviews
Help | Chromium Project | Sign in
(21)

Issue 2704663003: First version of benchmark health report. (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
3 days, 3 hours ago by sullivan
Modified:
2 days, 21 hours ago
Reviewers:
eakuefner, shatch
CC:
benry, catapult-reviews_chromium.org, perf-dashboard-reviews_chromium.org
Target Ref:
refs/heads/master
Project:
catapult
Visibility:
Public.

Description

First version of benchmark health report. The report consists of a backend which gathers data and 3 polymer UI elements. It currently shows 2 UIs: * A listing of benchmarks by master (defaults to ChromiumPerf): https://dev-sullivan-86772197-dot-chromeperf.appspot.com/benchmark_health_report * A simple report on the benchmark, including a list of recent alerts and which bot it is running on: https://dev-sullivan-86772197-dot-chromeperf.appspot.com/benchmark_health_report?benchmark=v8.browsing_desktop_ignition&master=ChromiumPerf&num_days=30 There are a lot of improvements we can make on the basic health report, but for this CL I'd like to focus on getting a base so it's easy for people to shard work and add new features as needed. I added server-side unit tests and basic instantiation tests for the UI, which should hopefully make it fast to iterate on. Will file bugs for additional features and issues, including: * Getting bisect quality stats into the report * Getting bug triage stats into the report * Handling tests that have multiple triage rotations * Adding warnings about missing data BUG=catapult:#3224 Review-Url: https://codereview.chromium.org/2704663003 Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/5720370545f8df69d23bcfbf221f86a282f79ded

Patch Set 1 #

Total comments: 20

Patch Set 2 : addressed review comments #

Total comments: 10

Patch Set 3 : Addressed review comments #

Patch Set 4 : Fix typo in test #

Unified diffs Side-by-side diffs Delta from patch set Stats (+755 lines, -0 lines) Patch
A dashboard/dashboard/benchmark_health_report.py View 1 2 1 chunk +89 lines, -0 lines 0 comments Download
A dashboard/dashboard/benchmark_health_report_test.py View 1 1 chunk +162 lines, -0 lines 0 comments Download
M dashboard/dashboard/dispatcher.py View 1 2 chunks +3 lines, -0 lines 0 comments Download
A dashboard/dashboard/elements/benchmark-health-report-details.html View 1 2 1 chunk +159 lines, -0 lines 0 comments Download
A dashboard/dashboard/elements/benchmark-health-report-list.html View 1 2 1 chunk +95 lines, -0 lines 0 comments Download
A dashboard/dashboard/elements/benchmark-health-report-page.html View 1 1 chunk +62 lines, -0 lines 0 comments Download
A dashboard/dashboard/elements/benchmark-health-report-page-test.html View 1 2 3 1 chunk +151 lines, -0 lines 0 comments Download
A dashboard/dashboard/static/benchmark_health_report.html View 1 1 chunk +34 lines, -0 lines 0 comments Download
Commit queue not available (can’t edit this change).

Messages

Total messages: 24 (9 generated)
sullivan
https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py File dashboard/dashboard/benchmark_quality_report.py (right): https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py#newcode47 dashboard/dashboard/benchmark_quality_report.py:47: response_values = self._GetResponseValuesForMaster(master) Current style of the dashboard is ...
3 days, 3 hours ago (2017-02-17 16:30:13 UTC) #2
nednguyen
On 2017/02/17 16:30:13, sullivan wrote: > https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py > File dashboard/dashboard/benchmark_quality_report.py (right): > > https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py#newcode47 > ...
3 days, 3 hours ago (2017-02-17 16:34:36 UTC) #3
shatch
https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py File dashboard/dashboard/benchmark_quality_report.py (right): https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py#newcode47 dashboard/dashboard/benchmark_quality_report.py:47: response_values = self._GetResponseValuesForMaster(master) On 2017/02/17 16:30:13, sullivan wrote: > ...
3 days, 1 hour ago (2017-02-17 18:03:47 UTC) #4
sullivan
I updated the name to benchmark health report as Ned suggested and addressed review comments. ...
3 days, 1 hour ago (2017-02-17 18:28:43 UTC) #6
shatch
lgtm from my end, really want to start hacking on this https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report_test.py File dashboard/dashboard/benchmark_quality_report_test.py (right): ...
3 days, 1 hour ago (2017-02-17 18:50:11 UTC) #8
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2704663003/20001
3 days, 1 hour ago (2017-02-17 18:50:59 UTC) #10
eakuefner
https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py File dashboard/dashboard/benchmark_quality_report.py (right): https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py#newcode47 dashboard/dashboard/benchmark_quality_report.py:47: response_values = self._GetResponseValuesForMaster(master) On 2017/02/17 at 18:03:47, shatch wrote: ...
3 days ago (2017-02-17 19:11:06 UTC) #11
eakuefner
Sorry, looks like my comments were mailed after the box was ticked. I've unticked the ...
3 days ago (2017-02-17 19:13:50 UTC) #13
sullivan
Didn't quite get to all the comments, new patchset not yet posted. https://codereview.chromium.org/2704663003/diff/1/dashboard/dashboard/benchmark_quality_report.py File dashboard/dashboard/benchmark_quality_report.py ...
3 days ago (2017-02-17 20:01:02 UTC) #14
eakuefner
https://codereview.chromium.org/2704663003/diff/20001/dashboard/dashboard/benchmark_health_report.py File dashboard/dashboard/benchmark_health_report.py (right): https://codereview.chromium.org/2704663003/diff/20001/dashboard/dashboard/benchmark_health_report.py#newcode58 dashboard/dashboard/benchmark_health_report.py:58: query = query.filter(anomaly.Anomaly.is_improvement == False) On 2017/02/17 at 20:01:02, ...
2 days, 23 hours ago (2017-02-17 20:13:48 UTC) #15
sullivan
Still addressing comments... https://codereview.chromium.org/2704663003/diff/20001/dashboard/dashboard/elements/benchmark-health-report-details.html File dashboard/dashboard/elements/benchmark-health-report-details.html (right): https://codereview.chromium.org/2704663003/diff/20001/dashboard/dashboard/elements/benchmark-health-report-details.html#newcode131 dashboard/dashboard/elements/benchmark-health-report-details.html:131: // Using != and == here ...
2 days, 23 hours ago (2017-02-17 20:30:11 UTC) #16
sullivan
Okay, this is ready for another look, all review comments shoudl be addressed. https://codereview.chromium.org/2704663003/diff/20001/dashboard/dashboard/elements/benchmark-health-report-details.html File ...
2 days, 22 hours ago (2017-02-17 21:29:44 UTC) #17
eakuefner
lgtm thanks
2 days, 22 hours ago (2017-02-17 21:58:39 UTC) #18
commit-bot: I haz the power
CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2704663003/60001
2 days, 21 hours ago (2017-02-17 22:21:19 UTC) #21
commit-bot: I haz the power
2 days, 21 hours ago (2017-02-17 22:42:54 UTC) #24
Message was sent while issue was closed.
Committed patchset #4 (id:60001) as
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f8e48bd