Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(56)

Side by Side Diff: tools/perf/docs/perf_regression_sheriffing.md

Issue 2467283006: Clarify documentation on reporting bad bisects and fix anchor links. (Closed)
Patch Set: Created 4 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « tools/perf/docs/perf_bot_sheriffing.md ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Perf Regression Sheriffing (go/perfregression-sheriff) 1 # Perf Regression Sheriffing (go/perfregression-sheriff)
2 2
3 The perf regression sheriff tracks performance regressions in Chrome's 3 The perf regression sheriff tracks performance regressions in Chrome's
4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) 4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md)
5 has been created to ensure the builds and tests stay green, so the perf 5 has been created to ensure the builds and tests stay green, so the perf
6 regression sheriff role is now entirely focused on performance. 6 regression sheriff role is now entirely focused on performance.
7 7
8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_ 2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** 8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_ 2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)**
9 9
10 ## Key Responsibilities 10 ## Key Responsibilities
11 11
12 * [Triage Regressions on the Perf Dashboard](#triage) 12 * [Triage Regressions on the Perf Dashboard](#Triage-Regressions-on-the-Perf-Da shboard)
13 * [Triaging Data Stoppage Alerts](#datastoppage) 13 * [Triaging Data Stoppage Alerts](#Triaging-Data-Stoppage-Alerts)
14 * [Follow up on Performance Regressions](#followup) 14 * [Follow up on Performance Regressions](#Follow-up-on-Performance-Regressions)
15 * [Give Feedback on our Infrastructure](#feedback) 15 * [Give Feedback on our Infrastructure](#Give-Feedback-on-our-Infrastructure)
16 16
17 ## <a name="triage">Triage Regressions on the Perf Dashboard</a> 17 ## Triage Regressions on the Perf Dashboard
18 18
19 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). 19 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts).
20 20
21 In the upper right corner, **sign in with your Chromium account**. Signing in is 21 In the upper right corner, **sign in with your Chromium account**. Signing in is
22 important in order to be able to kick off bisect jobs, and see data from 22 important in order to be able to kick off bisect jobs, and see data from
23 internal waterfalls. 23 internal waterfalls.
24 24
25 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There 25 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There
26 are two tables of alerts that may be shown: 26 are two tables of alerts that may be shown:
27 27
(...skipping 44 matching lines...) Expand 10 before | Expand all | Expand 10 after
72 tooltip on the graph. If you see any likely culprits, cc the authors on the 72 tooltip on the graph. If you see any likely culprits, cc the authors on the
73 bug. 73 bug.
74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically 74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically
75 kick off a bisect for each bug you file. But if you think the regression is 75 kick off a bisect for each bug you file. But if you think the regression is
76 much clearer on one platform, or a specific page of a page set, or you want 76 much clearer on one platform, or a specific page of a page set, or you want
77 to see a broader revision range feel free to click on the alert on that graph 77 to see a broader revision range feel free to click on the alert on that graph
78 and kick off a bisect for it. There should be capacity to kick off as many 78 and kick off a bisect for it. There should be capacity to kick off as many
79 bisects as you feel are necessary to investigate; [give feedback](#feedback) 79 bisects as you feel are necessary to investigate; [give feedback](#feedback)
80 below if you feel that is not the case. 80 below if you feel that is not the case.
81 81
82 ## <a name="datastoppage">Triaging Data Stoppage Alerts</a> 82 ## Triaging Data Stoppage Alerts
83 83
84 Data stoppage alerts are listed on the 84 Data stoppage alerts are listed on the
85 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever 85 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever
86 the dashboard is monitoring a metric, and that metric stops sending data, an 86 the dashboard is monitoring a metric, and that metric stops sending data, an
87 alert is fired. Some of these alerts are expected: 87 alert is fired. Some of these alerts are expected:
88 88
89 * When a telemetry benchmark is disabled, we get a data stoppage alert. 89 * When a telemetry benchmark is disabled, we get a data stoppage alert.
90 Check the [code for the benchmark](https://code.google.com/p/chromium/codes earch#chromium/src/tools/perf/benchmarks/) 90 Check the [code for the benchmark](https://code.google.com/p/chromium/codes earch#chromium/src/tools/perf/benchmarks/)
91 to see if it has been disabled, and if so associate the alert with the 91 to see if it has been disabled, and if so associate the alert with the
92 bug for the disable. 92 bug for the disable.
(...skipping 18 matching lines...) Expand all
111 bug as well. Check for any changes to the test in the revision range. 111 bug as well. Check for any changes to the test in the revision range.
112 * Go to the buildbot status page of the bot which should be running the test. 112 * Go to the buildbot status page of the bot which should be running the test.
113 Is it running the test? If not, note that in the bug. 113 Is it running the test? If not, note that in the bug.
114 * If it is running the test and the test is failing, diagnose as a test 114 * If it is running the test and the test is failing, diagnose as a test
115 failure. 115 failure.
116 * If it is running the test and the test is passing, check the `json.output` 116 * If it is running the test and the test is passing, check the `json.output`
117 link on the buildbot status page for the test. This is the data the test 117 link on the buildbot status page for the test. This is the data the test
118 sent to the perf dashboard. Are there null values? Sometimes it lists a 118 sent to the perf dashboard. Are there null values? Sometimes it lists a
119 reason as well. Please put your finding in the bug. 119 reason as well. Please put your finding in the bug.
120 120
121 ## <a name="followup">Follow up on Performance Regressions</a> 121 ## Follow up on Performance Regressions
122 122
123 During your shift, you should try to follow up on each of the bugs you filed. 123 During your shift, you should try to follow up on each of the bugs you filed.
124 Once you've triaged all the alerts, check to see if the bisects have come back, 124 Once you've triaged all the alerts, check to see if the bisects have come back,
125 or if they failed. If the results came back, and a culprit was found, follow up 125 or if they failed. If the results came back, and a culprit was found, follow up
126 with the CL author. If the bisects failed to update the bug with results, please 126 with the CL author. If the bisects failed to update the bug with results, please
127 file a bug on it (see [feedback](#feedback) links below). 127 file a bug on it (see [feedback](#feedback) links below).
128 128
129 Also during your shift, please spend any spare time driving down bugs from the 129 Also during your shift, please spend any spare time driving down bugs from the
130 [regression backlog](http://go/triage-backlog). Treat these bugs as you would 130 [regression backlog](http://go/triage-backlog). Treat these bugs as you would
131 your own -- investigate the regressions, find out what the next step should be, 131 your own -- investigate the regressions, find out what the next step should be,
(...skipping 10 matching lines...) Expand all
142 142
143 You should aim to end your shift with an empty backlog, but it's important to 143 You should aim to end your shift with an empty backlog, but it's important to
144 still advance each bug in a meaningful way. 144 still advance each bug in a meaningful way.
145 145
146 After your shift, please try to follow up on the bugs you filed weekly. Kick off 146 After your shift, please try to follow up on the bugs you filed weekly. Kick off
147 new bisects if the previous ones failed, and if the bisect picks a likely 147 new bisects if the previous ones failed, and if the bisect picks a likely
148 culprit follow up to ensure the CL author addresses the problem. If you are 148 culprit follow up to ensure the CL author addresses the problem. If you are
149 certain that a specific CL caused a performance regression, and the author does 149 certain that a specific CL caused a performance regression, and the author does
150 not have an immediate plan to address the problem, please revert the CL. 150 not have an immediate plan to address the problem, please revert the CL.
151 151
152 ## <a name="feedback">Give Feedback on our Infrastructure</a> 152 ## Give Feedback on our Infrastructure
153 153
154 Perf regression sheriffs have their eyes on the perf dashboard and bisects 154 Perf regression sheriffs have their eyes on the perf dashboard and bisects
155 more than anyone else, and their feedback is invaluable for making sure these 155 more than anyone else, and their feedback is invaluable for making sure these
156 tools are accurate and improving them. Please file bugs and feature requests 156 tools are accurate and improving them. Please file bugs and feature requests
157 as you see them: 157 as you see them:
158 158
159 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. 159 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar.
160 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit 160 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit
161 or missing a clear culprit, or not reproducing what appears to be a clear 161 or missing a clear culprit, or not reproducing what appears to be a clear
162 regression, please link the comment the bisect bot posted on the bug at 162 regression, please link the comment the bisect bot posted on the bug at
163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR 2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). 163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR 2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0).
164 The team triages these regularly. If you spot a really clear bug (bisect 164 The team triages these regularly. If you spot a really clear bug (bisect
165 job red, bugs not being updated with bisect results) please file it in 165 job red, bugs not being updated with bisect results) please file it in
166 crbug with label `Cr-Tests-AutoBisect`. 166 crbug with label `Cr-Tests-AutoBisect`. If a bisect problem is blocking a
167 perf regression bug triage, **please file a new bug with label
168 `Cr-Tests-AutoBisect` and block the regression bug on the bisect bug**. This
169 makes it much easier for the team to triage, dupe, and close bugs on the
170 infrastructure without affecting the state of the perf regression bugs.
167 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` 171 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry`
168 and [cc the owner](http://go/perf-owners). 172 and [cc the owner](http://go/perf-owners).
OLDNEW
« no previous file with comments | « tools/perf/docs/perf_bot_sheriffing.md ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698