OLD | NEW |
1 # Perf Regression Sheriffing (go/perfregression-sheriff) | 1 # Perf Regression Sheriffing (go/perfregression-sheriff) |
2 | 2 |
3 The perf regression sheriff tracks performance regressions in Chrome's | 3 The perf regression sheriff tracks performance regressions in Chrome's |
4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) | 4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) |
5 has been created to ensure the builds and tests stay green, so the perf | 5 has been created to ensure the builds and tests stay green, so the perf |
6 regression sheriff role is now entirely focused on performance. | 6 regression sheriff role is now entirely focused on performance. |
7 | 7 |
8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_
2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** | 8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_
2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** |
9 | 9 |
10 ## Key Responsibilities | 10 ## Key Responsibilities |
11 | 11 |
12 * [Triage Regressions on the Perf Dashboard](#triage) | 12 * [Triage Regressions on the Perf Dashboard](#Triage-Regressions-on-the-Perf-Da
shboard) |
13 * [Triaging Data Stoppage Alerts](#datastoppage) | 13 * [Triaging Data Stoppage Alerts](#Triaging-Data-Stoppage-Alerts) |
14 * [Follow up on Performance Regressions](#followup) | 14 * [Follow up on Performance Regressions](#Follow-up-on-Performance-Regressions) |
15 * [Give Feedback on our Infrastructure](#feedback) | 15 * [Give Feedback on our Infrastructure](#Give-Feedback-on-our-Infrastructure) |
16 | 16 |
17 ## <a name="triage">Triage Regressions on the Perf Dashboard</a> | 17 ## Triage Regressions on the Perf Dashboard |
18 | 18 |
19 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). | 19 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). |
20 | 20 |
21 In the upper right corner, **sign in with your Chromium account**. Signing in is | 21 In the upper right corner, **sign in with your Chromium account**. Signing in is |
22 important in order to be able to kick off bisect jobs, and see data from | 22 important in order to be able to kick off bisect jobs, and see data from |
23 internal waterfalls. | 23 internal waterfalls. |
24 | 24 |
25 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There | 25 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There |
26 are two tables of alerts that may be shown: | 26 are two tables of alerts that may be shown: |
27 | 27 |
(...skipping 44 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
72 tooltip on the graph. If you see any likely culprits, cc the authors on the | 72 tooltip on the graph. If you see any likely culprits, cc the authors on the |
73 bug. | 73 bug. |
74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically | 74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically |
75 kick off a bisect for each bug you file. But if you think the regression is | 75 kick off a bisect for each bug you file. But if you think the regression is |
76 much clearer on one platform, or a specific page of a page set, or you want | 76 much clearer on one platform, or a specific page of a page set, or you want |
77 to see a broader revision range feel free to click on the alert on that graph | 77 to see a broader revision range feel free to click on the alert on that graph |
78 and kick off a bisect for it. There should be capacity to kick off as many | 78 and kick off a bisect for it. There should be capacity to kick off as many |
79 bisects as you feel are necessary to investigate; [give feedback](#feedback) | 79 bisects as you feel are necessary to investigate; [give feedback](#feedback) |
80 below if you feel that is not the case. | 80 below if you feel that is not the case. |
81 | 81 |
82 ## <a name="datastoppage">Triaging Data Stoppage Alerts</a> | 82 ## Triaging Data Stoppage Alerts |
83 | 83 |
84 Data stoppage alerts are listed on the | 84 Data stoppage alerts are listed on the |
85 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever | 85 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever |
86 the dashboard is monitoring a metric, and that metric stops sending data, an | 86 the dashboard is monitoring a metric, and that metric stops sending data, an |
87 alert is fired. Some of these alerts are expected: | 87 alert is fired. Some of these alerts are expected: |
88 | 88 |
89 * When a telemetry benchmark is disabled, we get a data stoppage alert. | 89 * When a telemetry benchmark is disabled, we get a data stoppage alert. |
90 Check the [code for the benchmark](https://code.google.com/p/chromium/codes
earch#chromium/src/tools/perf/benchmarks/) | 90 Check the [code for the benchmark](https://code.google.com/p/chromium/codes
earch#chromium/src/tools/perf/benchmarks/) |
91 to see if it has been disabled, and if so associate the alert with the | 91 to see if it has been disabled, and if so associate the alert with the |
92 bug for the disable. | 92 bug for the disable. |
(...skipping 18 matching lines...) Expand all Loading... |
111 bug as well. Check for any changes to the test in the revision range. | 111 bug as well. Check for any changes to the test in the revision range. |
112 * Go to the buildbot status page of the bot which should be running the test. | 112 * Go to the buildbot status page of the bot which should be running the test. |
113 Is it running the test? If not, note that in the bug. | 113 Is it running the test? If not, note that in the bug. |
114 * If it is running the test and the test is failing, diagnose as a test | 114 * If it is running the test and the test is failing, diagnose as a test |
115 failure. | 115 failure. |
116 * If it is running the test and the test is passing, check the `json.output` | 116 * If it is running the test and the test is passing, check the `json.output` |
117 link on the buildbot status page for the test. This is the data the test | 117 link on the buildbot status page for the test. This is the data the test |
118 sent to the perf dashboard. Are there null values? Sometimes it lists a | 118 sent to the perf dashboard. Are there null values? Sometimes it lists a |
119 reason as well. Please put your finding in the bug. | 119 reason as well. Please put your finding in the bug. |
120 | 120 |
121 ## <a name="followup">Follow up on Performance Regressions</a> | 121 ## Follow up on Performance Regressions |
122 | 122 |
123 During your shift, you should try to follow up on each of the bugs you filed. | 123 During your shift, you should try to follow up on each of the bugs you filed. |
124 Once you've triaged all the alerts, check to see if the bisects have come back, | 124 Once you've triaged all the alerts, check to see if the bisects have come back, |
125 or if they failed. If the results came back, and a culprit was found, follow up | 125 or if they failed. If the results came back, and a culprit was found, follow up |
126 with the CL author. If the bisects failed to update the bug with results, please | 126 with the CL author. If the bisects failed to update the bug with results, please |
127 file a bug on it (see [feedback](#feedback) links below). | 127 file a bug on it (see [feedback](#feedback) links below). |
128 | 128 |
129 Also during your shift, please spend any spare time driving down bugs from the | 129 Also during your shift, please spend any spare time driving down bugs from the |
130 [regression backlog](http://go/triage-backlog). Treat these bugs as you would | 130 [regression backlog](http://go/triage-backlog). Treat these bugs as you would |
131 your own -- investigate the regressions, find out what the next step should be, | 131 your own -- investigate the regressions, find out what the next step should be, |
(...skipping 10 matching lines...) Expand all Loading... |
142 | 142 |
143 You should aim to end your shift with an empty backlog, but it's important to | 143 You should aim to end your shift with an empty backlog, but it's important to |
144 still advance each bug in a meaningful way. | 144 still advance each bug in a meaningful way. |
145 | 145 |
146 After your shift, please try to follow up on the bugs you filed weekly. Kick off | 146 After your shift, please try to follow up on the bugs you filed weekly. Kick off |
147 new bisects if the previous ones failed, and if the bisect picks a likely | 147 new bisects if the previous ones failed, and if the bisect picks a likely |
148 culprit follow up to ensure the CL author addresses the problem. If you are | 148 culprit follow up to ensure the CL author addresses the problem. If you are |
149 certain that a specific CL caused a performance regression, and the author does | 149 certain that a specific CL caused a performance regression, and the author does |
150 not have an immediate plan to address the problem, please revert the CL. | 150 not have an immediate plan to address the problem, please revert the CL. |
151 | 151 |
152 ## <a name="feedback">Give Feedback on our Infrastructure</a> | 152 ## Give Feedback on our Infrastructure |
153 | 153 |
154 Perf regression sheriffs have their eyes on the perf dashboard and bisects | 154 Perf regression sheriffs have their eyes on the perf dashboard and bisects |
155 more than anyone else, and their feedback is invaluable for making sure these | 155 more than anyone else, and their feedback is invaluable for making sure these |
156 tools are accurate and improving them. Please file bugs and feature requests | 156 tools are accurate and improving them. Please file bugs and feature requests |
157 as you see them: | 157 as you see them: |
158 | 158 |
159 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. | 159 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. |
160 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit | 160 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit |
161 or missing a clear culprit, or not reproducing what appears to be a clear | 161 or missing a clear culprit, or not reproducing what appears to be a clear |
162 regression, please link the comment the bisect bot posted on the bug at | 162 regression, please link the comment the bisect bot posted on the bug at |
163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). | 163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). |
164 The team triages these regularly. If you spot a really clear bug (bisect | 164 The team triages these regularly. If you spot a really clear bug (bisect |
165 job red, bugs not being updated with bisect results) please file it in | 165 job red, bugs not being updated with bisect results) please file it in |
166 crbug with label `Cr-Tests-AutoBisect`. | 166 crbug with label `Cr-Tests-AutoBisect`. If a bisect problem is blocking a |
| 167 perf regression bug triage, **please file a new bug with label |
| 168 `Cr-Tests-AutoBisect` and block the regression bug on the bisect bug**. This |
| 169 makes it much easier for the team to triage, dupe, and close bugs on the |
| 170 infrastructure without affecting the state of the perf regression bugs. |
167 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` | 171 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` |
168 and [cc the owner](http://go/perf-owners). | 172 and [cc the owner](http://go/perf-owners). |
OLD | NEW |