OLD | NEW |
1 # Perf Regression Sheriffing (go/perfregression-sheriff) | 1 # Perf Regression Sheriffing (go/perfregression-sheriff) |
2 | 2 |
3 The perf regression sheriff tracks performance regressions in Chrome's | 3 The perf regression sheriff tracks performance regressions in Chrome's |
4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) | 4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) |
5 has been created to ensure the builds and tests stay green, so the perf | 5 has been created to ensure the builds and tests stay green, so the perf |
6 regression sheriff role is now entirely focused on performance. | 6 regression sheriff role is now entirely focused on performance. |
7 | 7 |
8 ## Key Responsibilities | 8 ## Key Responsibilities |
9 | 9 |
10 * [Triage Regressions on the Perf Dashboard](#triage) | 10 * [Triage Regressions on the Perf Dashboard](#triage) |
11 * [Triaging Data Stoppage Alerts](#datastoppage) | 11 * [Triaging Data Stoppage Alerts](#datastoppage) |
12 * [Follow up on Performance Regressions](#followup) | 12 * [Follow up on Performance Regressions](#followup) |
13 * [Give Feedback on our Infrastructure](#feedback) | 13 * [Give Feedback on our Infrastructure](#feedback) |
14 | 14 |
15 ###<a name="triage"></a> Triage Regressions on the Perf Dashboard | 15 ##<a name="triage"></a> Triage Regressions on the Perf Dashboard |
16 | 16 |
17 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). | 17 Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). |
18 | 18 |
19 In the upper right corner, **sign in with your Chromium account**. Signing in is | 19 In the upper right corner, **sign in with your Chromium account**. Signing in is |
20 important in order to be able to kick off bisect jobs, and see data from | 20 important in order to be able to kick off bisect jobs, and see data from |
21 internal waterfalls. | 21 internal waterfalls. |
22 | 22 |
23 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There | 23 Pick up **Chromium Perf Sheriff** from "Select an item ▼" drop down menu. There |
24 are two tables of alerts that may be shown: | 24 are two tables of alerts that may be shown: |
25 | 25 |
(...skipping 40 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
66 tooltip on the graph. If you see any likely culprits, cc the authors on the | 66 tooltip on the graph. If you see any likely culprits, cc the authors on the |
67 bug. | 67 bug. |
68 5. **Optionally, kick off more bisects**. The perf dashboard will automatically | 68 5. **Optionally, kick off more bisects**. The perf dashboard will automatically |
69 kick off a bisect for each bug you file. But if you think the regression is | 69 kick off a bisect for each bug you file. But if you think the regression is |
70 much clearer on one platform, or a specific page of a page set, or you want | 70 much clearer on one platform, or a specific page of a page set, or you want |
71 to see a broader revision range feel free to click on the alert on that graph | 71 to see a broader revision range feel free to click on the alert on that graph |
72 and kick off a bisect for it. There should be capacity to kick off as many | 72 and kick off a bisect for it. There should be capacity to kick off as many |
73 bisects as you feel are necessary to investigate; [give feedback](#feedback) | 73 bisects as you feel are necessary to investigate; [give feedback](#feedback) |
74 below if you feel that is not the case. | 74 below if you feel that is not the case. |
75 | 75 |
76 ###<a name="datastoppage"></a> Triaging data stoppage alerts | 76 ##<a name="datastoppage"></a> Triaging data stoppage alerts |
77 | 77 |
78 Data stoppage alerts are listed on the | 78 Data stoppage alerts are listed on the |
79 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever | 79 [perf dashboard alerts page](https://chromeperf.appspot.com/alerts). Whenever |
80 the dashboard is monitoring a metric, and that metric stops sending data, an | 80 the dashboard is monitoring a metric, and that metric stops sending data, an |
81 alert is fired. Some of these alerts are expected: | 81 alert is fired. Some of these alerts are expected: |
82 | 82 |
83 * When a telemetry benchmark is disabled, we get a data stoppage alert. | 83 * When a telemetry benchmark is disabled, we get a data stoppage alert. |
84 Check the [code for the benchmark](https://code.google.com/p/chromium/codes
earch#chromium/src/tools/perf/benchmarks/) | 84 Check the [code for the benchmark](https://code.google.com/p/chromium/codes
earch#chromium/src/tools/perf/benchmarks/) |
85 to see if it has been disabled, and if so associate the alert with the | 85 to see if it has been disabled, and if so associate the alert with the |
86 bug for the disable. | 86 bug for the disable. |
(...skipping 14 matching lines...) Expand all Loading... |
101 bug as well. Check for any changes to the test in the revision range. | 101 bug as well. Check for any changes to the test in the revision range. |
102 * Go to the buildbot status page of the bot which should be running the test. | 102 * Go to the buildbot status page of the bot which should be running the test. |
103 Is it running the test? If not, note that in the bug. | 103 Is it running the test? If not, note that in the bug. |
104 * If it is running the test and the test is failing, diagnose as a test | 104 * If it is running the test and the test is failing, diagnose as a test |
105 failure. | 105 failure. |
106 * If it is running the test and the test is passing, check the `json.output` | 106 * If it is running the test and the test is passing, check the `json.output` |
107 link on the buildbot status page for the test. This is the data the test | 107 link on the buildbot status page for the test. This is the data the test |
108 sent to the perf dashboard. Are there null values? Sometimes it lists a | 108 sent to the perf dashboard. Are there null values? Sometimes it lists a |
109 reason as well. Please put your finding in the bug. | 109 reason as well. Please put your finding in the bug. |
110 | 110 |
111 ###<a name="followup"></a> Follow up on Performance Regressions | 111 ##<a name="followup"></a> Follow up on Performance Regressions |
112 | 112 |
113 During your shift, you should try to follow up on each of the bugs you filed. | 113 During your shift, you should try to follow up on each of the bugs you filed. |
114 Once you've triaged all the alerts, check to see if the bisects have come back, | 114 Once you've triaged all the alerts, check to see if the bisects have come back, |
115 or if they failed. If the results came back, and a culprit was found, follow up | 115 or if they failed. If the results came back, and a culprit was found, follow up |
116 with the CL author. If the bisects failed to update the bug with results, please | 116 with the CL author. If the bisects failed to update the bug with results, please |
117 file a bug on it (see [feedback](#feedback) links below). | 117 file a bug on it (see [feedback](#feedback) links below). |
118 | 118 |
119 Also during your shift, please spend any spare time driving down bugs from the | 119 Also during your shift, please spend any spare time driving down bugs from the |
120 [regression backlog](http://go/triage-backlog). Treat these bugs as you would | 120 [regression backlog](http://go/triage-backlog). Treat these bugs as you would |
121 your own - investigate the regressions, find out what the next step should be, | 121 your own - investigate the regressions, find out what the next step should be, |
122 and then move the bug along. As the backlog only contains bugs that haven't been | 122 and then move the bug along. As the backlog only contains bugs that haven't been |
123 modified in some time, you should be able to end your shift with an empty | 123 modified in some time, you should be able to end your shift with an empty |
124 backlog. | 124 backlog. |
125 | 125 |
126 After your shift, please try to follow up on the bugs you filed weekly. Kick off | 126 After your shift, please try to follow up on the bugs you filed weekly. Kick off |
127 new bisects if the previous ones failed, and if the bisect picks a likely | 127 new bisects if the previous ones failed, and if the bisect picks a likely |
128 culprit follow up to ensure the CL author addresses the problem. If you are | 128 culprit follow up to ensure the CL author addresses the problem. If you are |
129 certain that a specific CL caused a performance regression, and the author does | 129 certain that a specific CL caused a performance regression, and the author does |
130 not have an immediate plan to address the problem, please revert the CL. | 130 not have an immediate plan to address the problem, please revert the CL. |
131 | 131 |
132 ###<a name="feedback"></a> Give Feedback on our Infrastructure | 132 ##<a name="feedback"></a> Give Feedback on our Infrastructure |
133 | 133 |
134 Perf regression sheriffs have their eyes on the perf dashboard and bisects | 134 Perf regression sheriffs have their eyes on the perf dashboard and bisects |
135 more than anyone else, and their feedback is invaluable for making sure these | 135 more than anyone else, and their feedback is invaluable for making sure these |
136 tools are accurate and improving them. Please file bugs and feature requests | 136 tools are accurate and improving them. Please file bugs and feature requests |
137 as you see them: | 137 as you see them: |
138 | 138 |
139 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. | 139 * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. |
140 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit | 140 * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit |
141 or missing a clear culprit, or not reproducing what appears to be a clear | 141 or missing a clear culprit, or not reproducing what appears to be a clear |
142 regression, please link the comment the bisect bot posted on the bug at | 142 regression, please link the comment the bisect bot posted on the bug at |
143 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). | 143 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). |
144 The team triages these regularly. If you spot a really clear bug (bisect | 144 The team triages these regularly. If you spot a really clear bug (bisect |
145 job red, bugs not being updated with bisect results) please file it in | 145 job red, bugs not being updated with bisect results) please file it in |
146 crbug with label `Cr-Tests-AutoBisect`. | 146 crbug with label `Cr-Tests-AutoBisect`. |
147 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` | 147 * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` |
148 and [cc the owner](http://go/perf-owners). | 148 and [cc the owner](http://go/perf-owners). |
OLD | NEW |