OLD | NEW |
1 # Perf Regression Sheriffing (go/perfregression-sheriff) | 1 # Perf Regression Sheriffing (go/perfregression-sheriff) |
2 | 2 |
3 The perf regression sheriff tracks performance regressions in Chrome's | 3 The perf regression sheriff tracks performance regressions in Chrome's |
4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) | 4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) |
5 has been created to ensure the builds and tests stay green, so the perf | 5 has been created to ensure the builds and tests stay green, so the perf |
6 regression sheriff role is now entirely focused on performance. | 6 regression sheriff role is now entirely focused on performance. |
7 | 7 |
8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_
2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** | 8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_
2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** |
9 | 9 |
10 ## Key Responsibilities | 10 ## Key Responsibilities |
(...skipping 22 matching lines...) Expand all Loading... |
33 | 33 |
34 The list can be sorted by clicking on the column header. When you click on the | 34 The list can be sorted by clicking on the column header. When you click on the |
35 checkbox next to an alert, all the other alerts that occurred in the same | 35 checkbox next to an alert, all the other alerts that occurred in the same |
36 revision range will be highlighted. | 36 revision range will be highlighted. |
37 | 37 |
38 Check the boxes next to the alerts you want to take a look at, and click the | 38 Check the boxes next to the alerts you want to take a look at, and click the |
39 "Graph" button. You'll be taken to a page with a table at the top listing all | 39 "Graph" button. You'll be taken to a page with a table at the top listing all |
40 the alerts that have an overlapping revision range with the one you chose, and | 40 the alerts that have an overlapping revision range with the one you chose, and |
41 below it the dashboard shows graphs of all the alerts checked in that table. | 41 below it the dashboard shows graphs of all the alerts checked in that table. |
42 | 42 |
43 1. **Look at the graph**. | 43 1. **For alerts related to `resource_sizes`:** |
| 44 * Refer to [apk_size_regressions.md](apk_size_regressions.md). |
| 45 2. **Look at the graph**. |
44 * If the alert appears to be **within the noise**, click on the red | 46 * If the alert appears to be **within the noise**, click on the red |
45 exclamation point icon for it in the graph and hit the "Report Invalid | 47 exclamation point icon for it in the graph and hit the "Report Invalid |
46 Alert" button. | 48 Alert" button. |
47 * If the alert appears to be **reverting a recent improvement**, click on | 49 * If the alert appears to be **reverting a recent improvement**, click on |
48 the red exclamation point icon for it in the graph and hit the "Ignore | 50 the red exclamation point icon for it in the graph and hit the "Ignore |
49 Valid Alert" button. | 51 Valid Alert" button. |
50 * If the alert is **visibly to the left or the right of the | 52 * If the alert is **visibly to the left or the right of the |
51 actual regression**, click on it and use the "nudge" menu to move it into | 53 actual regression**, click on it and use the "nudge" menu to move it into |
52 place. | 54 place. |
53 * If there is a line labeled "ref" on the graph, that is the reference build
. | 55 * If there is a line labeled "ref" on the graph, that is the reference build
. |
54 It's an older version of Chrome, used to help us sort out whether a change | 56 It's an older version of Chrome, used to help us sort out whether a change |
55 to the bot or test might have caused the graph to jump, rather than a real | 57 to the bot or test might have caused the graph to jump, rather than a real |
56 performance regression. If **the ref build moved at the same time as the | 58 performance regression. If **the ref build moved at the same time as the |
57 alert**, click on the alert and hit the "Report Invalid Alert" button. | 59 alert**, click on the alert and hit the "Report Invalid Alert" button. |
58 2. **Look at the other alerts** in the table to see if any should be grouped tog
ether. | 60 3. **Look at the other alerts** in the table to see if any should be grouped tog
ether. |
59 Note that the bisect will automatically dupe bugs if it finds they have the | 61 Note that the bisect will automatically dupe bugs if it finds they have the |
60 same culprit, so you don't need to be too aggressive about grouping alerts | 62 same culprit, so you don't need to be too aggressive about grouping alerts |
61 that might not be related. Some signs alerts should be grouped together: | 63 that might not be related. Some signs alerts should be grouped together: |
62 * If they're all in the same test suite | 64 * If they're all in the same test suite |
63 * If they all regressed the same metric (a lot of commonality in the Test | 65 * If they all regressed the same metric (a lot of commonality in the Test |
64 column) | 66 column) |
65 3. **Triage the group of alerts**. Check all the alerts you believe are related, | 67 4. **Triage the group of alerts**. Check all the alerts you believe are related, |
66 and press the triage button. | 68 and press the triage button. |
67 * If one of the alerts already has a bug id, click "existing bug" and use | 69 * If one of the alerts already has a bug id, click "existing bug" and use |
68 that bug id. | 70 that bug id. |
69 * Otherwise click "new bug". Be sure to cc the | 71 * Otherwise click "new bug". Be sure to cc the |
70 [test owner](http://go/perf-owners) on the bug. | 72 [test owner](http://go/perf-owners) on the bug. |
71 4. **Look at the revision range** for the regression. You can see it in the | 73 5. **Look at the revision range** for the regression. You can see it in the |
72 tooltip on the graph. If you see any likely culprits, cc the authors on the | 74 tooltip on the graph. If you see any likely culprits, cc the authors on the |
73 bug. | 75 bug. |
74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically | 76 6. **Optionally, kick off more bisects**. The perf dashboard will automatically |
75 kick off a bisect for each bug you file. But if you think the regression is | 77 kick off a bisect for each bug you file. But if you think the regression is |
76 much clearer on one platform, or a specific page of a page set, or you want | 78 much clearer on one platform, or a specific page of a page set, or you want |
77 to see a broader revision range feel free to click on the alert on that graph | 79 to see a broader revision range feel free to click on the alert on that graph |
78 and kick off a bisect for it. There should be capacity to kick off as many | 80 and kick off a bisect for it. There should be capacity to kick off as many |
79 bisects as you feel are necessary to investigate; [give feedback](#feedback) | 81 bisects as you feel are necessary to investigate; [give feedback](#feedback) |
80 below if you feel that is not the case. | 82 below if you feel that is not the case. |
81 | 83 |
82 ## Triaging Data Stoppage Alerts | 84 ## Triaging Data Stoppage Alerts |
83 | 85 |
84 Data stoppage alerts are listed on the | 86 Data stoppage alerts are listed on the |
(...skipping 78 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). | 165 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR
2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). |
164 The team triages these regularly. If you spot a really clear bug (bisect | 166 The team triages these regularly. If you spot a really clear bug (bisect |
165 job red, bugs not being updated with bisect results) please file it in | 167 job red, bugs not being updated with bisect results) please file it in |
166 crbug with component `Tests>AutoBisect`. If a bisect problem is blocking a | 168 crbug with component `Tests>AutoBisect`. If a bisect problem is blocking a |
167 perf regression bug triage, **please file a new bug with component | 169 perf regression bug triage, **please file a new bug with component |
168 `Tests>AutoBisect` and block the regression bug on the bisect bug**. This | 170 `Tests>AutoBisect` and block the regression bug on the bisect bug**. This |
169 makes it much easier for the team to triage, dupe, and close bugs on the | 171 makes it much easier for the team to triage, dupe, and close bugs on the |
170 infrastructure without affecting the state of the perf regression bugs. | 172 infrastructure without affecting the state of the perf regression bugs. |
171 * **Noisy Tests**: Please file a bug in crbug with component `Tests>Telemetry` | 173 * **Noisy Tests**: Please file a bug in crbug with component `Tests>Telemetry` |
172 and [cc the owner](http://go/perf-owners). | 174 and [cc the owner](http://go/perf-owners). |
OLD | NEW |