Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(67)

Side by Side Diff: tools/perf/docs/perf_regression_sheriffing.md

Issue 2609893003: Add docs for dealing with resource_sizes perf alerts (Closed)
Patch Set: add note about bisecting rolls Created 3 years, 11 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « tools/perf/docs/apk_size_regressions.md ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Perf Regression Sheriffing (go/perfregression-sheriff) 1 # Perf Regression Sheriffing (go/perfregression-sheriff)
2 2
3 The perf regression sheriff tracks performance regressions in Chrome's 3 The perf regression sheriff tracks performance regressions in Chrome's
4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) 4 continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md)
5 has been created to ensure the builds and tests stay green, so the perf 5 has been created to ensure the builds and tests stay green, so the perf
6 regression sheriff role is now entirely focused on performance. 6 regression sheriff role is now entirely focused on performance.
7 7
8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_ 2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)** 8 **[Rotation calendar](https://calendar.google.com/calendar/embed?src=google.com_ 2fpmo740pd1unrui9d7cgpbg2k%40group.calendar.google.com)**
9 9
10 ## Key Responsibilities 10 ## Key Responsibilities
(...skipping 22 matching lines...) Expand all
33 33
34 The list can be sorted by clicking on the column header. When you click on the 34 The list can be sorted by clicking on the column header. When you click on the
35 checkbox next to an alert, all the other alerts that occurred in the same 35 checkbox next to an alert, all the other alerts that occurred in the same
36 revision range will be highlighted. 36 revision range will be highlighted.
37 37
38 Check the boxes next to the alerts you want to take a look at, and click the 38 Check the boxes next to the alerts you want to take a look at, and click the
39 "Graph" button. You'll be taken to a page with a table at the top listing all 39 "Graph" button. You'll be taken to a page with a table at the top listing all
40 the alerts that have an overlapping revision range with the one you chose, and 40 the alerts that have an overlapping revision range with the one you chose, and
41 below it the dashboard shows graphs of all the alerts checked in that table. 41 below it the dashboard shows graphs of all the alerts checked in that table.
42 42
43 1. **Look at the graph**. 43 1. **For alerts related to `resource_sizes`:**
44 * Refer to [apk_size_regressions.md](apk_size_regressions.md).
45 2. **Look at the graph**.
44 * If the alert appears to be **within the noise**, click on the red 46 * If the alert appears to be **within the noise**, click on the red
45 exclamation point icon for it in the graph and hit the "Report Invalid 47 exclamation point icon for it in the graph and hit the "Report Invalid
46 Alert" button. 48 Alert" button.
47 * If the alert appears to be **reverting a recent improvement**, click on 49 * If the alert appears to be **reverting a recent improvement**, click on
48 the red exclamation point icon for it in the graph and hit the "Ignore 50 the red exclamation point icon for it in the graph and hit the "Ignore
49 Valid Alert" button. 51 Valid Alert" button.
50 * If the alert is **visibly to the left or the right of the 52 * If the alert is **visibly to the left or the right of the
51 actual regression**, click on it and use the "nudge" menu to move it into 53 actual regression**, click on it and use the "nudge" menu to move it into
52 place. 54 place.
53 * If there is a line labeled "ref" on the graph, that is the reference build . 55 * If there is a line labeled "ref" on the graph, that is the reference build .
54 It's an older version of Chrome, used to help us sort out whether a change 56 It's an older version of Chrome, used to help us sort out whether a change
55 to the bot or test might have caused the graph to jump, rather than a real 57 to the bot or test might have caused the graph to jump, rather than a real
56 performance regression. If **the ref build moved at the same time as the 58 performance regression. If **the ref build moved at the same time as the
57 alert**, click on the alert and hit the "Report Invalid Alert" button. 59 alert**, click on the alert and hit the "Report Invalid Alert" button.
58 2. **Look at the other alerts** in the table to see if any should be grouped tog ether. 60 3. **Look at the other alerts** in the table to see if any should be grouped tog ether.
59 Note that the bisect will automatically dupe bugs if it finds they have the 61 Note that the bisect will automatically dupe bugs if it finds they have the
60 same culprit, so you don't need to be too aggressive about grouping alerts 62 same culprit, so you don't need to be too aggressive about grouping alerts
61 that might not be related. Some signs alerts should be grouped together: 63 that might not be related. Some signs alerts should be grouped together:
62 * If they're all in the same test suite 64 * If they're all in the same test suite
63 * If they all regressed the same metric (a lot of commonality in the Test 65 * If they all regressed the same metric (a lot of commonality in the Test
64 column) 66 column)
65 3. **Triage the group of alerts**. Check all the alerts you believe are related, 67 4. **Triage the group of alerts**. Check all the alerts you believe are related,
66 and press the triage button. 68 and press the triage button.
67 * If one of the alerts already has a bug id, click "existing bug" and use 69 * If one of the alerts already has a bug id, click "existing bug" and use
68 that bug id. 70 that bug id.
69 * Otherwise click "new bug". Be sure to cc the 71 * Otherwise click "new bug". Be sure to cc the
70 [test owner](http://go/perf-owners) on the bug. 72 [test owner](http://go/perf-owners) on the bug.
71 4. **Look at the revision range** for the regression. You can see it in the 73 5. **Look at the revision range** for the regression. You can see it in the
72 tooltip on the graph. If you see any likely culprits, cc the authors on the 74 tooltip on the graph. If you see any likely culprits, cc the authors on the
73 bug. 75 bug.
74 5. **Optionally, kick off more bisects**. The perf dashboard will automatically 76 6. **Optionally, kick off more bisects**. The perf dashboard will automatically
75 kick off a bisect for each bug you file. But if you think the regression is 77 kick off a bisect for each bug you file. But if you think the regression is
76 much clearer on one platform, or a specific page of a page set, or you want 78 much clearer on one platform, or a specific page of a page set, or you want
77 to see a broader revision range feel free to click on the alert on that graph 79 to see a broader revision range feel free to click on the alert on that graph
78 and kick off a bisect for it. There should be capacity to kick off as many 80 and kick off a bisect for it. There should be capacity to kick off as many
79 bisects as you feel are necessary to investigate; [give feedback](#feedback) 81 bisects as you feel are necessary to investigate; [give feedback](#feedback)
80 below if you feel that is not the case. 82 below if you feel that is not the case.
81 83
82 ## Triaging Data Stoppage Alerts 84 ## Triaging Data Stoppage Alerts
83 85
84 Data stoppage alerts are listed on the 86 Data stoppage alerts are listed on the
(...skipping 78 matching lines...) Expand 10 before | Expand all | Expand 10 after
163 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR 2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). 165 [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR 2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0).
164 The team triages these regularly. If you spot a really clear bug (bisect 166 The team triages these regularly. If you spot a really clear bug (bisect
165 job red, bugs not being updated with bisect results) please file it in 167 job red, bugs not being updated with bisect results) please file it in
166 crbug with component `Tests>AutoBisect`. If a bisect problem is blocking a 168 crbug with component `Tests>AutoBisect`. If a bisect problem is blocking a
167 perf regression bug triage, **please file a new bug with component 169 perf regression bug triage, **please file a new bug with component
168 `Tests>AutoBisect` and block the regression bug on the bisect bug**. This 170 `Tests>AutoBisect` and block the regression bug on the bisect bug**. This
169 makes it much easier for the team to triage, dupe, and close bugs on the 171 makes it much easier for the team to triage, dupe, and close bugs on the
170 infrastructure without affecting the state of the perf regression bugs. 172 infrastructure without affecting the state of the perf regression bugs.
171 * **Noisy Tests**: Please file a bug in crbug with component `Tests>Telemetry` 173 * **Noisy Tests**: Please file a bug in crbug with component `Tests>Telemetry`
172 and [cc the owner](http://go/perf-owners). 174 and [cc the owner](http://go/perf-owners).
OLDNEW
« no previous file with comments | « tools/perf/docs/apk_size_regressions.md ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698