Index: tools/perf/docs/perf_regression_sheriffing.md |
diff --git a/tools/perf/docs/perf_regression_sheriffing.md b/tools/perf/docs/perf_regression_sheriffing.md |
new file mode 100644 |
index 0000000000000000000000000000000000000000..da54d00a6403b07756ac17169eaaa4fbf793f492 |
--- /dev/null |
+++ b/tools/perf/docs/perf_regression_sheriffing.md |
@@ -0,0 +1,97 @@ |
+# Perf Regression Sheriffing |
+ |
+The perf regression sheriff tracks performance regressions in Chrome's |
+continuous integration tests. Note that a [new rotation](perf_bot_sheriffing.md) |
+has been created to ensure the builds and tests stay green, so the perf |
+regression sheriff role is now entirely focused on performance. |
+ |
+## Key Responsibilities |
+ |
+ * [Triage Regressions on the Perf Dashboard](#triage) |
+ * [Follow up on Performance Regressions](#followup) |
+ * [Give Feedback on our Infrastructure](#feedback) |
+ |
+###<a name="triage"></a> Triage Regressions on the Perf Dashboard |
+ |
+Open the perf dashboard [alerts page](https://chromeperf.appspot.com/alerts). |
+ |
+In the upper right corner, **sign in with your Google account**. Signing in is |
+important in order to be able to kick off bisect jobs, and see data from |
+internal waterfalls. |
+ |
+The page shows two lists; you are responsible for triaging |
+**Performance Alerts**. The list can be sorted by clicking on the column header. |
+When you click on the checkbox next to an alert, all the other alerts that |
+occurred in the same revision range will be highlighted. |
+ |
+Check the boxes next to the alerts you want to take a look at, and click the |
+"Graph" button. You'll be taken to a page with a table at the top listing all |
+the alerts that have an overlapping revision range with the one you chose, and |
+below it the dashboard shows graphs of all the alerts checked in that table. |
+ |
+1. **Look at the graph**. |
+ * If the alert appears to be **within the noise**, click on the red |
+ exclamation point icon for it in the graph and hit the "Invalid" button. |
+ * If the alert is **if the alert is visibly to the left or the right of the |
+ actual regression**, click on it and use the "nudge" menu to move it into |
+ place. |
+ * If there is a line labeled "ref" on the graph, that is the reference build. |
+ It's an older version of Chrome, used to help us sort out whether a change |
+ to the bot or test might have caused the graph to jump, rather than a real |
+ performance regression. If **the ref build moved at the same time as the |
+ alert**, click on the alert and hit the "Invalid" button. |
+2. **Look at the other alerts** in the table to see if any should be grouped together. |
+ Note that the bisect will automatically dupe bugs if it finds they have the |
+ same culprit, so you don't need to be too aggressive about grouping alerts |
+ that might not be related. Some signs alerts should be grouped together: |
+ * If they're all in the same test suite |
+ * If they all regressed the same metric (a lot of commonality in the Test |
+ column) |
+3. **Triage the group of alerts**. Check all the alerts you believe are related, |
+ and press the triage button. |
+ * If one of the alerts already has a bug id, click "existing bug" and use |
+ that bug id. |
+ * Otherwise click "new bug". Be sure to cc the |
+ [test owner](http://go/perf-owners) on the bug. |
+4. **Look at the revision range** for the regression. You can see it in the |
+ tooltip on the graph. If you see any likely culprits, cc the authors on the |
+ bug. |
+5. **Optionally, kick off more bisects**. The perf dashboard will automatically |
+ kick off a bisect for each bug you file. But if you think the regression is |
+ much clearer on one platform, or a specific page of a page set, or you want |
+ to see a broader revision range feel free to click on the alert on that graph |
+ and kick off a bisect for it. There should be capacity to kick off as many |
+ bisects as you feel are necessary to investigate; [give feedback](#feedback) |
+ below if you feel that is not the case. |
+ |
+###<a name="followup"></a> Follow up on Performance Regressions |
+ |
+During your shift, you should try to follow up on each of the bugs you filed. |
+Once you've triaged all the alerts, check to see if the bisects have come back, |
+or if they failed. If the results came back, and a culprit was found, follow up |
+with the CL author. If the bisects failed to update the bug with results, please |
+file a bug on it (see [feedback](#feedback) links below). |
+ |
+After your shift, please try to follow up on the bugs you filed weekly. Kick off |
+new bisects if the previous ones failed, and if the bisect picks a likely |
+culprit follow up to ensure the CL author addresses the problem. If you are |
+certain that a specific CL caused a performance regression, and the author does |
+not have an immediate plan to address the problem, please revert the CL. |
+ |
+###<a name="feedback"></a> Give Feedback on our Infrastructure |
+ |
+Perf regression sheriffs have their eyes on the perf dashboard and bisects |
+more than anyone else, and their feedback is invaluable for making sure these |
+tools are accurate and improving them. Please file bugs and feature requests |
+as you see them: |
+ |
+ * **Perf Dashboard**: Please use the red "Report Issue" link in the navbar. |
+ * **Perf Bisect/Trybots**: If a bisect is identifying the wrong CL as culprit |
+ or missing a clear culprit, or not reproducing what appears to be a clear |
+ regression, please link the comment the bisect bot posted on the bug at |
+ [go/bad-bisects](https://docs.google.com/spreadsheets/d/13PYIlRGE8eZzsrSocA3SR2LEHdzc8n9ORUoOE2vtO6I/edit#gid=0). |
+ The team triages these regularly. If you spot a really clear bug (bisect |
+ job red, bugs not being updated with bisect results) please file it in |
+ crbug with label `Cr-Tests-AutoBisect`. |
+ * **Noisy Tests**: Please file a bug in crbug with label `Cr-Tests-Telemetry` |
+ and [cc the owner](http://go/perf-owners). |