Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(449)

Side by Side Diff: docs/speed/addressing_performance_regressions.md

Issue 2949593002: Add a note to the documentation on assuming your CL not is the culprit. (Closed)
Patch Set: Created 3 years, 6 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Addressing Performance Regressions 1 # Addressing Performance Regressions
2 2
3 The bisect bot just picked your CL as the culprit in a performance regression 3 The bisect bot just picked your CL as the culprit in a performance regression
4 and assigned a bug to you! What should you do? Read on... 4 and assigned a bug to you! What should you do? Read on...
5 5
6 [TOC] 6 [TOC]
7 7
8 ## About our performance tests 8 ## About our performance tests
9 9
10 The [chromium.perf waterfall](perf_waterfall.md) is a continuous build which 10 The [chromium.perf waterfall](perf_waterfall.md) is a continuous build which
(...skipping 100 matching lines...) Expand 10 before | Expand all | Expand 10 after
111 open a the trace that was recorded during the performance test. 111 open a the trace that was recorded during the performance test.
112 * **Wait, what's a trace?** See the 112 * **Wait, what's a trace?** See the
113 [documentation on tracing](https://www.chromium.org/developers/how-tos/trace -event-profiling-tool) 113 [documentation on tracing](https://www.chromium.org/developers/how-tos/trace -event-profiling-tool)
114 to learn how to use traces to debug performance issues. 114 to learn how to use traces to debug performance issues.
115 * **Are there debugging tips specific to certain benchmarks?** 115 * **Are there debugging tips specific to certain benchmarks?**
116 * **[Memory](https://chromium.googlesource.com/chromium/src/+/master/docs/me mory-infra/memory_benchmarks.md)** 116 * **[Memory](https://chromium.googlesource.com/chromium/src/+/master/docs/me mory-infra/memory_benchmarks.md)**
117 * **[Android binary size](apk_size_regressions.md)** 117 * **[Android binary size](apk_size_regressions.md)**
118 118
119 ## If you don't believe your CL could be the cause 119 ## If you don't believe your CL could be the cause
120 120
121 *** promo
sullivan 2017/06/19 16:56:23 This "promo" block is a gitiles markdown extension
122 Please remember that our performance tests exist to catch unexpected
123 regressions. Often, the tests catch performance problems the CL author was
124 not aware of. Please look at the data carefully and understand what the test
125 is measuring before concluding that your CL is not related.
126 ***
127
121 There are some clear reasons to believe the bisect bot made a mistake: 128 There are some clear reasons to believe the bisect bot made a mistake:
122 129
123 * Your CL changes a test or some code that isn't compiled on the platform 130 * Your CL changes a test or some code that isn't compiled on the platform
124 that regressed. 131 that regressed.
125 * Your CL is completely unrelated to the metric that regressed. 132 * Your CL is completely unrelated to the metric that regressed.
126 * You looked at the numbers the bisect spit out (see example above; the first 133 * You looked at the numbers the bisect spit out (see example above; the first
127 column is the revision, the second column is the value at that revision, 134 column is the revision, the second column is the value at that revision,
128 and the third column is the standard deviation), and: 135 and the third column is the standard deviation), and:
129 * The change attributed to your CL seems well within the noise, or 136 * The change attributed to your CL seems well within the noise, or
130 * The change at your CL is an improvement (for example, the metric is bytes 137 * The change at your CL is an improvement (for example, the metric is bytes
(...skipping 76 matching lines...) Expand 10 before | Expand all | Expand 10 after
207 possible that your change did not regress performance, and only appears to 214 possible that your change did not regress performance, and only appears to
208 be a problem because the test is measuring incorrectly. If this is the 215 be a problem because the test is measuring incorrectly. If this is the
209 case, you must explain clearly what the issue with the test is, and why you 216 case, you must explain clearly what the issue with the test is, and why you
210 believe your change is performance neutral. Please include data from traces 217 believe your change is performance neutral. Please include data from traces
211 or other performance tools to clarify your claim. 218 or other performance tools to clarify your claim.
212 219
213 **In all cases,** make sure to cc the [benchmark owner](https://docs.google.com/ spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0) 220 **In all cases,** make sure to cc the [benchmark owner](https://docs.google.com/ spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0)
214 when writing a justification and WontFix-ing a bug. If you cannot come to an 221 when writing a justification and WontFix-ing a bug. If you cannot come to an
215 agreement with the benchmark owner, you can escalate to benhenry@chromium.org, 222 agreement with the benchmark owner, you can escalate to benhenry@chromium.org,
216 the owner of speed releasing. 223 the owner of speed releasing.
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698