OLD | NEW |
---|---|
1 # Addressing Performance Regressions | 1 # Addressing Performance Regressions |
2 | 2 |
3 The bisect bot just picked your CL as the culprit in a performance regression | 3 The bisect bot just picked your CL as the culprit in a performance regression |
4 and assigned a bug to you! What should you do? Read on... | 4 and assigned a bug to you! What should you do? Read on... |
5 | 5 |
6 [TOC] | 6 [TOC] |
7 | 7 |
8 ## About our performance tests | 8 ## About our performance tests |
9 | 9 |
10 The [chromium.perf waterfall](perf_waterfall.md) is a continuous build which | 10 The [chromium.perf waterfall](perf_waterfall.md) is a continuous build which |
(...skipping 100 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... | |
111 open a the trace that was recorded during the performance test. | 111 open a the trace that was recorded during the performance test. |
112 * **Wait, what's a trace?** See the | 112 * **Wait, what's a trace?** See the |
113 [documentation on tracing](https://www.chromium.org/developers/how-tos/trace -event-profiling-tool) | 113 [documentation on tracing](https://www.chromium.org/developers/how-tos/trace -event-profiling-tool) |
114 to learn how to use traces to debug performance issues. | 114 to learn how to use traces to debug performance issues. |
115 * **Are there debugging tips specific to certain benchmarks?** | 115 * **Are there debugging tips specific to certain benchmarks?** |
116 * **[Memory](https://chromium.googlesource.com/chromium/src/+/master/docs/me mory-infra/memory_benchmarks.md)** | 116 * **[Memory](https://chromium.googlesource.com/chromium/src/+/master/docs/me mory-infra/memory_benchmarks.md)** |
117 * **[Android binary size](apk_size_regressions.md)** | 117 * **[Android binary size](apk_size_regressions.md)** |
118 | 118 |
119 ## If you don't believe your CL could be the cause | 119 ## If you don't believe your CL could be the cause |
120 | 120 |
121 *** promo | |
sullivan
2017/06/19 16:56:23
This "promo" block is a gitiles markdown extension
| |
122 Please remember that our performance tests exist to catch unexpected | |
123 regressions. Often, the tests catch performance problems the CL author was | |
124 not aware of. Please look at the data carefully and understand what the test | |
125 is measuring before concluding that your CL is not related. | |
126 *** | |
127 | |
121 There are some clear reasons to believe the bisect bot made a mistake: | 128 There are some clear reasons to believe the bisect bot made a mistake: |
122 | 129 |
123 * Your CL changes a test or some code that isn't compiled on the platform | 130 * Your CL changes a test or some code that isn't compiled on the platform |
124 that regressed. | 131 that regressed. |
125 * Your CL is completely unrelated to the metric that regressed. | 132 * Your CL is completely unrelated to the metric that regressed. |
126 * You looked at the numbers the bisect spit out (see example above; the first | 133 * You looked at the numbers the bisect spit out (see example above; the first |
127 column is the revision, the second column is the value at that revision, | 134 column is the revision, the second column is the value at that revision, |
128 and the third column is the standard deviation), and: | 135 and the third column is the standard deviation), and: |
129 * The change attributed to your CL seems well within the noise, or | 136 * The change attributed to your CL seems well within the noise, or |
130 * The change at your CL is an improvement (for example, the metric is bytes | 137 * The change at your CL is an improvement (for example, the metric is bytes |
(...skipping 76 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... | |
207 possible that your change did not regress performance, and only appears to | 214 possible that your change did not regress performance, and only appears to |
208 be a problem because the test is measuring incorrectly. If this is the | 215 be a problem because the test is measuring incorrectly. If this is the |
209 case, you must explain clearly what the issue with the test is, and why you | 216 case, you must explain clearly what the issue with the test is, and why you |
210 believe your change is performance neutral. Please include data from traces | 217 believe your change is performance neutral. Please include data from traces |
211 or other performance tools to clarify your claim. | 218 or other performance tools to clarify your claim. |
212 | 219 |
213 **In all cases,** make sure to cc the [benchmark owner](https://docs.google.com/ spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0) | 220 **In all cases,** make sure to cc the [benchmark owner](https://docs.google.com/ spreadsheets/d/1xaAo0_SU3iDfGdqDJZX_jRV0QtkufwHUKH3kQKF3YQs/edit#gid=0) |
214 when writing a justification and WontFix-ing a bug. If you cannot come to an | 221 when writing a justification and WontFix-ing a bug. If you cannot come to an |
215 agreement with the benchmark owner, you can escalate to benhenry@chromium.org, | 222 agreement with the benchmark owner, you can escalate to benhenry@chromium.org, |
216 the owner of speed releasing. | 223 the owner of speed releasing. |
OLD | NEW |