OLD | NEW |
1 # How Chrome Measures Performance | 1 # How Chrome Measures Performance |
2 | 2 |
3 Chrome collects performance data both in the lab, and from end users. There are | 3 Chrome collects performance data both in the lab, and from end users. There are |
4 thousands of individual metrics. This is an overview of how to sort through | 4 thousands of individual metrics. This is an overview of how to sort through |
5 them at a high level. | 5 them at a high level. |
6 | 6 |
7 ## Tracks and Metrics | 7 ## Tracks and Metrics |
8 | 8 |
9 At a high level, performance work in Chrome is categorized into **tracks**, | 9 At a high level, performance work in Chrome is categorized into **tracks**, |
10 like loading, memory, and power. Each track has high-level metrics associated | 10 like loading, memory, and power. Each track has high-level metrics associated |
11 with it. | 11 with it. |
12 | 12 |
13 * **[An overview of tracks](performance_tracks.md)**: lists the tracks and key
contact points. | 13 * **[An overview of tracks](performance_tracks.md)**: lists the tracks and key
contact points. |
14 * **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit): | 14 * **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit)**: |
15 the important high-level metrics we measure for each track. | 15 the important high-level metrics we measure for each track. |
16 | 16 |
17 ## Laboratory Metrics | 17 ## Laboratory Metrics |
18 | 18 |
19 Chrome has multiple performance labs in which benchmarks are run on continuous | 19 Chrome has multiple performance labs in which benchmarks are run on continuous |
20 builds to pinpoint performance regressions down to individual changelists. | 20 builds to pinpoint performance regressions down to individual changelists. |
21 | 21 |
22 ### The chromium.perf lab | 22 ### The chromium.perf lab |
23 | 23 |
24 The main lab for performance monitoring is chromium.perf. It continuously tests | 24 The main lab for performance monitoring is chromium.perf. It continuously tests |
25 chromium commits and is monitored by the perf sheriff rotation. | 25 chromium commits and is monitored by the perf sheriff rotation. |
26 | 26 |
27 * **[What is the perf waterfall?](perf_waterfall.md)** An overview of the | 27 * **[What is the perf waterfall?](perf_waterfall.md)** An overview of the |
28 waterfall that runs the continuous build. | 28 waterfall that runs the continuous build. |
29 * **[How telemetry works](https://github.com/catapult-project/catapult/blob/ma
ster/telemetry/README.md): | 29 * **[How telemetry works](https://github.com/catapult-project/catapult/blob/ma
ster/telemetry/README.md)**: |
30 An overview of telemetry, our performance testing harness. | 30 An overview of telemetry, our performance testing harness. |
31 * **[How perf bisects work](bisects.md): An overview of the bisect bots, | 31 * **[How perf bisects work](bisects.md)**: An overview of the bisect bots, |
32 which narrow down regressions over a CL range to a specific commit. | 32 which narrow down regressions over a CL range to a specific commit. |
33 * **Benchmarks* | 33 * **Benchmarks** |
34 * **[Benchmark Policy](https://docs.google.com/document/d/1ni2MIeVnlH4bTj4yv
EDMVNxgL73PqK_O9_NUm3NW3BA/edit)**: | 34 * **[Benchmark Policy](https://docs.google.com/document/d/1ni2MIeVnlH4bTj4yv
EDMVNxgL73PqK_O9_NUm3NW3BA/edit)**: |
35 An overview of the benchmark harnesses available in Chrome, and how to | 35 An overview of the benchmark harnesses available in Chrome, and how to |
36 find the right place to add a new test case. | 36 find the right place to add a new test case. |
37 * **[System health benchmarks](https://docs.google.com/document/d/1BM_6lBrPz
pMNMtcyi2NFKGIzmzIQ1oH3OlNG27kDGNU/edit?ts=57e92782)**: | 37 * **[System health benchmarks](https://docs.google.com/document/d/1BM_6lBrPz
pMNMtcyi2NFKGIzmzIQ1oH3OlNG27kDGNU/edit?ts=57e92782)**: |
38 The system health benchmarks measure the speed launch metrics on | 38 The system health benchmarks measure the speed launch metrics on |
39 real-world web use scenarios. | 39 real-world web use scenarios. |
40 * **[How to run on perf trybots](perf_trybots.md)**: Have an unsubmitted | 40 * **[How to run on perf trybots](perf_trybots.md)**: Have an unsubmitted |
41 CL and want to run benchmarks on it? Need to try a variety of hardware and | 41 CL and want to run benchmarks on it? Need to try a variety of hardware and |
42 operating systems? Use the perf trybots. | 42 operating systems? Use the perf trybots. |
43 * **[How to run telemetry locally](https://github.com/catapult-project/catapul
t/blob/master/telemetry/docs/run_benchmarks_locally.md)**: | 43 * **[How to run telemetry locally](https://github.com/catapult-project/catapul
t/blob/master/telemetry/docs/run_benchmarks_locally.md)**: |
(...skipping 10 matching lines...) Expand all Loading... |
54 * **[Lab Spotlight: Cluster telemetry](https://docs.google.com/document/d/1Ghq
osQcwsy6F-eBAmFn_ITDF7_Iv_rY9FhCKwAnk9qQ/edit)**: | 54 * **[Lab Spotlight: Cluster telemetry](https://docs.google.com/document/d/1Ghq
osQcwsy6F-eBAmFn_ITDF7_Iv_rY9FhCKwAnk9qQ/edit)**: |
55 Need to run a performance test over thousands of pages? Check out cluster | 55 Need to run a performance test over thousands of pages? Check out cluster |
56 telemetry! | 56 telemetry! |
57 | 57 |
58 ## End-user metrics | 58 ## End-user metrics |
59 | 59 |
60 The **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit)** | 60 The **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit)** |
61 doc explains metrics available in UMA for end user performance. If you want to | 61 doc explains metrics available in UMA for end user performance. If you want to |
62 test how your change impacts these metrics for end users, you'll probably want | 62 test how your change impacts these metrics for end users, you'll probably want |
63 to **[Run a Finch Trial](http://goto.google.com/finch101)**. | 63 to **[Run a Finch Trial](http://goto.google.com/finch101)**. |
OLD | NEW |