OLD | NEW |
(Empty) | |
| 1 # How Chrome Measures Performance |
| 2 |
| 3 Chrome collects performance data both in the lab, and from end users. There are |
| 4 thousands of individual metrics. This is an overview of how to sort through |
| 5 them at a high level. |
| 6 |
| 7 ## Tracks and Metrics |
| 8 |
| 9 At a high level, performance work in Chrome is categorized into **tracks**, |
| 10 like loading, memory, and power. Each track has high-level metrics associated |
| 11 with it. |
| 12 |
| 13 * **[An overview of tracks](performance_tracks.md)**: lists the tracks and key
contact points. |
| 14 * **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit): |
| 15 the important high-level metrics we measure for each track. |
| 16 |
| 17 ## Laboratory Metrics |
| 18 |
| 19 Chrome has multiple performance labs in which benchmarks are run on continuous |
| 20 builds to pinpoint performance regressions down to individual changelists. |
| 21 |
| 22 ### The chromium.perf lab |
| 23 |
| 24 The main lab for performance monitoring is chromium.perf. It continuously tests |
| 25 chromium commits and is monitored by the perf sheriff rotation. |
| 26 |
| 27 * **[What is the perf waterfall?](perf_waterfall.md)** An overview of the |
| 28 waterfall that runs the continuous build. |
| 29 * **[How telemetry works](https://github.com/catapult-project/catapult/blob/ma
ster/telemetry/README.md): |
| 30 An overview of telemetry, our performance testing harness. |
| 31 * **[How perf bisects work](bisects.md): An overview of the bisect bots, |
| 32 which narrow down regressions over a CL range to a specific commit. |
| 33 * **Benchmarks* |
| 34 * **[Benchmark Policy](https://docs.google.com/document/d/1ni2MIeVnlH4bTj4yv
EDMVNxgL73PqK_O9_NUm3NW3BA/edit)**: |
| 35 An overview of the benchmark harnesses available in Chrome, and how to |
| 36 find the right place to add a new test case. |
| 37 * **[System health benchmarks](https://docs.google.com/document/d/1BM_6lBrPz
pMNMtcyi2NFKGIzmzIQ1oH3OlNG27kDGNU/edit?ts=57e92782)**: |
| 38 The system health benchmarks measure the speed launch metrics on |
| 39 real-world web use scenarios. |
| 40 * **[How to run on perf trybots](perf_trybots.md)**: Have an unsubmitted |
| 41 CL and want to run benchmarks on it? Need to try a variety of hardware and |
| 42 operating systems? Use the perf trybots. |
| 43 * **[How to run telemetry locally](https://github.com/catapult-project/catapul
t/blob/master/telemetry/docs/run_benchmarks_locally.md)**: |
| 44 Instructions on running telemetry benchmarks on your local machine. |
| 45 * **[List of platforms in the lab](perf_lab_platforms.md)**: Devices, |
| 46 configurations, and OSes the chromium.perf lab tests on. |
| 47 |
| 48 ### Other performance labs |
| 49 |
| 50 There are several other performance labs for specialized use: |
| 51 |
| 52 * **[Lab Spotlight: AV Lab (Googlers only)](http://goto.google.com/av-analysis
-service)**: |
| 53 Learn all about audio/video quality testing. |
| 54 * **[Lab Spotlight: Cluster telemetry](https://docs.google.com/document/d/1Ghq
osQcwsy6F-eBAmFn_ITDF7_Iv_rY9FhCKwAnk9qQ/edit)**: |
| 55 Need to run a performance test over thousands of pages? Check out cluster |
| 56 telemetry! |
| 57 |
| 58 ## End-user metrics |
| 59 |
| 60 The **[Speed Launch Metrics](https://docs.google.com/document/d/1Ww487ZskJ-xBmJG
wPO-XPz_QcJvw-kSNffm0nPhVpj8/edit)** |
| 61 doc explains metrics available in UMA for end user performance. If you want to |
| 62 test how your change impacts these metrics for end users, you'll probably want |
| 63 to **[Run a Finch Trial](http://goto.google.com/finch101)**. |
OLD | NEW |