| OLD | NEW |
| 1 # Blink Performance Tests | 1 # Blink Performance Tests |
| 2 | 2 |
| 3 [TOC] | 3 [TOC] |
| 4 | 4 |
| 5 ## Overview | 5 ## Overview |
| 6 | 6 |
| 7 Blink perf tests are used for micro benchmarking the surface of Blink that | 7 Blink perf tests are used for micro benchmarking the surface of Blink that |
| 8 is exposed to the Web. They are the counterpart of [LayoutTests/] | 8 is exposed to the Web. They are the counterpart of [LayoutTests/](../../../docs/
testing/layout_tests.md) |
| 9 (https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_tes
ts.md) | |
| 10 but for performance coverage. | 9 but for performance coverage. |
| 11 | 10 |
| 12 ## Writing Tests | 11 ## Writing Tests |
| 13 Each test entry point is a HTML file written using | 12 Each test entry point is a HTML file written using |
| 14 [runner.js](resources/runner.js) | 13 [runner.js](resources/runner.js) |
| 15 testing framework. The test file is placed inside a sub folder of | 14 testing framework. The test file is placed inside a sub folder of |
| 16 [Blink/PerformanceTests/](.) | 15 [Blink/PerformanceTests/](.) |
| 17 and is started by importing `runner.js` script into the document: | 16 and is started by importing `runner.js` script into the document: |
| 18 ``` | 17 ``` |
| 19 <script src="../resources/runner.js"></script> | 18 <script src="../resources/runner.js"></script> |
| (...skipping 22 matching lines...) Expand all Loading... |
| 42 setup: function () { ... }, // test setup logic, called once before each run | 41 setup: function () { ... }, // test setup logic, called once before each run |
| 43 run: function () { ... }, // contains the code to benchmark | 42 run: function () { ... }, // contains the code to benchmark |
| 44 iterationCount: 5 // repeat the test 5 times | 43 iterationCount: 5 // repeat the test 5 times |
| 45 }); | 44 }); |
| 46 ``` | 45 ``` |
| 47 | 46 |
| 48 In the case of `PerfTestRunner.measureRunsPerSecond`, each run invokes | 47 In the case of `PerfTestRunner.measureRunsPerSecond`, each run invokes |
| 49 `test.run` multiple times. | 48 `test.run` multiple times. |
| 50 | 49 |
| 51 **Tracing support** | 50 **Tracing support** |
| 51 |
| 52 When the test is run through Telemetry, you can also collect timing of trace | 52 When the test is run through Telemetry, you can also collect timing of trace |
| 53 events that happen during each run by specifying `tracingCategories` & | 53 events that happen during each run by specifying `tracingCategories` & |
| 54 `traceEventsToMeasure` in the test object. For example: | 54 `traceEventsToMeasure` in the test object. For example: |
| 55 | 55 |
| 56 ``` | 56 ``` |
| 57 PerfTestRunner.measureTime({ | 57 PerfTestRunner.measureTime({ |
| 58 ... | 58 ... |
| 59 run: foo, | 59 run: foo, |
| 60 iterationCount: 3, | 60 iterationCount: 3, |
| 61 tracingCategories: 'blink', | 61 tracingCategories: 'blink', |
| (...skipping 13 matching lines...) Expand all Loading... |
| 75 v1 v2 v3 v4 v5 v6 | 75 v1 v2 v3 v4 v5 v6 |
| 76 ``` | 76 ``` |
| 77 | 77 |
| 78 Besides outputting timeseries `[u1, u2, u3]`, telemetry perf test runner will | 78 Besides outputting timeseries `[u1, u2, u3]`, telemetry perf test runner will |
| 79 also compute the total CPU times for trace events 'A' & 'B' per `foo()` run: | 79 also compute the total CPU times for trace events 'A' & 'B' per `foo()` run: |
| 80 | 80 |
| 81 * CPU times of trace events A: `[v0 + v2, v4, 0.0]` | 81 * CPU times of trace events A: `[v0 + v2, v4, 0.0]` |
| 82 * CPU times of trace events B: `[0.0, v3, v5]` | 82 * CPU times of trace events B: `[0.0, v3, v5]` |
| 83 | 83 |
| 84 Example tracing synchronous tests: | 84 Example tracing synchronous tests: |
| 85 [append-child-measure-time.html](TestData/append-child-measure-time.html) | 85 |
| 86 [simple-html-measure-page-load-time.html](TestData/simple-html-measure-page-load
-time.html) | 86 * [append-child-measure-time.html](TestData/append-child-measure-time.html) |
| 87 |
| 88 * [simple-html-measure-page-load-time.html](TestData/simple-html-measure-page-
load-time.html) |
| 87 | 89 |
| 88 | 90 |
| 89 ### Asynchronous Perf Tests | 91 ### Asynchronous Perf Tests |
| 90 In asynchronous perf test, you define your test scheduler and do your own | 92 In asynchronous perf test, you define your test scheduler and do your own |
| 91 measurement. For example: | 93 measurement. For example: |
| 92 | 94 |
| 93 ``` | 95 ``` |
| 94 var isDone = false; | 96 var isDone = false; |
| 95 var startTime; | 97 var startTime; |
| 96 | 98 |
| (...skipping 22 matching lines...) Expand all Loading... |
| 119 }); | 121 }); |
| 120 ``` | 122 ``` |
| 121 | 123 |
| 122 In the example above, the call | 124 In the example above, the call |
| 123 `PerfTestRunner.measureValueAsync(value)` send the metric of a single run to | 125 `PerfTestRunner.measureValueAsync(value)` send the metric of a single run to |
| 124 the test runner and also let the runner know that it has finished a single run. | 126 the test runner and also let the runner know that it has finished a single run. |
| 125 Once the number of run reaches `iterationCount` (6 in the example above), the | 127 Once the number of run reaches `iterationCount` (6 in the example above), the |
| 126 `done` callback is invoked, setting the your test state to finished. | 128 `done` callback is invoked, setting the your test state to finished. |
| 127 | 129 |
| 128 **Tracing support** | 130 **Tracing support** |
| 131 |
| 129 Like synchronous perf tests, tracing metrics are only available when you run | 132 Like synchronous perf tests, tracing metrics are only available when you run |
| 130 your tests with Telemetry. | 133 your tests with Telemetry. |
| 131 | 134 |
| 132 Unlike synchronous perf tests which the test runner framework handles test | 135 Unlike synchronous perf tests which the test runner framework handles test |
| 133 scheduling and tracing coverage for you, for most asynchronous tests, you need | 136 scheduling and tracing coverage for you, for most asynchronous tests, you need |
| 134 to manually mark when the async test begins | 137 to manually mark when the async test begins |
| 135 (`PerfTestRunner.addRunTestStartMarker`) and ends | 138 (`PerfTestRunner.addRunTestStartMarker`) and ends |
| 136 (`PerfTestRunner.addRunTestEndMarker`). Once those are marked, specifying | 139 (`PerfTestRunner.addRunTestEndMarker`). Once those are marked, specifying |
| 137 `tracingCategories` and `traceEventsToMeasure` will output CPU time metrics | 140 `tracingCategories` and `traceEventsToMeasure` will output CPU time metrics |
| 138 of trace events that happen during test runs in the fashion similar to the | 141 of trace events that happen during test runs in the fashion similar to the |
| 139 example of synchronous tracing test above. | 142 example of synchronous tracing test above. |
| 140 | 143 |
| 141 Example of tracing asynchronous tests: | 144 Example of tracing asynchronous tests: |
| 145 |
| 142 [color-changes-measure-frame-time.html](TestData/color-changes-measure-frame-tim
e.html) | 146 [color-changes-measure-frame-time.html](TestData/color-changes-measure-frame-tim
e.html) |
| 147 |
| 143 [simple-blob-measure-async.html](TestData/simple-blob-measure-async.html) | 148 [simple-blob-measure-async.html](TestData/simple-blob-measure-async.html) |
| 144 | 149 |
| 145 | 150 |
| 146 ## Running Tests | 151 ## Running Tests |
| 147 | 152 |
| 148 ** Running tests directly in browser ** | 153 ** Running tests directly in browser ** |
| 149 Most of Blink Performance tests should be runnable by just open the test file | 154 Most of Blink Performance tests should be runnable by just open the test file |
| 150 directly in the browser. However, features like tracing metrics & HTML results | 155 directly in the browser. However, features like tracing metrics & HTML results |
| 151 viewer won't be supported. | 156 viewer won't be supported. |
| 152 | 157 |
| 153 ** Running tests with Telemetry ** | 158 ** Running tests with Telemetry ** |
| 154 Assuming your current directory is chromium/src/, you can run tests with: | 159 Assuming your current directory is chromium/src/, you can run tests with: |
| 160 |
| 155 `./tools/perf/run_benchmark blink_perf [--test-path=<path to your tests>]` | 161 `./tools/perf/run_benchmark blink_perf [--test-path=<path to your tests>]` |
| 156 | 162 |
| 157 For information about all supported options, run: | 163 For information about all supported options, run: |
| 164 |
| 158 `./tools/perf/run_benchmark blink_perf --help` | 165 `./tools/perf/run_benchmark blink_perf --help` |
| OLD | NEW |