OLD | NEW |
1 # Blink Performance Tests | 1 # Blink Performance Tests |
2 | 2 |
3 [TOC] | 3 [TOC] |
4 | 4 |
5 ## Overview | 5 ## Overview |
6 | 6 |
7 Blink perf tests are used for micro benchmarking the surface of Blink that | 7 Blink perf tests are used for micro benchmarking the surface of Blink that |
8 is exposed to the Web. They are the counterpart of [LayoutTests/] | 8 is exposed to the Web. They are the counterpart of [LayoutTests/](../../../docs/
testing/layout_tests.md) |
9 (https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_tes
ts.md) | |
10 but for performance coverage. | 9 but for performance coverage. |
11 | 10 |
12 ## Writing Tests | 11 ## Writing Tests |
13 Each test entry point is a HTML file written using | 12 Each test entry point is a HTML file written using |
14 [runner.js](resources/runner.js) | 13 [runner.js](resources/runner.js) |
15 testing framework. The test file is placed inside a sub folder of | 14 testing framework. The test file is placed inside a sub folder of |
16 [Blink/PerformanceTests/](.) | 15 [Blink/PerformanceTests/](.) |
17 and is started by importing `runner.js` script into the document: | 16 and is started by importing `runner.js` script into the document: |
18 ``` | 17 ``` |
19 <script src="../resources/runner.js"></script> | 18 <script src="../resources/runner.js"></script> |
(...skipping 22 matching lines...) Expand all Loading... |
42 setup: function () { ... }, // test setup logic, called once before each run | 41 setup: function () { ... }, // test setup logic, called once before each run |
43 run: function () { ... }, // contains the code to benchmark | 42 run: function () { ... }, // contains the code to benchmark |
44 iterationCount: 5 // repeat the test 5 times | 43 iterationCount: 5 // repeat the test 5 times |
45 }); | 44 }); |
46 ``` | 45 ``` |
47 | 46 |
48 In the case of `PerfTestRunner.measureRunsPerSecond`, each run invokes | 47 In the case of `PerfTestRunner.measureRunsPerSecond`, each run invokes |
49 `test.run` multiple times. | 48 `test.run` multiple times. |
50 | 49 |
51 **Tracing support** | 50 **Tracing support** |
| 51 |
52 When the test is run through Telemetry, you can also collect timing of trace | 52 When the test is run through Telemetry, you can also collect timing of trace |
53 events that happen during each run by specifying `tracingCategories` & | 53 events that happen during each run by specifying `tracingCategories` & |
54 `traceEventsToMeasure` in the test object. For example: | 54 `traceEventsToMeasure` in the test object. For example: |
55 | 55 |
56 ``` | 56 ``` |
57 PerfTestRunner.measureTime({ | 57 PerfTestRunner.measureTime({ |
58 ... | 58 ... |
59 run: foo, | 59 run: foo, |
60 iterationCount: 3, | 60 iterationCount: 3, |
61 tracingCategories: 'blink', | 61 tracingCategories: 'blink', |
(...skipping 13 matching lines...) Expand all Loading... |
75 v1 v2 v3 v4 v5 v6 | 75 v1 v2 v3 v4 v5 v6 |
76 ``` | 76 ``` |
77 | 77 |
78 Besides outputting timeseries `[u1, u2, u3]`, telemetry perf test runner will | 78 Besides outputting timeseries `[u1, u2, u3]`, telemetry perf test runner will |
79 also compute the total CPU times for trace events 'A' & 'B' per `foo()` run: | 79 also compute the total CPU times for trace events 'A' & 'B' per `foo()` run: |
80 | 80 |
81 * CPU times of trace events A: `[v0 + v2, v4, 0.0]` | 81 * CPU times of trace events A: `[v0 + v2, v4, 0.0]` |
82 * CPU times of trace events B: `[0.0, v3, v5]` | 82 * CPU times of trace events B: `[0.0, v3, v5]` |
83 | 83 |
84 Example tracing synchronous tests: | 84 Example tracing synchronous tests: |
85 [append-child-measure-time.html](TestData/append-child-measure-time.html) | 85 |
86 [simple-html-measure-page-load-time.html](TestData/simple-html-measure-page-load
-time.html) | 86 * [append-child-measure-time.html](TestData/append-child-measure-time.html) |
| 87 |
| 88 * [simple-html-measure-page-load-time.html](TestData/simple-html-measure-page-
load-time.html) |
87 | 89 |
88 | 90 |
89 ### Asynchronous Perf Tests | 91 ### Asynchronous Perf Tests |
90 In asynchronous perf test, you define your test scheduler and do your own | 92 In asynchronous perf test, you define your test scheduler and do your own |
91 measurement. For example: | 93 measurement. For example: |
92 | 94 |
93 ``` | 95 ``` |
94 var isDone = false; | 96 var isDone = false; |
95 var startTime; | 97 var startTime; |
96 | 98 |
(...skipping 22 matching lines...) Expand all Loading... |
119 }); | 121 }); |
120 ``` | 122 ``` |
121 | 123 |
122 In the example above, the call | 124 In the example above, the call |
123 `PerfTestRunner.measureValueAsync(value)` send the metric of a single run to | 125 `PerfTestRunner.measureValueAsync(value)` send the metric of a single run to |
124 the test runner and also let the runner know that it has finished a single run. | 126 the test runner and also let the runner know that it has finished a single run. |
125 Once the number of run reaches `iterationCount` (6 in the example above), the | 127 Once the number of run reaches `iterationCount` (6 in the example above), the |
126 `done` callback is invoked, setting the your test state to finished. | 128 `done` callback is invoked, setting the your test state to finished. |
127 | 129 |
128 **Tracing support** | 130 **Tracing support** |
| 131 |
129 Like synchronous perf tests, tracing metrics are only available when you run | 132 Like synchronous perf tests, tracing metrics are only available when you run |
130 your tests with Telemetry. | 133 your tests with Telemetry. |
131 | 134 |
132 Unlike synchronous perf tests which the test runner framework handles test | 135 Unlike synchronous perf tests which the test runner framework handles test |
133 scheduling and tracing coverage for you, for most asynchronous tests, you need | 136 scheduling and tracing coverage for you, for most asynchronous tests, you need |
134 to manually mark when the async test begins | 137 to manually mark when the async test begins |
135 (`PerfTestRunner.addRunTestStartMarker`) and ends | 138 (`PerfTestRunner.addRunTestStartMarker`) and ends |
136 (`PerfTestRunner.addRunTestEndMarker`). Once those are marked, specifying | 139 (`PerfTestRunner.addRunTestEndMarker`). Once those are marked, specifying |
137 `tracingCategories` and `traceEventsToMeasure` will output CPU time metrics | 140 `tracingCategories` and `traceEventsToMeasure` will output CPU time metrics |
138 of trace events that happen during test runs in the fashion similar to the | 141 of trace events that happen during test runs in the fashion similar to the |
139 example of synchronous tracing test above. | 142 example of synchronous tracing test above. |
140 | 143 |
141 Example of tracing asynchronous tests: | 144 Example of tracing asynchronous tests: |
| 145 |
142 [color-changes-measure-frame-time.html](TestData/color-changes-measure-frame-tim
e.html) | 146 [color-changes-measure-frame-time.html](TestData/color-changes-measure-frame-tim
e.html) |
| 147 |
143 [simple-blob-measure-async.html](TestData/simple-blob-measure-async.html) | 148 [simple-blob-measure-async.html](TestData/simple-blob-measure-async.html) |
144 | 149 |
145 | 150 |
146 ## Running Tests | 151 ## Running Tests |
147 | 152 |
148 ** Running tests directly in browser ** | 153 ** Running tests directly in browser ** |
149 Most of Blink Performance tests should be runnable by just open the test file | 154 Most of Blink Performance tests should be runnable by just open the test file |
150 directly in the browser. However, features like tracing metrics & HTML results | 155 directly in the browser. However, features like tracing metrics & HTML results |
151 viewer won't be supported. | 156 viewer won't be supported. |
152 | 157 |
153 ** Running tests with Telemetry ** | 158 ** Running tests with Telemetry ** |
154 Assuming your current directory is chromium/src/, you can run tests with: | 159 Assuming your current directory is chromium/src/, you can run tests with: |
| 160 |
155 `./tools/perf/run_benchmark blink_perf [--test-path=<path to your tests>]` | 161 `./tools/perf/run_benchmark blink_perf [--test-path=<path to your tests>]` |
156 | 162 |
157 For information about all supported options, run: | 163 For information about all supported options, run: |
| 164 |
158 `./tools/perf/run_benchmark blink_perf --help` | 165 `./tools/perf/run_benchmark blink_perf --help` |
OLD | NEW |