OLD | NEW |
(Empty) | |
| 1 # Blink Performance Tests |
| 2 |
| 3 [TOC] |
| 4 |
| 5 ## Overview |
| 6 |
| 7 Blink perf tests are used for micro benchmarking the surface of Blink that |
| 8 is exposed to the Web. They are the counterpart of [LayoutTests/] |
| 9 (https://chromium.googlesource.com/chromium/src/+/master/docs/testing/layout_tes
ts.md) |
| 10 but for performance coverage. |
| 11 |
| 12 ## Writing Tests |
| 13 Each test entry point is a HTML file written using |
| 14 [runner.js](resources/runner.js) |
| 15 testing framework. The test file is placed inside a sub folder of |
| 16 [Blink/PerformanceTests/](.) |
| 17 and is started by importing `runner.js` script into the document: |
| 18 ``` |
| 19 <script src="../resources/runner.js"></script> |
| 20 |
| 21 ``` |
| 22 |
| 23 ### Synchronous Perf Tests |
| 24 In a nutshell, to measure speed of synchronous code encapsulated in a test run |
| 25 method `F`, synchronous perf tests exercises this loop: |
| 26 |
| 27 ``` |
| 28 FOR i = 1 to NUMBER_OF_REPEAT |
| 29 Start timer |
| 30 F() |
| 31 Stop timer |
| 32 ``` |
| 33 |
| 34 Depending on how fast `F` runs, one can choose between |
| 35 `PerfTestRunner.measureTime` or `PerfTestRunner.measureRunsPerSecond` |
| 36 (very fast). In either case, you create a test object & run by invoking the |
| 37 measure method as follow: |
| 38 |
| 39 ``` |
| 40 PerfTestRunner.measureTime({ // the "test" object |
| 41 description: '...', |
| 42 setup: function () { ... }, // test setup logic, called once before each run |
| 43 run: function () { ... }, // contains the code to benchmark |
| 44 iterationCount: 5 // repeat the test 5 times |
| 45 }); |
| 46 ``` |
| 47 |
| 48 In the case of `PerfTestRunner.measureRunsPerSecond`, each run invokes |
| 49 `test.run` multiple times. |
| 50 |
| 51 **Tracing support** |
| 52 When the test is run through Telemetry, you can also collect timing of trace |
| 53 events that happen during each run by specifying `tracingCategories` & |
| 54 `traceEventsToMeasure` in the test object. For example: |
| 55 |
| 56 ``` |
| 57 PerfTestRunner.measureTime({ |
| 58 ... |
| 59 run: foo, |
| 60 iterationCount: 3, |
| 61 tracingCategories: 'blink', |
| 62 traceEventsToMeasure: ['A', 'B'], |
| 63 }); |
| 64 ``` |
| 65 To illustrate what the framework computes, imaging the test timeline as |
| 66 follow: |
| 67 |
| 68 ``` |
| 69 Test run times (time duration of each slice is right under it): |
| 70 -----------[ foo ]-----[ foo ]-----[ foo ]------ |
| 71 u1 u2 u3 |
| 72 ---------------[ A ]------------------------------------------------------ |
| 73 v0 |
| 74 -----------------[ A ]--[ A ]---------[ B ]--[A]----------[ B ]--[C]-------- |
| 75 v1 v2 v3 v4 v5 v6 |
| 76 ``` |
| 77 |
| 78 Besides outputting timeseries `[u1, u2, u3]`, telemetry perf test runner will |
| 79 also compute the total CPU times for trace events 'A' & 'B' per `foo()` run: |
| 80 |
| 81 * CPU times of trace events A: `[v0 + v2, v4, 0.0]` |
| 82 * CPU times of trace events B: `[0.0, v3, v5]` |
| 83 |
| 84 Example tracing synchronous tests: |
| 85 [append-child-measure-time.html](TestData/append-child-measure-time.html) |
| 86 [simple-html-measure-page-load-time.html](TestData/simple-html-measure-page-load
-time.html) |
| 87 |
| 88 |
| 89 ### Asynchronous Perf Tests |
| 90 In asynchronous perf test, you define your test scheduler and do your own |
| 91 measurement. For example: |
| 92 |
| 93 ``` |
| 94 var isDone = false; |
| 95 var startTime; |
| 96 |
| 97 function runTest() { |
| 98 if (startTime) { |
| 99 PerfTestRunner.measureValueAsync(PerfTestRunner.now() - startTime); |
| 100 PerfTestRunner.addRunTestEndMarker(); // For tracing metrics |
| 101 } |
| 102 if (!done) { |
| 103 PerfTestRunner.addRunTestStartMarker(); |
| 104 startTime = PerfTestRunner.now(); // For tracing metrics |
| 105 // runTest will be invoked after the async operation finish |
| 106 runAsyncOperation(runTest); |
| 107 } |
| 108 } |
| 109 |
| 110 PerfTestRunner.startMeasureValuesAsync({ |
| 111 unit: 'ms', |
| 112 done: function () { |
| 113 isDone = true; |
| 114 }, |
| 115 run: function() { |
| 116 runTest(); |
| 117 }, |
| 118 iterationCount: 6, |
| 119 }); |
| 120 ``` |
| 121 |
| 122 In the example above, the call |
| 123 `PerfTestRunner.measureValueAsync(value)` send the metric of a single run to |
| 124 the test runner and also let the runner know that it has finished a single run. |
| 125 Once the number of run reaches `iterationCount` (6 in the example above), the |
| 126 `done` callback is invoked, setting the your test state to finished. |
| 127 |
| 128 **Tracing support** |
| 129 Like synchronous perf tests, tracing metrics are only available when you run |
| 130 your tests with Telemetry. |
| 131 |
| 132 Unlike synchronous perf tests which the test runner framework handles test |
| 133 scheduling and tracing coverage for you, for most asynchronous tests, you need |
| 134 to manually mark when the async test begins |
| 135 (`PerfTestRunner.addRunTestStartMarker`) and ends |
| 136 (`PerfTestRunner.addRunTestEndMarker`). Once those are marked, specifying |
| 137 `tracingCategories` and `traceEventsToMeasure` will output CPU time metrics |
| 138 of trace events that happen during test runs in the fashion similar to the |
| 139 example of synchronous tracing test above. |
| 140 |
| 141 Example of tracing asynchronous tests: |
| 142 [color-changes-measure-frame-time.html](TestData/color-changes-measure-frame-tim
e.html) |
| 143 [simple-blob-measure-async.html](TestData/simple-blob-measure-async.html) |
| 144 |
| 145 |
| 146 ## Running Tests |
| 147 |
| 148 ** Running tests directly in browser ** |
| 149 Most of Blink Performance tests should be runnable by just open the test file |
| 150 directly in the browser. However, features like tracing metrics & HTML results |
| 151 viewer won't be supported. |
| 152 |
| 153 ** Running tests with Telemetry ** |
| 154 Assuming your current directory is chromium/src/, you can run tests with: |
| 155 `./tools/perf/run_benchmark blink_perf [--test-path=<path to your tests>]` |
| 156 |
| 157 For information about all supported options, run: |
| 158 `./tools/perf/run_benchmark blink_perf --help` |
OLD | NEW |