Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(471)

Side by Side Diff: mojo/devtools/common/docs/mojo_benchmark.md

Issue 1423233002: Centralize mojo_benchmark documentation. (Closed) Base URL: git@github.com:domokit/mojo.git@master
Patch Set: Created 5 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | mojo/devtools/common/mojo_benchmark » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # mojo_benchmark 1 # mojo_benchmark
2 2
3 `mojo_benchmark` allows you to run performance tests for any Mojo application 3 `mojo_benchmark` allows you to run performance tests for any Mojo application
4 that participates in the [tracing 4 that participates in the [tracing
5 ecosystem](https://github.com/domokit/mojo/blob/master/mojo/services/tracing/int erfaces/tracing.mojom) 5 ecosystem](https://github.com/domokit/mojo/blob/master/mojo/services/tracing/int erfaces/tracing.mojom)
6 with no app changes required. 6 with no app changes required.
7 7
8 The script reads a list of benchmarks to run from a file, runs each benchmark in 8 The script reads a list of benchmarks to run from a file, runs each benchmark in
9 controlled caching conditions with tracing enabled and performs specified 9 controlled caching conditions with tracing enabled and performs specified
10 measurements on the collected trace data. 10 measurements on the collected trace data.
11 11
12 ## Defining benchmarks 12 ## Defining benchmarks
13 13
14 `mojo_benchmark` runs performance tests defined in a benchmark file. The 14 `mojo_benchmark` runs performance tests defined in a benchmark file. The
15 benchmark file is a Python dictionary of the following format: 15 benchmark file is a Python program setting a dictionary of the following format:
16 16
17 ```python 17 ```python
18 benchmarks = [ 18 benchmarks = [
19 { 19 {
20 'name': '<name of the benchmark>', 20 'name': '<name of the benchmark>',
21 'app': '<url of the app to benchmark>', 21 'app': '<url of the app to benchmark>',
22 'shell-args': [], 22 'shell-args': [],
23 'duration': <duration in seconds>, 23 'duration': <duration in seconds>,
24 24
25 # List of measurements to make. 25 # List of measurements to make.
26 'measurements': [ 26 'measurements': [
27 '<measurement type>/<event category>/<event name>', 27 {
28 'name': <my_measurement>,
29 'spec': <spec>,
30 },
31 (...)
28 ], 32 ],
29 }, 33 },
30 ] 34 ]
31 ``` 35 ```
32 36
37 The benchmark file may reference the `target_os` global that will be any of
38 ('android', 'linux'), indicating the system on which the benchmarks are run.
39
40 ### Measurement specs
41
33 The following types of measurements are available: 42 The following types of measurements are available:
34 43
35 - `time_until` - measures time until the first occurence of the specified event 44 - `time_until`
36 - `avg_duration` - measures the average duration of all instances of the 45 - `time_between`
37 specified event 46 - `avg_duration`
47 - `percentile_duration`
48
49 `time_until` records the time until the first occurence of the targeted event.
50 The underlying benchmark runner records the time origin just before issuing the
51 connection call to the application being benchmarked. Results of `time_until`
52 measurements are relative to this time. Spec format:
53
54 ```
55 'time_until/<category>/<event>'
56 ```
57
58 `time_between` records the time between the first occurence of the first
59 targeted event and the first occurence of the second targeted event. Spec
60 format:
61
62 ```
63 'time_between/<category1>/<event1>/<category2>/<event2>'
64 ```
65
66 `avg_duration` records the average duration of all occurences of the targeted
67 event. Spec format:
68
69 ```
70 'avg_duration/<category>/<event>'
71 ```
72
73 `percentile_duration` records the value at the given percentile of durations of
74 all occurences of the targeted event. Spec format:
75
76 ```
77 'percentile_duration/<category>/<event>/<percentile>'
78 ```
79
80 where `<percentile>` is a number between 0.0 and 0.1.
38 81
39 ## Caching 82 ## Caching
40 83
41 The script runs each benchmark twice. The first run (**cold start**) clears 84 The script runs each benchmark twice. The first run (**cold start**) clears
42 caches of the following apps on startup: 85 caches of the following apps on startup:
43 86
44 - network_service.mojo 87 - `network_service.mojo`
45 - url_response_disk_cache.mojo 88 - `url_response_disk_cache.mojo`
46 89
47 The second run (**warm start**) runs immediately afterwards, without clearing 90 The second run (**warm start**) runs immediately afterwards, without clearing
48 any caches. 91 any caches.
49 92
50 ## Time origin
51
52 The underlying benchmark runner records the time origin just before issuing the
53 connection call to the application being benchmarked. Results of `time_until`
54 measurements are relative to this time.
55
56 ## Example 93 ## Example
57 94
58 For an app that records a trace event named "initialized" in category "my_app" 95 For an app that records a trace event named "initialized" in category "my_app"
59 once its initialization is complete, we can benchmark the initialization time of 96 once its initialization is complete, we can benchmark the initialization time of
60 the app (from the moment someone tries to connect to it to the app completing 97 the app (from the moment someone tries to connect to it to the app completing
61 its initialization) using the following benchmark file: 98 its initialization) using the following benchmark file:
62 99
63 ```python 100 ```python
64 benchmarks = [ 101 benchmarks = [
65 { 102 {
(...skipping 27 matching lines...) Expand all
93 130
94 If no `--server-url` is specified, the script assumes that a local instance of 131 If no `--server-url` is specified, the script assumes that a local instance of
95 the dashboard is running at `http://localhost:8080`. The script assumes that the 132 the dashboard is running at `http://localhost:8080`. The script assumes that the
96 working directory from which it is called is a git repository and queries it to 133 working directory from which it is called is a git repository and queries it to
97 determine the sequential number identifying the revision (as the number of 134 determine the sequential number identifying the revision (as the number of
98 commits in the current branch in the repository). 135 commits in the current branch in the repository).
99 136
100 For more information refer to: 137 For more information refer to:
101 138
102 - [Catapult project](https://github.com/catapult-project/catapult) 139 - [Catapult project](https://github.com/catapult-project/catapult)
OLDNEW
« no previous file with comments | « no previous file | mojo/devtools/common/mojo_benchmark » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698