OLD | NEW |
(Empty) | |
| 1 # Benchmark Tools |
| 2 |
| 3 ## compare_bench.py |
| 4 |
| 5 The `compare_bench.py` utility which can be used to compare the result of benchm
arks. |
| 6 The program is invoked like: |
| 7 |
| 8 ``` bash |
| 9 $ compare_bench.py <old-benchmark> <new-benchmark> [benchmark options]... |
| 10 ``` |
| 11 |
| 12 Where `<old-benchmark>` and `<new-benchmark>` either specify a benchmark executa
ble file, or a JSON output file. The type of the input file is automatically det
ected. If a benchmark executable is specified then the benchmark is run to obtai
n the results. Otherwise the results are simply loaded from the output file. |
| 13 |
| 14 The sample output using the JSON test files under `Inputs/` gives: |
| 15 |
| 16 ``` bash |
| 17 $ ./compare_bench.py ./gbench/Inputs/test1_run1.json ./gbench/Inputs/test1_run2.
json |
| 18 Comparing ./gbench/Inputs/test1_run1.json to ./gbench/Inputs/test1_run2.json |
| 19 Benchmark Time CPU |
| 20 ---------------------------------------------- |
| 21 BM_SameTimes +0.00 +0.00 |
| 22 BM_2xFaster -0.50 -0.50 |
| 23 BM_2xSlower +1.00 +1.00 |
| 24 BM_10PercentFaster -0.10 -0.10 |
| 25 BM_10PercentSlower +0.10 +0.10 |
| 26 ``` |
| 27 |
| 28 When a benchmark executable is run, the raw output from the benchmark is printed
in real time to stdout. The sample output using `benchmark/basic_test` for both
arguments looks like: |
| 29 |
| 30 ``` |
| 31 ./compare_bench.py test/basic_test test/basic_test --benchmark_filter=BM_empty
.* |
| 32 RUNNING: test/basic_test --benchmark_filter=BM_empty.* |
| 33 Run on (4 X 4228.32 MHz CPU s) |
| 34 2016-08-02 19:21:33 |
| 35 Benchmark Time CPU Iterations |
| 36 -------------------------------------------------------------------- |
| 37 BM_empty 9 ns 9 ns 79545455 |
| 38 BM_empty/threads:4 4 ns 9 ns 75268816 |
| 39 BM_empty_stop_start 8 ns 8 ns 83333333 |
| 40 BM_empty_stop_start/threads:4 3 ns 8 ns 83333332 |
| 41 RUNNING: test/basic_test --benchmark_filter=BM_empty.* |
| 42 Run on (4 X 4228.32 MHz CPU s) |
| 43 2016-08-02 19:21:35 |
| 44 Benchmark Time CPU Iterations |
| 45 -------------------------------------------------------------------- |
| 46 BM_empty 9 ns 9 ns 76086957 |
| 47 BM_empty/threads:4 4 ns 9 ns 76086956 |
| 48 BM_empty_stop_start 8 ns 8 ns 87500000 |
| 49 BM_empty_stop_start/threads:4 3 ns 8 ns 88607596 |
| 50 Comparing test/basic_test to test/basic_test |
| 51 Benchmark Time CPU |
| 52 --------------------------------------------------------- |
| 53 BM_empty +0.00 +0.00 |
| 54 BM_empty/threads:4 +0.00 +0.00 |
| 55 BM_empty_stop_start +0.00 +0.00 |
| 56 BM_empty_stop_start/threads:4 +0.00 +0.00 |
| 57 ``` |
| 58 |
| 59 Obviously this example doesn't give any useful output, but it's intended to show
the output format when 'compare_bench.py' needs to run benchmarks. |
OLD | NEW |