| OLD | NEW |
| 1 # Analysis Server Benchmarks | 1 # Analysis Server Benchmarks |
| 2 | 2 |
| 3 ## How to run the benchmarks | 3 ## How to run the benchmarks |
| 4 | 4 |
| 5 To see a list of all available benchmarks, run: | 5 To see a list of all available benchmarks, run: |
| 6 | 6 |
| 7 ``` | 7 ``` |
| 8 dart benchmarks/benchmarks.dart list | 8 dart benchmark/benchmarks.dart list |
| 9 ``` | 9 ``` |
| 10 | 10 |
| 11 To run an individual benchmark, run: | 11 To run an individual benchmark, run: |
| 12 | 12 |
| 13 ``` | 13 ``` |
| 14 dart benchmarks/benchmarks.dart run <benchmark-id> | 14 dart benchmark/benchmarks.dart run <benchmark-id> |
| 15 ``` | 15 ``` |
| 16 | 16 |
| 17 ## How they're tested | 17 ## How they're tested |
| 18 | 18 |
| 19 In order to make sure that our benchmarks don't regress in terms of their | 19 In order to make sure that our benchmarks don't regress in terms of their |
| 20 ability to run, we create one unit test per benchmark, and run those tests | 20 ability to run, we create one unit test per benchmark, and run those tests |
| 21 as part of our normal CI test suite. | 21 as part of our normal CI test suite. |
| 22 | 22 |
| 23 To save time on the CI, we only run one iteration of each benchmark | 23 To save time on the CI, we only run one iteration of each benchmark |
| 24 (`--repeat=1`), and we run the benchmark on a smaller data set (`--quick`). | 24 (`--repeat=1`), and we run the benchmark on a smaller data set (`--quick`). |
| 25 | 25 |
| 26 See `test/benchmark_test.dart`. | 26 See `test/benchmark_test.dart`. |
| 27 | 27 |
| 28 ## To add a new benchmark | 28 ## To add a new benchmark |
| 29 | 29 |
| 30 Register the new benchmark in then main() method of benchmarks/benchmarks.dart. | 30 Register the new benchmark in the `main()` method of benchmark/benchmarks.dart. |
| 31 | 31 |
| 32 ## On the bots | 32 ## On the bots |
| 33 | 33 |
| 34 Our benchmarks run on a continuous performance testing system. It will run | 34 Our benchmarks run on a continuous performance testing system. Currently, the |
| 35 any benchmark produced by the `benchmarks/benchmarks.dart list` command. | 35 benchmarks need to be manually registered ahead of time. |
| 36 | |
| 37 To not run a benchmark on the bot, define the benchmark with the `disable` flag. | |
| OLD | NEW |