| OLD | NEW |
| 1 # Running Benchmarks | 1 # Running Benchmarks |
| 2 | 2 |
| 3 There are two entry points for running benchmarks: | 3 There are two entry points for running benchmarks: |
| 4 * **main.dart** - a general Dart application for running performance benchmarks | 4 * **main.dart** - a general Dart application for running performance benchmarks |
| 5 * **local_runner.dart** - an example Dart application | 5 * **local_runner.dart** - an example Dart application |
| 6 which sets up the local environment | 6 which sets up the local environment |
| 7 and then calls main.dart to run performance benchmarks | 7 and then calls main.dart to run performance benchmarks |
| 8 | 8 |
| 9 ## local_runner.dart | 9 ## local_runner.dart |
| 10 | 10 |
| 11 This Dart application is one example for running performance benchmarks. | 11 This Dart application is one example for running performance benchmarks. |
| 12 When run, this application 1) extracts a branch from a git repository | 12 When run, this application 1) extracts a branch from a git repository |
| 13 into a temporary directory, and 2) creates a symlink to the out or xcodebuild | 13 into a temporary directory, and 2) creates a symlink to the out or xcodebuild |
| 14 directory for proper package-root package resolution. | 14 directory for proper package-root package resolution. |
| 15 Once setup is complete, this applications calls main.dart |
| 16 |
| 15 The required command line arguments are | 17 The required command line arguments are |
| 16 * **gitDir** = a path to the git repository containing the initial target source | 18 * **gitDir** = a path to the git repository containing the initial target source |
| 17 * **branch** = the branch containing the initial target source | 19 * **branch** = the branch containing the initial target source |
| 18 * **inputFile** = the instrumentation or log file | 20 * **inputFile** = the instrumentation or log file |
| 19 | 21 |
| 20 Once setup is complete, this applications calls main.dart | 22 Additional arguments are passed directly to main.dart. |
| 23 For example, you may want to specify --newTaskModel to measure performance |
| 24 with the new task model versus the old task model, |
| 25 or if the log was recorded on one machine and is played back on another, |
| 26 then you might need to specify -m<oldSrcPath>,<newSrcPath> |
| 27 to map the source paths for playback. |
| 28 When specifying additional arguments, any occurrences of @tmpSrcDir@ |
| 29 will be replaced with the absolute path of the temporary directory |
| 30 into which the source was extracted. |
| 21 | 31 |
| 22 ## main.dart | 32 ## main.dart |
| 23 | 33 |
| 24 This Dart application reads an instrumentation or local log file produced by | 34 This Dart application reads an instrumentation or local log file produced by |
| 25 analysis server, "replays" that interaction with the analysis server, | 35 analysis server, "replays" that interaction with the analysis server, |
| 26 compares the notifications and responses with what was recorded in the log, | 36 compares the notifications and responses with what was recorded in the log, |
| 27 and produces a report. It assumes that the environment for playback has | 37 and produces a report. It assumes that the environment for playback has |
| 28 already been setup. | 38 already been setup. |
| 29 The required command line arguments are | 39 The required command line arguments are |
| 30 * **-i, --input <filePath>** | 40 * **-i, --input <filePath>** |
| 31 The input file specifying how this client should interact with the server. | 41 The input file specifying how this client should interact with the server. |
| 32 If the input file name is "stdin", then the instructions are read from stdin. | 42 If the input file name is "stdin", then the instructions are read from stdin. |
| 33 * **-m, --map <oldSrcPath>,<newSrcPath>** | 43 * **-m, --map <oldSrcPath>,<newSrcPath>** |
| 34 This option defines a mapping from the original source directory <oldSrcPath> | 44 This option defines a mapping from the original source directory <oldSrcPath> |
| 35 when the instrumentation or log file was generated | 45 when the instrumentation or log file was generated |
| 36 to the target source directory <newSrcPath> used during performance testing. | 46 to the target source directory <newSrcPath> used during performance testing. |
| 37 Multiple mappings can be specified. | 47 Multiple mappings can be specified. |
| 38 WARNING: The contents of the target directory will be modified | 48 WARNING: The contents of the target directory will be modified |
| 39 * **-t, --tmpSrcDir <dirPath>** | 49 * **-t, --tmpSrcDir <dirPath>** |
| 40 The temporary directory containing source used during performance measurement. | 50 The temporary directory containing source used during performance measurement. |
| 41 WARNING: The contents of the target directory will be modified | 51 WARNING: The contents of the target directory will be modified |
| 42 * **--newTaskModel** enable the use of the new task model | 52 * **--newTaskModel** enable the use of the new task model |
| 43 * **-d, --diagnosticPort** localhost port on which server | 53 * **-d, --diagnosticPort** localhost port on which server |
| 44 will provide diagnostic web pages | 54 will provide diagnostic web pages |
| 45 * **-v, --verbose** Verbose logging | 55 * **-v, --verbose** Verbose logging |
| 46 * **--vv** Extra verbose logging | 56 * **--vv** Extra verbose logging |
| 47 * **-h, --help** Print this help information | 57 * **-h, --help** Print this help information |
| 58 |
| 59 For each request recorded in the input file, |
| 60 the application sends a corresponding request to the analysis server |
| 61 and waits up to 60 seconds for a response to that request. |
| 62 If a response in not received in that time, then the application exits. |
| 63 Any responses that are received are compared with the recorded response. |
| 64 |
| 65 For each analysis-complete notification recorded in the input file, |
| 66 the application waits for the corresponding analysis-complete notification |
| 67 from the running analysis server. |
| 68 While it is waiting for an analysis-complete notification, |
| 69 the application monitors the stream of notifications. |
| 70 If there is a period of more than 60 seconds during which no communication |
| 71 is received from the server, the application assumes that the server is hung |
| 72 and exits. |
| OLD | NEW |