| OLD | NEW |
| 1 # Running Benchmarks | 1 # Running Benchmarks |
| 2 | 2 |
| 3 There are two entry points for running benchmarks: | 3 There are two entry points for running benchmarks: |
| 4 * **main.dart** - a general Dart application for running performance benchmarks | 4 * **main.dart** - a general Dart application for running performance benchmarks |
| 5 * **local_runner.dart** - an example Dart application | 5 * **local_runner.dart** - an example Dart application |
| 6 which sets up the local environment | 6 which sets up the local environment |
| 7 and then calls main.dart to run performance benchmarks | 7 and then calls main.dart to run performance benchmarks |
| 8 | 8 |
| 9 ## local_runner.dart | 9 ## local_runner.dart |
| 10 | 10 |
| 11 This Dart application is one example for running performance benchmarks. | 11 This Dart application is one example for running performance benchmarks. |
| 12 When run, this application 1) extracts a branch from a git repository | 12 When run, this application 1) extracts a branch from a git repository |
| 13 into a temporary directory, and 2) creates a symlink to the out or xcodebuild | 13 into a temporary directory, and 2) creates a symlink to the out or xcodebuild |
| 14 directory for proper package-root package resolution. | 14 directory for proper package-root package resolution. |
| 15 Once setup is complete, this applications calls main.dart | 15 Once setup is complete, this applications calls main.dart |
| 16 | 16 |
| 17 The required command line arguments are | 17 The required command line arguments are |
| 18 * **gitDir** = a path to the git repository containing the initial target source | 18 * **gitDir** = a path to the git repository containing the initial target source |
| 19 * **branch** = the branch containing the initial target source | 19 * **branch** = the branch containing the initial target source |
| 20 * **inputFile** = the instrumentation or log file | 20 * **inputFile** = the instrumentation or log file |
| 21 | 21 |
| 22 Additional arguments are passed directly to main.dart. | 22 Additional arguments are passed directly to main.dart. |
| 23 For example, you may want to specify --newTaskModel to measure performance | 23 For example, if the log was recorded on one machine and is played back on anothe
r, |
| 24 with the new task model versus the old task model, | |
| 25 or if the log was recorded on one machine and is played back on another, | |
| 26 then you might need to specify -m<oldSrcPath>,<newSrcPath> | 24 then you might need to specify -m<oldSrcPath>,<newSrcPath> |
| 27 to map the source paths for playback. | 25 to map the source paths for playback. |
| 28 When specifying additional arguments, any occurrences of @tmpSrcDir@ | 26 When specifying additional arguments, any occurrences of @tmpSrcDir@ |
| 29 will be replaced with the absolute path of the temporary directory | 27 will be replaced with the absolute path of the temporary directory |
| 30 into which the source was extracted. | 28 into which the source was extracted. |
| 31 | 29 |
| 32 ## main.dart | 30 ## main.dart |
| 33 | 31 |
| 34 This Dart application reads an instrumentation or local log file produced by | 32 This Dart application reads an instrumentation or local log file produced by |
| 35 analysis server, "replays" that interaction with the analysis server, | 33 analysis server, "replays" that interaction with the analysis server, |
| 36 compares the notifications and responses with what was recorded in the log, | 34 compares the notifications and responses with what was recorded in the log, |
| 37 and produces a report. It assumes that the environment for playback has | 35 and produces a report. It assumes that the environment for playback has |
| 38 already been setup. | 36 already been setup. |
| 39 The required command line arguments are | 37 The required command line arguments are |
| 40 * **-i, --input <filePath>** | 38 * **-i, --input <filePath>** |
| 41 The input file specifying how this client should interact with the server. | 39 The input file specifying how this client should interact with the server. |
| 42 If the input file name is "stdin", then the instructions are read from stdin. | 40 If the input file name is "stdin", then the instructions are read from stdin. |
| 43 * **-m, --map <oldSrcPath>,<newSrcPath>** | 41 * **-m, --map <oldSrcPath>,<newSrcPath>** |
| 44 This option defines a mapping from the original source directory <oldSrcPath> | 42 This option defines a mapping from the original source directory <oldSrcPath> |
| 45 when the instrumentation or log file was generated | 43 when the instrumentation or log file was generated |
| 46 to the target source directory <newSrcPath> used during performance testing. | 44 to the target source directory <newSrcPath> used during performance testing. |
| 47 Multiple mappings can be specified. | 45 Multiple mappings can be specified. |
| 48 WARNING: The contents of the target directory will be modified | 46 WARNING: The contents of the target directory will be modified |
| 49 * **-t, --tmpSrcDir <dirPath>** | 47 * **-t, --tmpSrcDir <dirPath>** |
| 50 The temporary directory containing source used during performance measurement. | 48 The temporary directory containing source used during performance measurement. |
| 51 WARNING: The contents of the target directory will be modified | 49 WARNING: The contents of the target directory will be modified |
| 52 * **--newTaskModel** enable the use of the new task model | |
| 53 * **-d, --diagnosticPort** localhost port on which server | 50 * **-d, --diagnosticPort** localhost port on which server |
| 54 will provide diagnostic web pages | 51 will provide diagnostic web pages |
| 55 * **-v, --verbose** Verbose logging | 52 * **-v, --verbose** Verbose logging |
| 56 * **--vv** Extra verbose logging | 53 * **--vv** Extra verbose logging |
| 57 * **-h, --help** Print this help information | 54 * **-h, --help** Print this help information |
| 58 | 55 |
| 59 For each request recorded in the input file, | 56 For each request recorded in the input file, |
| 60 the application sends a corresponding request to the analysis server | 57 the application sends a corresponding request to the analysis server |
| 61 and waits up to 60 seconds for a response to that request. | 58 and waits up to 60 seconds for a response to that request. |
| 62 If a response in not received in that time, then the application exits. | 59 If a response in not received in that time, then the application exits. |
| 63 Any responses that are received are compared with the recorded response. | 60 Any responses that are received are compared with the recorded response. |
| 64 | 61 |
| 65 For each analysis-complete notification recorded in the input file, | 62 For each analysis-complete notification recorded in the input file, |
| 66 the application waits for the corresponding analysis-complete notification | 63 the application waits for the corresponding analysis-complete notification |
| 67 from the running analysis server. | 64 from the running analysis server. |
| 68 While it is waiting for an analysis-complete notification, | 65 While it is waiting for an analysis-complete notification, |
| 69 the application monitors the stream of notifications. | 66 the application monitors the stream of notifications. |
| 70 If there is a period of more than 60 seconds during which no communication | 67 If there is a period of more than 60 seconds during which no communication |
| 71 is received from the server, the application assumes that the server is hung | 68 is received from the server, the application assumes that the server is hung |
| 72 and exits. | 69 and exits. |
| OLD | NEW |