| OLD | NEW |
| 1 Each package may include a configuration file that applies to the package as a | 1 Each package may include a configuration file that applies to the package as a |
| 2 whole. This file can be used to provide custom defaults for various options, to | 2 whole. This file can be used to provide custom defaults for various options, to |
| 3 define configuration for multiple files, and more. | 3 define configuration for multiple files, and more. |
| 4 | 4 |
| 5 The file is named `dart_test.yaml` and lives at the root of the package, next to | 5 The file is named `dart_test.yaml` and lives at the root of the package, next to |
| 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an | 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an |
| 7 example: | 7 example: |
| 8 | 8 |
| 9 [YAML]: http://yaml.org/ | 9 [YAML]: http://yaml.org/ |
| 10 | 10 |
| 11 ```yaml | 11 ```yaml |
| 12 # This package's tests are very slow. Double the default timeout. | 12 # This package's tests are very slow. Double the default timeout. |
| 13 timeout: 2x | 13 timeout: 2x |
| 14 | 14 |
| 15 # This is a browser-only package, so test on content shell by default. | 15 # This is a browser-only package, so test on content shell by default. |
| 16 platforms: [content-shell] | 16 platforms: [content-shell] |
| 17 |
| 18 tags: |
| 19 # Integration tests are even slower, so increase the timeout again. |
| 20 integration: {timeout: 2x} |
| 21 |
| 22 # Sanity tests are quick and verify that nothing is obviously wrong. Declaring |
| 23 # the tag allows us to tag tests with it without getting warnings. |
| 24 sanity: |
| 17 ``` | 25 ``` |
| 18 | 26 |
| 19 * [Config Fields](#config-fields) | 27 * [Test Configuration](#test-configuration) |
| 28 * [`timeout`](#timeout) |
| 29 * [`verbose_trace`](#verbose_trace) |
| 30 * [`js_trace`](#js_trace) |
| 31 * [Runner Configuration](#runner-configuration) |
| 20 * [`paths`](#paths) | 32 * [`paths`](#paths) |
| 21 * [`filename`](#filename) | 33 * [`filename`](#filename) |
| 22 * [`platforms`](#platforms) | 34 * [`platforms`](#platforms) |
| 23 * [`concurrency`](#concurrency) | 35 * [`concurrency`](#concurrency) |
| 24 * [`pub_serve`](#pub_serve) | 36 * [`pub_serve`](#pub_serve) |
| 25 * [`timeout`](#timeout) | |
| 26 * [`reporter`](#reporter) | 37 * [`reporter`](#reporter) |
| 27 * [`verbose_trace`](#verbose_trace) | 38 * [Configuring Tags](#configuring-tags) |
| 28 * [`js_trace`](#js_trace) | |
| 29 | 39 |
| 30 ## Config Fields | 40 ## Test Configuration |
| 31 | 41 |
| 32 These fields directly affect the behavior of the test runner. They go at the | 42 There are two major categories of configuration field: "test" and "runner". Test |
| 33 root of the configuration file. | 43 configuration controls how individual tests run, while |
| 44 [runner configuration](#runner-configuration) controls the test runner as a |
| 45 whole. Both types of fields may be used at the top level of a configuration |
| 46 file. However, because different tests can have different test configuration, |
| 47 only test configuration fields may be used to |
| 48 [configure tags](#configuring-tags). |
| 49 |
| 50 ### `timeout` |
| 51 |
| 52 This field indicates how much time the test runner should allow a test to remain |
| 53 inactive before it considers that test to have failed. It has three possible |
| 54 formats: |
| 55 |
| 56 * The string "none" indicates that tests should never time out. |
| 57 |
| 58 * A number followed by a unit abbreviation indicates an exact time. For example, |
| 59 "1m" means a timeout of one minute, and "30s" means a timeout of thirty |
| 60 seconds. Multiple numbers can be combined, as in "1m 30s". |
| 61 |
| 62 * A number followed by "x" indicates a multiple. This is applied to the default |
| 63 value of 30s. |
| 64 |
| 65 ```yaml |
| 66 timeout: 1m |
| 67 ``` |
| 68 |
| 69 ### `verbose_trace` |
| 70 |
| 71 This boolean field controls whether or not stack traces caused by errors are |
| 72 trimmed to remove internal stack frames. This includes frames from the Dart core |
| 73 libraries, the [`stack_trace`][stack_trace] package, and the `test` package |
| 74 itself. It defaults to `false`. |
| 75 |
| 76 [stack_trace]: https://pub.dartlang.org/packages/stack_trace |
| 77 |
| 78 ```yaml |
| 79 verbose_trace: true |
| 80 ``` |
| 81 |
| 82 ### `js_trace` |
| 83 |
| 84 This boolean field controls whether or not stack traces caused by errors that |
| 85 occur while running Dart compiled to JS are converted back to Dart style. This |
| 86 conversion uses the source map generated by `dart2js` to approximate the |
| 87 original Dart line, column, and in some cases member name for each stack frame. |
| 88 It defaults to `false`. |
| 89 |
| 90 ```yaml |
| 91 js_trace: true |
| 92 ``` |
| 93 |
| 94 ## Runner Configuration |
| 95 |
| 96 Unlike [test configuration](#test-configuration), runner configuration affects |
| 97 the test runner as a whole rather than individual tests. It can only be used at |
| 98 the top level of the configuration file. |
| 34 | 99 |
| 35 ### `paths` | 100 ### `paths` |
| 36 | 101 |
| 37 This field indicates the default paths that the test runner should run. These | 102 This field indicates the default paths that the test runner should run. These |
| 38 paths are usually directories, although single filenames may be used as well. | 103 paths are usually directories, although single filenames may be used as well. |
| 39 Paths must be relative, and they must be in URL format so that they're | 104 Paths must be relative, and they must be in URL format so that they're |
| 40 compatible across operating systems. This defaults to `[test]`. | 105 compatible across operating systems. This defaults to `[test]`. |
| 41 | 106 |
| 42 ```yaml | 107 ```yaml |
| 43 paths: [dart/test] | 108 paths: [dart/test] |
| (...skipping 52 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 96 | 161 |
| 97 This field indicates that the test runner should run against a `pub serve` | 162 This field indicates that the test runner should run against a `pub serve` |
| 98 instance by default, and provides the port number for that instance. Note that | 163 instance by default, and provides the port number for that instance. Note that |
| 99 if there is no `pub serve` instance running at that port, running the tests will | 164 if there is no `pub serve` instance running at that port, running the tests will |
| 100 fail by default. | 165 fail by default. |
| 101 | 166 |
| 102 ```yaml | 167 ```yaml |
| 103 pub_serve: 8081 | 168 pub_serve: 8081 |
| 104 ``` | 169 ``` |
| 105 | 170 |
| 106 ### `timeout` | |
| 107 | |
| 108 This field indicates how much time the test runner should allow a test to remain | |
| 109 inactive before it considers that test to have failed. It has three possible | |
| 110 formats: | |
| 111 | |
| 112 * The string "none" indicates that tests should never time out. | |
| 113 | |
| 114 * A number followed by a unit abbreviation indicates an exact time. For example, | |
| 115 "1m" means a timeout of one minute, and "30s" means a timeout of thirty | |
| 116 seconds. Multiple numbers can be combined, as in "1m 30s". | |
| 117 | |
| 118 * A number followed by "x" indicates a multiple. This is applied to the default | |
| 119 value of 30s. | |
| 120 | |
| 121 ```yaml | |
| 122 timeout: 1m | |
| 123 ``` | |
| 124 | |
| 125 ### `reporter` | 171 ### `reporter` |
| 126 | 172 |
| 127 This field indicates the default reporter to use. It may be set to "compact", | 173 This field indicates the default reporter to use. It may be set to "compact", |
| 128 "expanded", or "json" (although why anyone would want to default to JSON is | 174 "expanded", or "json" (although why anyone would want to default to JSON is |
| 129 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. | 175 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. |
| 130 | 176 |
| 131 ```yaml | 177 ```yaml |
| 132 reporter: expanded | 178 reporter: expanded |
| 133 ``` | 179 ``` |
| 134 | 180 |
| 135 ### `verbose_trace` | 181 ## Configuring Tags |
| 136 | 182 |
| 137 This boolean field controls whether or not stack traces caused by errors are | 183 The `tag` field can be used to apply [test configuration](#test-configuration) |
| 138 trimmed to remove internal stack frames. This includes frames from the Dart core | 184 to all tests [with a given tag][tagging tests]. It takes a map from tag names to |
| 139 libraries, the [`stack_trace`][stack_trace] package, and the `test` package | 185 configuration maps. These configuration maps are just like the top level of the |
| 140 itself. It defaults to `false`. | 186 configuration file, except that they may not contain |
| 187 [runner configuration](#runner-configuration). |
| 141 | 188 |
| 142 [stack_trace]: https://pub.dartlang.org/packages/stack_trace | 189 [tagging tests]: https://github.com/dart-lang/test/blob/master/README.md#tagging
-tests |
| 143 | 190 |
| 144 ```yaml | 191 ```yaml |
| 145 verbose_trace: true | 192 tags: |
| 193 # Integration tests need more time to run. |
| 194 integration: |
| 195 timeout: 1m |
| 146 ``` | 196 ``` |
| 147 | 197 |
| 148 ### `js_trace` | 198 Tags may also have no configuration associated with them. The test runner prints |
| 149 | 199 a warning whenever it encounters a tag it doesn't recognize, and this the best |
| 150 This boolean field controls whether or not stack traces caused by errors that | 200 way to tell it that a tag exists. |
| 151 occur while running Dart compiled to JS are converted back to Dart style. This | |
| 152 conversion uses the source map generated by `dart2js` to approximate the | |
| 153 original Dart line, column, and in some cases member name for each stack frame. | |
| 154 It defaults to `false`. | |
| 155 | 201 |
| 156 ```yaml | 202 ```yaml |
| 157 js_trace: true | 203 # We occasionally want to use --tags or --exclude-tags on these tags. |
| 204 tags: |
| 205 # A test that spawns a browser. |
| 206 browser: |
| 207 |
| 208 # A test that needs Ruby installed. |
| 209 ruby: |
| 158 ``` | 210 ``` |
| 211 |
| 212 Tag configuration is applied at whatever level the tag appears—so if a group is |
| 213 tagged as `integration`, its timeout will take precedence over the suite's |
| 214 timeout but not any tests'. If the group itself had a timeout declared, the |
| 215 group's explicit timeout would take precedence over the tag. |
| 216 |
| 217 If multiple tags appear at the same level, and they have conflicting |
| 218 configurations, the test runner *does not guarantee* what order they'll be |
| 219 resolved in. In practice, conflicting configuration is pretty unlikely and it's |
| 220 easy to just explicitly specify what you want on the test itself. |
| OLD | NEW |