| OLD | NEW | 
|---|
| 1 Each package may include a configuration file that applies to the package as a | 1 Each package may include a configuration file that applies to the package as a | 
| 2 whole. This file can be used to provide custom defaults for various options, to | 2 whole. This file can be used to provide custom defaults for various options, to | 
| 3 define configuration for multiple files, and more. | 3 define configuration for multiple files, and more. | 
| 4 | 4 | 
| 5 The file is named `dart_test.yaml` and lives at the root of the package, next to | 5 The file is named `dart_test.yaml` and lives at the root of the package, next to | 
| 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an | 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an | 
| 7 example: | 7 example: | 
| 8 | 8 | 
| 9 [YAML]: http://yaml.org/ | 9 [YAML]: http://yaml.org/ | 
| 10 | 10 | 
| (...skipping 33 matching lines...) Expand 10 before | Expand all | Expand 10 after  Loading... | 
| 44   * [`reporter`](#reporter) | 44   * [`reporter`](#reporter) | 
| 45 * [Configuring Tags](#configuring-tags) | 45 * [Configuring Tags](#configuring-tags) | 
| 46   * [`tags`](#tags) | 46   * [`tags`](#tags) | 
| 47   * [`add_tags`](#add_tags) | 47   * [`add_tags`](#add_tags) | 
| 48 * [Configuring Platforms](#configuring-platforms) | 48 * [Configuring Platforms](#configuring-platforms) | 
| 49   * [`on_os`](#on_os) | 49   * [`on_os`](#on_os) | 
| 50   * [`on_platform`](#on_platform) | 50   * [`on_platform`](#on_platform) | 
| 51 * [Configuration Presets](#configuration-presets) | 51 * [Configuration Presets](#configuration-presets) | 
| 52   * [`presets`](#presets) | 52   * [`presets`](#presets) | 
| 53   * [`add_preset`](#add_preset) | 53   * [`add_preset`](#add_preset) | 
|  | 54 * [Global Configuration](#global-configuration) | 
| 54 | 55 | 
| 55 ## Test Configuration | 56 ## Test Configuration | 
| 56 | 57 | 
| 57 There are two major categories of configuration field: "test" and "runner". Test | 58 There are two major categories of configuration field: "test" and "runner". Test | 
| 58 configuration controls how individual tests run, while | 59 configuration controls how individual tests run, while | 
| 59 [runner configuration](#runner-configuration) controls the test runner as a | 60 [runner configuration](#runner-configuration) controls the test runner as a | 
| 60 whole. Both types of fields may be used at the top level of a configuration | 61 whole. Both types of fields may be used at the top level of a configuration | 
| 61 file. However, because different tests can have different test configuration, | 62 file. However, because different tests can have different test configuration, | 
| 62 only test configuration fields may be used to [configure tags](#tags) or | 63 only test configuration fields may be used to [configure tags](#tags) or | 
| 63 [platforms](#on_platform). | 64 [platforms](#on_platform). | 
| (...skipping 50 matching lines...) Expand 10 before | Expand all | Expand 10 after  Loading... | 
| 114 whether the tests are skipped or a string indicating the reason they're skipped. | 115 whether the tests are skipped or a string indicating the reason they're skipped. | 
| 115 | 116 | 
| 116 [test]: https://www.dartdocs.org/documentation/test/0.12.10+2/test/test.html | 117 [test]: https://www.dartdocs.org/documentation/test/0.12.10+2/test/test.html | 
| 117 | 118 | 
| 118 ```yaml | 119 ```yaml | 
| 119 tags: | 120 tags: | 
| 120   chrome: | 121   chrome: | 
| 121     skip: "Our Chrome launcher is busted. See issue 1234." | 122     skip: "Our Chrome launcher is busted. See issue 1234." | 
| 122 ``` | 123 ``` | 
| 123 | 124 | 
|  | 125 This field is not supported in the | 
|  | 126 [global configuration file](#global-configuration). | 
|  | 127 | 
| 124 ### `test_on` | 128 ### `test_on` | 
| 125 | 129 | 
| 126 This field declares which platforms a test supports. It takes a | 130 This field declares which platforms a test supports. It takes a | 
| 127 [platform selector][platform selectors] and only allows tests to run on | 131 [platform selector][platform selectors] and only allows tests to run on | 
| 128 platforms that match the selector. It's often used with | 132 platforms that match the selector. It's often used with | 
| 129 [specific tags](#configuring-tags) to ensure that certain features will only be | 133 [specific tags](#configuring-tags) to ensure that certain features will only be | 
| 130 tested on supported platforms. | 134 tested on supported platforms. | 
| 131 | 135 | 
| 132 [platform selectors]: https://github.com/dart-lang/test/blob/master/README.md#pl
     atform-selectors | 136 [platform selectors]: https://github.com/dart-lang/test/blob/master/README.md#pl
     atform-selectors | 
| 133 | 137 | 
| 134 ```yaml | 138 ```yaml | 
| 135 tags: | 139 tags: | 
| 136   # Internet Explorer doesn't support promises yet. | 140   # Internet Explorer doesn't support promises yet. | 
| 137   promises: {test_on: "browser && !ie"} | 141   promises: {test_on: "browser && !ie"} | 
| 138 ``` | 142 ``` | 
| 139 | 143 | 
| 140 The field can also be used at the top level of the configuration file to | 144 The field can also be used at the top level of the configuration file to | 
| 141 indicate that the entire package only supports a particular platform. If someone | 145 indicate that the entire package only supports a particular platform. If someone | 
| 142 tries to run the tests on an unsupported platform, the runner will print a | 146 tries to run the tests on an unsupported platform, the runner will print a | 
| 143 warning and skip that platform. | 147 warning and skip that platform. | 
| 144 | 148 | 
| 145 ```yaml | 149 ```yaml | 
| 146 # This package uses dart:io. | 150 # This package uses dart:io. | 
| 147 test_on: vm | 151 test_on: vm | 
| 148 ``` | 152 ``` | 
| 149 | 153 | 
|  | 154 This field is not supported in the | 
|  | 155 [global configuration file](#global-configuration). | 
|  | 156 | 
| 150 ## Runner Configuration | 157 ## Runner Configuration | 
| 151 | 158 | 
| 152 Unlike [test configuration](#test-configuration), runner configuration affects | 159 Unlike [test configuration](#test-configuration), runner configuration affects | 
| 153 the test runner as a whole rather than individual tests. It can only be used at | 160 the test runner as a whole rather than individual tests. It can only be used at | 
| 154 the top level of the configuration file. | 161 the top level of the configuration file. | 
| 155 | 162 | 
| 156 ### `paths` | 163 ### `paths` | 
| 157 | 164 | 
| 158 This field indicates the default paths that the test runner should run. These | 165 This field indicates the default paths that the test runner should run. These | 
| 159 paths are usually directories, although single filenames may be used as well. | 166 paths are usually directories, although single filenames may be used as well. | 
| 160 Paths must be relative, and they must be in URL format so that they're | 167 Paths must be relative, and they must be in URL format so that they're | 
| 161 compatible across operating systems. This defaults to `[test]`. | 168 compatible across operating systems. This defaults to `[test]`. | 
| 162 | 169 | 
| 163 ```yaml | 170 ```yaml | 
| 164 paths: [dart/test] | 171 paths: [dart/test] | 
| 165 | 172 | 
| 166 paths: | 173 paths: | 
| 167 - test/instantaneous | 174 - test/instantaneous | 
| 168 - test/fast | 175 - test/fast | 
| 169 - test/middling | 176 - test/middling | 
| 170 ``` | 177 ``` | 
| 171 | 178 | 
|  | 179 This field is not supported in the | 
|  | 180 [global configuration file](#global-configuration). | 
|  | 181 | 
| 172 ### `filename` | 182 ### `filename` | 
| 173 | 183 | 
| 174 This field indicates the filename pattern that the test runner uses to find test | 184 This field indicates the filename pattern that the test runner uses to find test | 
| 175 files in directories. All files in directories passed on the command line (or in | 185 files in directories. All files in directories passed on the command line (or in | 
| 176 directories in [`paths`](#paths), if none are passed) whose basenames match this | 186 directories in [`paths`](#paths), if none are passed) whose basenames match this | 
| 177 pattern will be loaded and run as tests. | 187 pattern will be loaded and run as tests. | 
| 178 | 188 | 
| 179 This supports the full [glob syntax][]. However, since it's only compared | 189 This supports the full [glob syntax][]. However, since it's only compared | 
| 180 against a path's basename, path separators aren't especially useful. It defaults | 190 against a path's basename, path separators aren't especially useful. It defaults | 
| 181 to `"*_test.dart"`. | 191 to `"*_test.dart"`. | 
| 182 | 192 | 
| 183 ```yaml | 193 ```yaml | 
| 184 filename: "test_*.dart" | 194 filename: "test_*.dart" | 
| 185 ``` | 195 ``` | 
| 186 | 196 | 
| 187 [glob syntax]: https://github.com/dart-lang/glob#syntax | 197 [glob syntax]: https://github.com/dart-lang/glob#syntax | 
| 188 | 198 | 
|  | 199 This field is not supported in the | 
|  | 200 [global configuration file](#global-configuration). | 
|  | 201 | 
| 189 ### `names` | 202 ### `names` | 
| 190 | 203 | 
| 191 This field causes the runner to only run tests whose names match the given | 204 This field causes the runner to only run tests whose names match the given | 
| 192 regular expressions. A test's name must match *all* regular expressions in | 205 regular expressions. A test's name must match *all* regular expressions in | 
| 193 `names`, as well as containing all strings in [`plain_names`](#plain_names), in | 206 `names`, as well as containing all strings in [`plain_names`](#plain_names), in | 
| 194 order to be run. | 207 order to be run. | 
| 195 | 208 | 
| 196 This is usually used in a [preset](#configuration-presets) to make it possible | 209 This is usually used in a [preset](#configuration-presets) to make it possible | 
| 197 to quickly select a given set of tests. | 210 to quickly select a given set of tests. | 
| 198 | 211 | 
| 199 ```yaml | 212 ```yaml | 
| 200 presets: | 213 presets: | 
| 201   # Pass "-P chrome" to run only Chrome tests. | 214   # Pass "-P chrome" to run only Chrome tests. | 
| 202   chrome: | 215   chrome: | 
| 203     names: | 216     names: | 
| 204     - "^browser:" | 217     - "^browser:" | 
| 205     - "[Cc]hrome" | 218     - "[Cc]hrome" | 
| 206 ``` | 219 ``` | 
| 207 | 220 | 
|  | 221 This field is not supported in the | 
|  | 222 [global configuration file](#global-configuration). | 
|  | 223 | 
| 208 ### `plain_names` | 224 ### `plain_names` | 
| 209 | 225 | 
| 210 This field causes the runner to only run tests whose names contain the given | 226 This field causes the runner to only run tests whose names contain the given | 
| 211 strings. A test's name must contain *all* strings in `plain_names`, as well as | 227 strings. A test's name must contain *all* strings in `plain_names`, as well as | 
| 212 matching all regular expressions in [`names`](#names), in order to be run. | 228 matching all regular expressions in [`names`](#names), in order to be run. | 
| 213 | 229 | 
| 214 This is usually used in a [preset](#configuration-presets) to make it possible | 230 This is usually used in a [preset](#configuration-presets) to make it possible | 
| 215 to quickly select a given set of tests. | 231 to quickly select a given set of tests. | 
| 216 | 232 | 
| 217 ```yaml | 233 ```yaml | 
| 218 presets: | 234 presets: | 
| 219   # Pass "-P ie" to run only Internet Explorer tests. | 235   # Pass "-P ie" to run only Internet Explorer tests. | 
| 220   ie: | 236   ie: | 
| 221     plain_names: | 237     plain_names: | 
| 222     - "IE" | 238     - "IE" | 
| 223     - "Internet Explorer" | 239     - "Internet Explorer" | 
| 224 ``` | 240 ``` | 
| 225 | 241 | 
|  | 242 This field is not supported in the | 
|  | 243 [global configuration file](#global-configuration). | 
|  | 244 | 
| 226 ### `include_tags` | 245 ### `include_tags` | 
| 227 | 246 | 
| 228 This field causes the runner to only run tests whose tags match the given | 247 This field causes the runner to only run tests whose tags match the given | 
| 229 [boolean selector][]. If both `include_tags` and [`exclude_tags`](#exclude_tags) | 248 [boolean selector][]. If both `include_tags` and [`exclude_tags`](#exclude_tags) | 
| 230 are used, the exclusions take precedence. | 249 are used, the exclusions take precedence. | 
| 231 | 250 | 
| 232 [boolean selector]: https://github.com/dart-lang/boolean_selector/blob/master/RE
     ADME.md | 251 [boolean selector]: https://github.com/dart-lang/boolean_selector/blob/master/RE
     ADME.md | 
| 233 | 252 | 
| 234 This is usually used in a [preset](#configuration-preset) to make it possible to | 253 This is usually used in a [preset](#configuration-preset) to make it possible to | 
| 235 quickly select a set of tests. | 254 quickly select a set of tests. | 
| 236 | 255 | 
| 237 ```yaml | 256 ```yaml | 
| 238 presets: | 257 presets: | 
| 239   # Pass "-P windowless" to run tests that don't open browser windows. | 258   # Pass "-P windowless" to run tests that don't open browser windows. | 
| 240   windowless: | 259   windowless: | 
| 241     include_tags: !browser || content-shell | 260     include_tags: !browser || content-shell | 
| 242 ``` | 261 ``` | 
| 243 | 262 | 
|  | 263 This field is not supported in the | 
|  | 264 [global configuration file](#global-configuration). | 
|  | 265 | 
| 244 ### `exclude_tags` | 266 ### `exclude_tags` | 
| 245 | 267 | 
| 246 This field causes the runner not to run tests whose tags match the given | 268 This field causes the runner not to run tests whose tags match the given | 
| 247 [boolean selector][]. If both [`include_tags`](#include_tags) and `exclude_tags` | 269 [boolean selector][]. If both [`include_tags`](#include_tags) and `exclude_tags` | 
| 248 are used, the exclusions take precedence. | 270 are used, the exclusions take precedence. | 
| 249 | 271 | 
| 250 This is usually used in a [preset](#configuration-preset) to make it possible to | 272 This is usually used in a [preset](#configuration-preset) to make it possible to | 
| 251 quickly select a set of tests. | 273 quickly select a set of tests. | 
| 252 | 274 | 
| 253 ```yaml | 275 ```yaml | 
| 254 presets: | 276 presets: | 
| 255   # Pass "-P windowless" to run tests that don't open browser windows. | 277   # Pass "-P windowless" to run tests that don't open browser windows. | 
| 256   windowless: | 278   windowless: | 
| 257     exclude_tags: browser && !content-shell | 279     exclude_tags: browser && !content-shell | 
| 258 ``` | 280 ``` | 
| 259 | 281 | 
|  | 282 This field is not supported in the | 
|  | 283 [global configuration file](#global-configuration). | 
|  | 284 | 
| 260 ### `platforms` | 285 ### `platforms` | 
| 261 | 286 | 
| 262 This field indicates which platforms tests should run on by default. It allows | 287 This field indicates which platforms tests should run on by default. It allows | 
| 263 the same platform identifiers that can be passed to `--platform`. If multiple | 288 the same platform identifiers that can be passed to `--platform`. If multiple | 
| 264 platforms are included, the test runner will default to running tests on all of | 289 platforms are included, the test runner will default to running tests on all of | 
| 265 them. This defaults to `[vm]`. | 290 them. This defaults to `[vm]`. | 
| 266 | 291 | 
| 267 ```yaml | 292 ```yaml | 
| 268 platforms: [content_shell] | 293 platforms: [content_shell] | 
| 269 | 294 | 
| (...skipping 46 matching lines...) Expand 10 before | Expand all | Expand 10 after  Loading... | 
| 316 ### `reporter` | 341 ### `reporter` | 
| 317 | 342 | 
| 318 This field indicates the default reporter to use. It may be set to "compact", | 343 This field indicates the default reporter to use. It may be set to "compact", | 
| 319 "expanded", or "json" (although why anyone would want to default to JSON is | 344 "expanded", or "json" (although why anyone would want to default to JSON is | 
| 320 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. | 345 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. | 
| 321 | 346 | 
| 322 ```yaml | 347 ```yaml | 
| 323 reporter: expanded | 348 reporter: expanded | 
| 324 ``` | 349 ``` | 
| 325 | 350 | 
|  | 351 This field is not supported in the | 
|  | 352 [global configuration file](#global-configuration). | 
|  | 353 | 
| 326 ## Configuring Tags | 354 ## Configuring Tags | 
| 327 | 355 | 
| 328 ### `tags` | 356 ### `tags` | 
| 329 | 357 | 
| 330 The `tag` field can be used to apply [test configuration](#test-configuration) | 358 The `tag` field can be used to apply [test configuration](#test-configuration) | 
| 331 to all tests [with a given tag][tagging tests] or set of tags. It takes a map | 359 to all tests [with a given tag][tagging tests] or set of tags. It takes a map | 
| 332 from tag selectors to configuration maps. These configuration maps are just like | 360 from tag selectors to configuration maps. These configuration maps are just like | 
| 333 the top level of the configuration file, except that they may not contain | 361 the top level of the configuration file, except that they may not contain | 
| 334 [runner configuration](#runner-configuration). | 362 [runner configuration](#runner-configuration). | 
| 335 | 363 | 
| (...skipping 35 matching lines...) Expand 10 before | Expand all | Expand 10 after  Loading... | 
| 371 Tag configuration is applied at whatever level the tag appears—so if a group is | 399 Tag configuration is applied at whatever level the tag appears—so if a group is | 
| 372 tagged as `integration`, its timeout will take precedence over the suite's | 400 tagged as `integration`, its timeout will take precedence over the suite's | 
| 373 timeout but not any tests'. If the group itself had a timeout declared, the | 401 timeout but not any tests'. If the group itself had a timeout declared, the | 
| 374 group's explicit timeout would take precedence over the tag. | 402 group's explicit timeout would take precedence over the tag. | 
| 375 | 403 | 
| 376 If multiple tags appear at the same level, and they have conflicting | 404 If multiple tags appear at the same level, and they have conflicting | 
| 377 configurations, the test runner *does not guarantee* what order they'll be | 405 configurations, the test runner *does not guarantee* what order they'll be | 
| 378 resolved in. In practice, conflicting configuration is pretty unlikely and it's | 406 resolved in. In practice, conflicting configuration is pretty unlikely and it's | 
| 379 easy to just explicitly specify what you want on the test itself. | 407 easy to just explicitly specify what you want on the test itself. | 
| 380 | 408 | 
| 381 This field counts as [test configuration](#test-configuration). | 409 This field counts as [test configuration](#test-configuration). It is not | 
|  | 410 supported in the [global configuration file](#global-configuration). | 
| 382 | 411 | 
| 383 ### `add_tags` | 412 ### `add_tags` | 
| 384 | 413 | 
| 385 This field adds additional tags. It's technically | 414 This field adds additional tags. It's technically | 
| 386 [test configuration](#test-configuration), but it's usually used in more | 415 [test configuration](#test-configuration), but it's usually used in more | 
| 387 specific contexts. For example, when included in a tag's configuration, it can | 416 specific contexts. For example, when included in a tag's configuration, it can | 
| 388 be used to enable tag inheritance, where adding one tag implicitly adds another | 417 be used to enable tag inheritance, where adding one tag implicitly adds another | 
| 389 as well. It takes a list of tag name strings. | 418 as well. It takes a list of tag name strings. | 
| 390 | 419 | 
| 391 ```yaml | 420 ```yaml | 
| 392 tags: | 421 tags: | 
| 393   # Any test that spawns a browser. | 422   # Any test that spawns a browser. | 
| 394   browser: | 423   browser: | 
| 395     timeout: 2x | 424     timeout: 2x | 
| 396 | 425 | 
| 397   # Tests that spawn specific browsers. These automatically get the browser tag | 426   # Tests that spawn specific browsers. These automatically get the browser tag | 
| 398   # as well. | 427   # as well. | 
| 399   chrome: {add_tags: [browser]} | 428   chrome: {add_tags: [browser]} | 
| 400   firefox: {add_tags: [browser]} | 429   firefox: {add_tags: [browser]} | 
| 401   safari: {add_tags: [browser]} | 430   safari: {add_tags: [browser]} | 
| 402   ie: {add_tags: [browser]} | 431   ie: {add_tags: [browser]} | 
| 403 ``` | 432 ``` | 
| 404 | 433 | 
|  | 434 This field is not supported in the | 
|  | 435 [global configuration file](#global-configuration). | 
|  | 436 | 
| 405 ## Configuring Platforms | 437 ## Configuring Platforms | 
| 406 | 438 | 
| 407 There are two different kinds of platform configuration. | 439 There are two different kinds of platform configuration. | 
| 408 [Operating system configuration](#on_os) cares about the operating system on | 440 [Operating system configuration](#on_os) cares about the operating system on | 
| 409 which test runner is running. It sets global configuration for the runner on | 441 which test runner is running. It sets global configuration for the runner on | 
| 410 particular OSes. [Test platform configuration](#on_platform), on the other hand, | 442 particular OSes. [Test platform configuration](#on_platform), on the other hand, | 
| 411 cares about the platform the *test* is running on (like the | 443 cares about the platform the *test* is running on (like the | 
| 412 [`@OnPlatform` annotation][@OnPlatform]). It sets configuration for particular | 444 [`@OnPlatform` annotation][@OnPlatform]). It sets configuration for particular | 
| 413 tests that are running on matching platforms. | 445 tests that are running on matching platforms. | 
| 414 | 446 | 
| (...skipping 107 matching lines...) Expand 10 before | Expand all | Expand 10 after  Loading... | 
| 522 presets: | 554 presets: | 
| 523   # Shortcut for running only browser tests. | 555   # Shortcut for running only browser tests. | 
| 524   browser: | 556   browser: | 
| 525     paths: [test/runner/browser] | 557     paths: [test/runner/browser] | 
| 526 | 558 | 
| 527   # Shortcut for running only Chrome tests. | 559   # Shortcut for running only Chrome tests. | 
| 528   chrome: | 560   chrome: | 
| 529     filename: "chrome_*_test.dart" | 561     filename: "chrome_*_test.dart" | 
| 530     add_presets: [browser] | 562     add_presets: [browser] | 
| 531 ``` | 563 ``` | 
|  | 564 | 
|  | 565 ## Global Configuration | 
|  | 566 | 
|  | 567 The test runner also supports a global configuration file. On Windows, this | 
|  | 568 file's local defaults to `%LOCALAPPDATA%\DartTest.yaml`. On Unix, it defaults to | 
|  | 569 `~/.dart_test.yaml`. It can also be explicitly set using the `DART_TEST_CONFIG` | 
|  | 570 environment variable. | 
|  | 571 | 
|  | 572 The global configuration file supports a subset of the fields supported by the | 
|  | 573 package-specific configuration file. In general, it doesn't support fields that | 
|  | 574 are closely tied to the structure of an individual package. Fields that are not | 
|  | 575 supported in the global configuration file say so in their documentation. | 
| OLD | NEW | 
|---|