OLD | NEW |
1 Each package may include a configuration file that applies to the package as a | 1 Each package may include a configuration file that applies to the package as a |
2 whole. This file can be used to provide custom defaults for various options, to | 2 whole. This file can be used to provide custom defaults for various options, to |
3 define configuration for multiple files, and more. | 3 define configuration for multiple files, and more. |
4 | 4 |
5 The file is named `dart_test.yaml` and lives at the root of the package, next to | 5 The file is named `dart_test.yaml` and lives at the root of the package, next to |
6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an | 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an |
7 example: | 7 example: |
8 | 8 |
9 [YAML]: http://yaml.org/ | 9 [YAML]: http://yaml.org/ |
10 | 10 |
11 ```yaml | 11 ```yaml |
12 # This package's tests are very slow. Double the default timeout. | 12 # This package's tests are very slow. Double the default timeout. |
13 timeout: 2x | 13 timeout: 2x |
14 | 14 |
15 # This is a browser-only package, so test on content shell by default. | 15 # This is a browser-only package, so test on content shell by default. |
16 platforms: [content-shell] | 16 platforms: [content-shell] |
| 17 |
| 18 tags: |
| 19 # Integration tests are even slower, so increase the timeout again. |
| 20 integration: {timeout: 2x} |
| 21 |
| 22 # Sanity tests are quick and verify that nothing is obviously wrong. Declaring |
| 23 # the tag allows us to tag tests with it without getting warnings. |
| 24 sanity: |
17 ``` | 25 ``` |
18 | 26 |
19 * [Config Fields](#config-fields) | 27 * [Test Configuration](#test-configuration) |
| 28 * [`timeout`](#timeout) |
| 29 * [`verbose_trace`](#verbose_trace) |
| 30 * [`js_trace`](#js_trace) |
| 31 * [Runner Configuration](#runner-configuration) |
20 * [`paths`](#paths) | 32 * [`paths`](#paths) |
21 * [`filename`](#filename) | 33 * [`filename`](#filename) |
22 * [`platforms`](#platforms) | 34 * [`platforms`](#platforms) |
23 * [`concurrency`](#concurrency) | 35 * [`concurrency`](#concurrency) |
24 * [`pub_serve`](#pub_serve) | 36 * [`pub_serve`](#pub_serve) |
25 * [`timeout`](#timeout) | |
26 * [`reporter`](#reporter) | 37 * [`reporter`](#reporter) |
27 * [`verbose_trace`](#verbose_trace) | 38 * [Configuring Tags](#configuring-tags) |
28 * [`js_trace`](#js_trace) | |
29 | 39 |
30 ## Config Fields | 40 ## Test Configuration |
31 | 41 |
32 These fields directly affect the behavior of the test runner. They go at the | 42 There are two major categories of configuration field: "test" and "runner". Test |
33 root of the configuration file. | 43 configuration controls how individual tests run, while |
| 44 [runner configuration](#runner-configuration) controls the test runner as a |
| 45 whole. Both types of fields may be used at the top level of a configuration |
| 46 file. However, because different tests can have different test configuration, |
| 47 only test configuration fields may be used to |
| 48 [configure tags](#configuring-tags). |
| 49 |
| 50 ### `timeout` |
| 51 |
| 52 This field indicates how much time the test runner should allow a test to remain |
| 53 inactive before it considers that test to have failed. It has three possible |
| 54 formats: |
| 55 |
| 56 * The string "none" indicates that tests should never time out. |
| 57 |
| 58 * A number followed by a unit abbreviation indicates an exact time. For example, |
| 59 "1m" means a timeout of one minute, and "30s" means a timeout of thirty |
| 60 seconds. Multiple numbers can be combined, as in "1m 30s". |
| 61 |
| 62 * A number followed by "x" indicates a multiple. This is applied to the default |
| 63 value of 30s. |
| 64 |
| 65 ```yaml |
| 66 timeout: 1m |
| 67 ``` |
| 68 |
| 69 ### `verbose_trace` |
| 70 |
| 71 This boolean field controls whether or not stack traces caused by errors are |
| 72 trimmed to remove internal stack frames. This includes frames from the Dart core |
| 73 libraries, the [`stack_trace`][stack_trace] package, and the `test` package |
| 74 itself. It defaults to `false`. |
| 75 |
| 76 [stack_trace]: https://pub.dartlang.org/packages/stack_trace |
| 77 |
| 78 ```yaml |
| 79 verbose_trace: true |
| 80 ``` |
| 81 |
| 82 ### `js_trace` |
| 83 |
| 84 This boolean field controls whether or not stack traces caused by errors that |
| 85 occur while running Dart compiled to JS are converted back to Dart style. This |
| 86 conversion uses the source map generated by `dart2js` to approximate the |
| 87 original Dart line, column, and in some cases member name for each stack frame. |
| 88 It defaults to `false`. |
| 89 |
| 90 ```yaml |
| 91 js_trace: true |
| 92 ``` |
| 93 |
| 94 ## Runner Configuration |
| 95 |
| 96 Unlike [test configuration](#test-configuration), runner configuration affects |
| 97 the test runner as a whole rather than individual tests. It can only be used at |
| 98 the top level of the configuration file. |
34 | 99 |
35 ### `paths` | 100 ### `paths` |
36 | 101 |
37 This field indicates the default paths that the test runner should run. These | 102 This field indicates the default paths that the test runner should run. These |
38 paths are usually directories, although single filenames may be used as well. | 103 paths are usually directories, although single filenames may be used as well. |
39 Paths must be relative, and they must be in URL format so that they're | 104 Paths must be relative, and they must be in URL format so that they're |
40 compatible across operating systems. This defaults to `[test]`. | 105 compatible across operating systems. This defaults to `[test]`. |
41 | 106 |
42 ```yaml | 107 ```yaml |
43 paths: [dart/test] | 108 paths: [dart/test] |
(...skipping 52 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
96 | 161 |
97 This field indicates that the test runner should run against a `pub serve` | 162 This field indicates that the test runner should run against a `pub serve` |
98 instance by default, and provides the port number for that instance. Note that | 163 instance by default, and provides the port number for that instance. Note that |
99 if there is no `pub serve` instance running at that port, running the tests will | 164 if there is no `pub serve` instance running at that port, running the tests will |
100 fail by default. | 165 fail by default. |
101 | 166 |
102 ```yaml | 167 ```yaml |
103 pub_serve: 8081 | 168 pub_serve: 8081 |
104 ``` | 169 ``` |
105 | 170 |
106 ### `timeout` | |
107 | |
108 This field indicates how much time the test runner should allow a test to remain | |
109 inactive before it considers that test to have failed. It has three possible | |
110 formats: | |
111 | |
112 * The string "none" indicates that tests should never time out. | |
113 | |
114 * A number followed by a unit abbreviation indicates an exact time. For example, | |
115 "1m" means a timeout of one minute, and "30s" means a timeout of thirty | |
116 seconds. Multiple numbers can be combined, as in "1m 30s". | |
117 | |
118 * A number followed by "x" indicates a multiple. This is applied to the default | |
119 value of 30s. | |
120 | |
121 ```yaml | |
122 timeout: 1m | |
123 ``` | |
124 | |
125 ### `reporter` | 171 ### `reporter` |
126 | 172 |
127 This field indicates the default reporter to use. It may be set to "compact", | 173 This field indicates the default reporter to use. It may be set to "compact", |
128 "expanded", or "json" (although why anyone would want to default to JSON is | 174 "expanded", or "json" (although why anyone would want to default to JSON is |
129 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. | 175 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. |
130 | 176 |
131 ```yaml | 177 ```yaml |
132 reporter: expanded | 178 reporter: expanded |
133 ``` | 179 ``` |
134 | 180 |
135 ### `verbose_trace` | 181 ## Configuring Tags |
136 | 182 |
137 This boolean field controls whether or not stack traces caused by errors are | 183 The `tag` field can be used to apply [test configuration](#test-configuration) |
138 trimmed to remove internal stack frames. This includes frames from the Dart core | 184 to all tests [with a given tag][tagging tests]. It takes a map from tag names to |
139 libraries, the [`stack_trace`][stack_trace] package, and the `test` package | 185 configuration maps. These configuration maps are just like the top level of the |
140 itself. It defaults to `false`. | 186 configuration file, except that they may not contain |
| 187 [runner configuration](#runner-configuration). |
141 | 188 |
142 [stack_trace]: https://pub.dartlang.org/packages/stack_trace | 189 [tagging tests]: https://github.com/dart-lang/test/blob/master/README.md#tagging
-tests |
143 | 190 |
144 ```yaml | 191 ```yaml |
145 verbose_trace: true | 192 tags: |
| 193 # Integration tests need more time to run. |
| 194 integration: |
| 195 timeout: 1m |
146 ``` | 196 ``` |
147 | 197 |
148 ### `js_trace` | 198 Tags may also have no configuration associated with them. The test runner prints |
149 | 199 a warning whenever it encounters a tag it doesn't recognize, and this the best |
150 This boolean field controls whether or not stack traces caused by errors that | 200 way to tell it that a tag exists. |
151 occur while running Dart compiled to JS are converted back to Dart style. This | |
152 conversion uses the source map generated by `dart2js` to approximate the | |
153 original Dart line, column, and in some cases member name for each stack frame. | |
154 It defaults to `false`. | |
155 | 201 |
156 ```yaml | 202 ```yaml |
157 js_trace: true | 203 # We occasionally want to use --tags or --exclude-tags on these tags. |
| 204 tags: |
| 205 # A test that spawns a browser. |
| 206 browser: |
| 207 |
| 208 # A test that needs Ruby installed. |
| 209 ruby: |
158 ``` | 210 ``` |
| 211 |
| 212 Tag configuration is applied at whatever level the tag appears—so if a group is |
| 213 tagged as `integration`, its timeout will take precedence over the suite's |
| 214 timeout but not any tests'. If the group itself had a timeout declared, the |
| 215 group's explicit timeout would take precedence over the tag. |
| 216 |
| 217 If multiple tags appear at the same level, and they have conflicting |
| 218 configurations, the test runner *does not guarantee* what order they'll be |
| 219 resolved in. In practice, conflicting configuration is pretty unlikely and it's |
| 220 easy to just explicitly specify what you want on the test itself. |
OLD | NEW |