Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(76)

Side by Side Diff: doc/package_config.md

Issue 1691173002: Support tag configuration. (Closed) Base URL: git@github.com:dart-lang/test@master
Patch Set: Code review changes Created 4 years, 10 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « README.md ('k') | lib/src/backend/metadata.dart » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Each package may include a configuration file that applies to the package as a 1 Each package may include a configuration file that applies to the package as a
2 whole. This file can be used to provide custom defaults for various options, to 2 whole. This file can be used to provide custom defaults for various options, to
3 define configuration for multiple files, and more. 3 define configuration for multiple files, and more.
4 4
5 The file is named `dart_test.yaml` and lives at the root of the package, next to 5 The file is named `dart_test.yaml` and lives at the root of the package, next to
6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an 6 the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an
7 example: 7 example:
8 8
9 [YAML]: http://yaml.org/ 9 [YAML]: http://yaml.org/
10 10
11 ```yaml 11 ```yaml
12 # This package's tests are very slow. Double the default timeout. 12 # This package's tests are very slow. Double the default timeout.
13 timeout: 2x 13 timeout: 2x
14 14
15 # This is a browser-only package, so test on content shell by default. 15 # This is a browser-only package, so test on content shell by default.
16 platforms: [content-shell] 16 platforms: [content-shell]
17
18 tags:
19 # Integration tests are even slower, so increase the timeout again.
20 integration: {timeout: 2x}
21
22 # Sanity tests are quick and verify that nothing is obviously wrong. Declaring
23 # the tag allows us to tag tests with it without getting warnings.
24 sanity:
17 ``` 25 ```
18 26
19 * [Config Fields](#config-fields) 27 * [Test Configuration](#test-configuration)
28 * [`timeout`](#timeout)
29 * [`verbose_trace`](#verbose_trace)
30 * [`js_trace`](#js_trace)
31 * [Runner Configuration](#runner-configuration)
20 * [`paths`](#paths) 32 * [`paths`](#paths)
21 * [`filename`](#filename) 33 * [`filename`](#filename)
22 * [`platforms`](#platforms) 34 * [`platforms`](#platforms)
23 * [`concurrency`](#concurrency) 35 * [`concurrency`](#concurrency)
24 * [`pub_serve`](#pub_serve) 36 * [`pub_serve`](#pub_serve)
25 * [`timeout`](#timeout)
26 * [`reporter`](#reporter) 37 * [`reporter`](#reporter)
27 * [`verbose_trace`](#verbose_trace) 38 * [Configuring Tags](#configuring-tags)
28 * [`js_trace`](#js_trace)
29 39
30 ## Config Fields 40 ## Test Configuration
31 41
32 These fields directly affect the behavior of the test runner. They go at the 42 There are two major categories of configuration field: "test" and "runner". Test
33 root of the configuration file. 43 configuration controls how individual tests run, while
44 [runner configuration](#runner-configuration) controls the test runner as a
45 whole. Both types of fields may be used at the top level of a configuration
46 file. However, because different tests can have different test configuration,
47 only test configuration fields may be used to
48 [configure tags](#configuring-tags).
49
50 ### `timeout`
51
52 This field indicates how much time the test runner should allow a test to remain
53 inactive before it considers that test to have failed. It has three possible
54 formats:
55
56 * The string "none" indicates that tests should never time out.
57
58 * A number followed by a unit abbreviation indicates an exact time. For example,
59 "1m" means a timeout of one minute, and "30s" means a timeout of thirty
60 seconds. Multiple numbers can be combined, as in "1m 30s".
61
62 * A number followed by "x" indicates a multiple. This is applied to the default
63 value of 30s.
64
65 ```yaml
66 timeout: 1m
67 ```
68
69 ### `verbose_trace`
70
71 This boolean field controls whether or not stack traces caused by errors are
72 trimmed to remove internal stack frames. This includes frames from the Dart core
73 libraries, the [`stack_trace`][stack_trace] package, and the `test` package
74 itself. It defaults to `false`.
75
76 [stack_trace]: https://pub.dartlang.org/packages/stack_trace
77
78 ```yaml
79 verbose_trace: true
80 ```
81
82 ### `js_trace`
83
84 This boolean field controls whether or not stack traces caused by errors that
85 occur while running Dart compiled to JS are converted back to Dart style. This
86 conversion uses the source map generated by `dart2js` to approximate the
87 original Dart line, column, and in some cases member name for each stack frame.
88 It defaults to `false`.
89
90 ```yaml
91 js_trace: true
92 ```
93
94 ## Runner Configuration
95
96 Unlike [test configuration](#test-configuration), runner configuration affects
97 the test runner as a whole rather than individual tests. It can only be used at
98 the top level of the configuration file.
34 99
35 ### `paths` 100 ### `paths`
36 101
37 This field indicates the default paths that the test runner should run. These 102 This field indicates the default paths that the test runner should run. These
38 paths are usually directories, although single filenames may be used as well. 103 paths are usually directories, although single filenames may be used as well.
39 Paths must be relative, and they must be in URL format so that they're 104 Paths must be relative, and they must be in URL format so that they're
40 compatible across operating systems. This defaults to `[test]`. 105 compatible across operating systems. This defaults to `[test]`.
41 106
42 ```yaml 107 ```yaml
43 paths: [dart/test] 108 paths: [dart/test]
(...skipping 52 matching lines...) Expand 10 before | Expand all | Expand 10 after
96 161
97 This field indicates that the test runner should run against a `pub serve` 162 This field indicates that the test runner should run against a `pub serve`
98 instance by default, and provides the port number for that instance. Note that 163 instance by default, and provides the port number for that instance. Note that
99 if there is no `pub serve` instance running at that port, running the tests will 164 if there is no `pub serve` instance running at that port, running the tests will
100 fail by default. 165 fail by default.
101 166
102 ```yaml 167 ```yaml
103 pub_serve: 8081 168 pub_serve: 8081
104 ``` 169 ```
105 170
106 ### `timeout`
107
108 This field indicates how much time the test runner should allow a test to remain
109 inactive before it considers that test to have failed. It has three possible
110 formats:
111
112 * The string "none" indicates that tests should never time out.
113
114 * A number followed by a unit abbreviation indicates an exact time. For example,
115 "1m" means a timeout of one minute, and "30s" means a timeout of thirty
116 seconds. Multiple numbers can be combined, as in "1m 30s".
117
118 * A number followed by "x" indicates a multiple. This is applied to the default
119 value of 30s.
120
121 ```yaml
122 timeout: 1m
123 ```
124
125 ### `reporter` 171 ### `reporter`
126 172
127 This field indicates the default reporter to use. It may be set to "compact", 173 This field indicates the default reporter to use. It may be set to "compact",
128 "expanded", or "json" (although why anyone would want to default to JSON is 174 "expanded", or "json" (although why anyone would want to default to JSON is
129 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. 175 beyond me). It defaults to "expanded" on Windows and "compact" everywhere else.
130 176
131 ```yaml 177 ```yaml
132 reporter: expanded 178 reporter: expanded
133 ``` 179 ```
134 180
135 ### `verbose_trace` 181 ## Configuring Tags
136 182
137 This boolean field controls whether or not stack traces caused by errors are 183 The `tag` field can be used to apply [test configuration](#test-configuration)
138 trimmed to remove internal stack frames. This includes frames from the Dart core 184 to all tests [with a given tag][tagging tests]. It takes a map from tag names to
139 libraries, the [`stack_trace`][stack_trace] package, and the `test` package 185 configuration maps. These configuration maps are just like the top level of the
140 itself. It defaults to `false`. 186 configuration file, except that they may not contain
187 [runner configuration](#runner-configuration).
141 188
142 [stack_trace]: https://pub.dartlang.org/packages/stack_trace 189 [tagging tests]: https://github.com/dart-lang/test/blob/master/README.md#tagging -tests
143 190
144 ```yaml 191 ```yaml
145 verbose_trace: true 192 tags:
193 # Integration tests need more time to run.
194 integration:
195 timeout: 1m
146 ``` 196 ```
147 197
148 ### `js_trace` 198 Tags may also have no configuration associated with them. The test runner prints
149 199 a warning whenever it encounters a tag it doesn't recognize, and this the best
150 This boolean field controls whether or not stack traces caused by errors that 200 way to tell it that a tag exists.
151 occur while running Dart compiled to JS are converted back to Dart style. This
152 conversion uses the source map generated by `dart2js` to approximate the
153 original Dart line, column, and in some cases member name for each stack frame.
154 It defaults to `false`.
155 201
156 ```yaml 202 ```yaml
157 js_trace: true 203 # We occasionally want to use --tags or --exclude-tags on these tags.
204 tags:
205 # A test that spawns a browser.
206 browser:
207
208 # A test that needs Ruby installed.
209 ruby:
158 ``` 210 ```
211
212 Tag configuration is applied at whatever level the tag appears—so if a group is
213 tagged as `integration`, its timeout will take precedence over the suite's
214 timeout but not any tests'. If the group itself had a timeout declared, the
215 group's explicit timeout would take precedence over the tag.
216
217 If multiple tags appear at the same level, and they have conflicting
218 configurations, the test runner *does not guarantee* what order they'll be
219 resolved in. In practice, conflicting configuration is pretty unlikely and it's
220 easy to just explicitly specify what you want on the test itself.
OLDNEW
« no previous file with comments | « README.md ('k') | lib/src/backend/metadata.dart » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698