Index: doc/package_config.md |
diff --git a/doc/package_config.md b/doc/package_config.md |
index e13824bc76a788261ceecaf83ab0f1addc3f01b9..c110ec3e0b08702c9b89293bb6e39104c119fea9 100644 |
--- a/doc/package_config.md |
+++ b/doc/package_config.md |
@@ -14,23 +14,88 @@ timeout: 2x |
# This is a browser-only package, so test on content shell by default. |
platforms: [content-shell] |
+ |
+tags: |
+ # Integration tests are even slower, so increase the timeout again. |
+ integration: {timeout: 2x} |
+ |
+ # Sanity tests are quick and verify that nothing is obviously wrong. Declaring |
+ # the tag allows us to tag tests with it without getting warnings. |
+ sanity: |
``` |
-* [Config Fields](#config-fields) |
+* [Test Configuration](#test-configuration) |
+ * [`timeout`](#timeout) |
+ * [`verbose_trace`](#verbose_trace) |
+ * [`js_trace`](#js_trace) |
+* [Runner Configuration](#runner-configuration) |
* [`paths`](#paths) |
* [`filename`](#filename) |
* [`platforms`](#platforms) |
* [`concurrency`](#concurrency) |
* [`pub_serve`](#pub_serve) |
- * [`timeout`](#timeout) |
* [`reporter`](#reporter) |
- * [`verbose_trace`](#verbose_trace) |
- * [`js_trace`](#js_trace) |
+* [Configuring Tags](#configuring-tags) |
+ |
+## Test Configuration |
+ |
+There are two major categories of configuration field: "test" and "runner". Test |
+configuration controls how individual tests run, while |
+[runner configuration](#runner-configuration) controls the test runner as a |
+whole. Both types of fields may be used at the top level of a configuration |
+file. However, because different tests can have different test configuration, |
+only test configuration fields may be used to |
+[configure tags](#configuring-tags). |
+ |
+### `timeout` |
+ |
+This field indicates how much time the test runner should allow a test to remain |
+inactive before it considers that test to have failed. It has three possible |
+formats: |
+ |
+* The string "none" indicates that tests should never time out. |
+ |
+* A number followed by a unit abbreviation indicates an exact time. For example, |
+ "1m" means a timeout of one minute, and "30s" means a timeout of thirty |
+ seconds. Multiple numbers can be combined, as in "1m 30s". |
+ |
+* A number followed by "x" indicates a multiple. This is applied to the default |
+ value of 30s. |
+ |
+```yaml |
+timeout: 1m |
+``` |
-## Config Fields |
+### `verbose_trace` |
-These fields directly affect the behavior of the test runner. They go at the |
-root of the configuration file. |
+This boolean field controls whether or not stack traces caused by errors are |
+trimmed to remove internal stack frames. This includes frames from the Dart core |
+libraries, the [`stack_trace`][stack_trace] package, and the `test` package |
+itself. It defaults to `false`. |
+ |
+[stack_trace]: https://pub.dartlang.org/packages/stack_trace |
+ |
+```yaml |
+verbose_trace: true |
+``` |
+ |
+### `js_trace` |
+ |
+This boolean field controls whether or not stack traces caused by errors that |
+occur while running Dart compiled to JS are converted back to Dart style. This |
+conversion uses the source map generated by `dart2js` to approximate the |
+original Dart line, column, and in some cases member name for each stack frame. |
+It defaults to `false`. |
+ |
+```yaml |
+js_trace: true |
+``` |
+ |
+## Runner Configuration |
+ |
+Unlike [test configuration](#test-configuration), runner configuration affects |
+the test runner as a whole rather than individual tests. It can only be used at |
+the top level of the configuration file. |
### `paths` |
@@ -103,25 +168,6 @@ fail by default. |
pub_serve: 8081 |
``` |
-### `timeout` |
- |
-This field indicates how much time the test runner should allow a test to remain |
-inactive before it considers that test to have failed. It has three possible |
-formats: |
- |
-* The string "none" indicates that tests should never time out. |
- |
-* A number followed by a unit abbreviation indicates an exact time. For example, |
- "1m" means a timeout of one minute, and "30s" means a timeout of thirty |
- seconds. Multiple numbers can be combined, as in "1m 30s". |
- |
-* A number followed by "x" indicates a multiple. This is applied to the default |
- value of 30s. |
- |
-```yaml |
-timeout: 1m |
-``` |
- |
### `reporter` |
This field indicates the default reporter to use. It may be set to "compact", |
@@ -132,27 +178,43 @@ beyond me). It defaults to "expanded" on Windows and "compact" everywhere else. |
reporter: expanded |
``` |
-### `verbose_trace` |
+## Configuring Tags |
-This boolean field controls whether or not stack traces caused by errors are |
-trimmed to remove internal stack frames. This includes frames from the Dart core |
-libraries, the [`stack_trace`][stack_trace] package, and the `test` package |
-itself. It defaults to `false`. |
+The `tag` field can be used to apply [test configuration](#test-configuration) |
+to all tests [with a given tag][tagging tests]. It takes a map from tag names to |
+configuration maps. These configuration maps are just like the top level of the |
+configuration file, except that they may not contain |
+[runner configuration](#runner-configuration). |
-[stack_trace]: https://pub.dartlang.org/packages/stack_trace |
+[tagging tests]: https://github.com/dart-lang/test/blob/master/README.md#tagging-tests |
```yaml |
-verbose_trace: true |
+tags: |
+ # Integration tests need more time to run. |
+ integration: |
+ timeout: 1m |
``` |
-### `js_trace` |
- |
-This boolean field controls whether or not stack traces caused by errors that |
-occur while running Dart compiled to JS are converted back to Dart style. This |
-conversion uses the source map generated by `dart2js` to approximate the |
-original Dart line, column, and in some cases member name for each stack frame. |
-It defaults to `false`. |
+Tags may also have no configuration associated with them. The test runner prints |
+a warning whenever it encounters a tag it doesn't recognize, and this the best |
+way to tell it that a tag exists. |
```yaml |
-js_trace: true |
+# We occasionally want to use --tags or --exclude-tags on these tags. |
+tags: |
+ # A test that spawns a browser. |
+ browser: |
+ |
+ # A test that needs Ruby installed. |
+ ruby: |
``` |
+ |
+Tag configuration is applied at whatever level the tag appears—so if a group is |
+tagged as `integration`, its timeout will take precedence over the suite's |
+timeout but not any tests'. If the group itself had a timeout declared, the |
+group's explicit timeout would take precedence over the tag. |
+ |
+If multiple tags appear at the same level, and they have conflicting |
+configurations, the test runner *does not guarantee* what order they'll be |
+resolved in. In practice, conflicting configuration is pretty unlikely and it's |
+easy to just explicitly specify what you want on the test itself. |