Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(61)

Unified Diff: doc/package_config.md

Issue 1829483002: Add support for a global configuration file. (Closed) Base URL: git@github.com:dart-lang/test@master
Patch Set: Code review changes Created 4 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « doc/configuration.md ('k') | lib/src/executable.dart » ('j') | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: doc/package_config.md
diff --git a/doc/package_config.md b/doc/package_config.md
deleted file mode 100644
index 974e7ce98269bd8a3ed418d125ae44fb60b52160..0000000000000000000000000000000000000000
--- a/doc/package_config.md
+++ /dev/null
@@ -1,531 +0,0 @@
-Each package may include a configuration file that applies to the package as a
-whole. This file can be used to provide custom defaults for various options, to
-define configuration for multiple files, and more.
-
-The file is named `dart_test.yaml` and lives at the root of the package, next to
-the package's pubspec. Like the pubspec, it's a [YAML][] file. Here's an
-example:
-
-[YAML]: http://yaml.org/
-
-```yaml
-# This package's tests are very slow. Double the default timeout.
-timeout: 2x
-
-# This is a browser-only package, so test on content shell by default.
-platforms: [content-shell]
-
-tags:
- # Integration tests are even slower, so increase the timeout again.
- integration: {timeout: 2x}
-
- # Sanity tests are quick and verify that nothing is obviously wrong. Declaring
- # the tag allows us to tag tests with it without getting warnings.
- sanity:
-```
-
-* [Test Configuration](#test-configuration)
- * [`timeout`](#timeout)
- * [`verbose_trace`](#verbose_trace)
- * [`js_trace`](#js_trace)
- * [`skip`](#skip)
- * [`test_on`](#test_on)
-* [Runner Configuration](#runner-configuration)
- * [`paths`](#paths)
- * [`filename`](#filename)
- * [`names`](#names)
- * [`plain_names`](#plain_names)
- * [`include_tags`](#include_tags)
- * [`exclude_tags`](#exclude_tags)
- * [`platforms`](#platforms)
- * [`concurrency`](#concurrency)
- * [`pause_after_load`](#pause_after_load)
- * [`pub_serve`](#pub_serve)
- * [`reporter`](#reporter)
-* [Configuring Tags](#configuring-tags)
- * [`tags`](#tags)
- * [`add_tags`](#add_tags)
-* [Configuring Platforms](#configuring-platforms)
- * [`on_os`](#on_os)
- * [`on_platform`](#on_platform)
-* [Configuration Presets](#configuration-presets)
- * [`presets`](#presets)
- * [`add_preset`](#add_preset)
-
-## Test Configuration
-
-There are two major categories of configuration field: "test" and "runner". Test
-configuration controls how individual tests run, while
-[runner configuration](#runner-configuration) controls the test runner as a
-whole. Both types of fields may be used at the top level of a configuration
-file. However, because different tests can have different test configuration,
-only test configuration fields may be used to [configure tags](#tags) or
-[platforms](#on_platform).
-
-### `timeout`
-
-This field indicates how much time the test runner should allow a test to remain
-inactive before it considers that test to have failed. It has three possible
-formats:
-
-* The string "none" indicates that tests should never time out.
-
-* A number followed by a unit abbreviation indicates an exact time. For example,
- "1m" means a timeout of one minute, and "30s" means a timeout of thirty
- seconds. Multiple numbers can be combined, as in "1m 30s".
-
-* A number followed by "x" indicates a multiple. This is applied to the default
- value of 30s.
-
-```yaml
-timeout: 1m
-```
-
-### `verbose_trace`
-
-This boolean field controls whether or not stack traces caused by errors are
-trimmed to remove internal stack frames. This includes frames from the Dart core
-libraries, the [`stack_trace`][stack_trace] package, and the `test` package
-itself. It defaults to `false`.
-
-[stack_trace]: https://pub.dartlang.org/packages/stack_trace
-
-```yaml
-verbose_trace: true
-```
-
-### `js_trace`
-
-This boolean field controls whether or not stack traces caused by errors that
-occur while running Dart compiled to JS are converted back to Dart style. This
-conversion uses the source map generated by `dart2js` to approximate the
-original Dart line, column, and in some cases member name for each stack frame.
-It defaults to `false`.
-
-```yaml
-js_trace: true
-```
-
-### `skip`
-
-This field controls whether or not tests are skipped. It's usually applied to
-[specific tags](#configuring-tags) rather than used at the top level. Like the
-`skip` parameter for [`test()`][test], it can either be a boolean indicating
-whether the tests are skipped or a string indicating the reason they're skipped.
-
-[test]: https://www.dartdocs.org/documentation/test/0.12.10+2/test/test.html
-
-```yaml
-tags:
- chrome:
- skip: "Our Chrome launcher is busted. See issue 1234."
-```
-
-### `test_on`
-
-This field declares which platforms a test supports. It takes a
-[platform selector][platform selectors] and only allows tests to run on
-platforms that match the selector. It's often used with
-[specific tags](#configuring-tags) to ensure that certain features will only be
-tested on supported platforms.
-
-[platform selectors]: https://github.com/dart-lang/test/blob/master/README.md#platform-selectors
-
-```yaml
-tags:
- # Internet Explorer doesn't support promises yet.
- promises: {test_on: "browser && !ie"}
-```
-
-The field can also be used at the top level of the configuration file to
-indicate that the entire package only supports a particular platform. If someone
-tries to run the tests on an unsupported platform, the runner will print a
-warning and skip that platform.
-
-```yaml
-# This package uses dart:io.
-test_on: vm
-```
-
-## Runner Configuration
-
-Unlike [test configuration](#test-configuration), runner configuration affects
-the test runner as a whole rather than individual tests. It can only be used at
-the top level of the configuration file.
-
-### `paths`
-
-This field indicates the default paths that the test runner should run. These
-paths are usually directories, although single filenames may be used as well.
-Paths must be relative, and they must be in URL format so that they're
-compatible across operating systems. This defaults to `[test]`.
-
-```yaml
-paths: [dart/test]
-
-paths:
-- test/instantaneous
-- test/fast
-- test/middling
-```
-
-### `filename`
-
-This field indicates the filename pattern that the test runner uses to find test
-files in directories. All files in directories passed on the command line (or in
-directories in [`paths`](#paths), if none are passed) whose basenames match this
-pattern will be loaded and run as tests.
-
-This supports the full [glob syntax][]. However, since it's only compared
-against a path's basename, path separators aren't especially useful. It defaults
-to `"*_test.dart"`.
-
-```yaml
-filename: "test_*.dart"
-```
-
-[glob syntax]: https://github.com/dart-lang/glob#syntax
-
-### `names`
-
-This field causes the runner to only run tests whose names match the given
-regular expressions. A test's name must match *all* regular expressions in
-`names`, as well as containing all strings in [`plain_names`](#plain_names), in
-order to be run.
-
-This is usually used in a [preset](#configuration-presets) to make it possible
-to quickly select a given set of tests.
-
-```yaml
-presets:
- # Pass "-P chrome" to run only Chrome tests.
- chrome:
- names:
- - "^browser:"
- - "[Cc]hrome"
-```
-
-### `plain_names`
-
-This field causes the runner to only run tests whose names contain the given
-strings. A test's name must contain *all* strings in `plain_names`, as well as
-matching all regular expressions in [`names`](#names), in order to be run.
-
-This is usually used in a [preset](#configuration-presets) to make it possible
-to quickly select a given set of tests.
-
-```yaml
-presets:
- # Pass "-P ie" to run only Internet Explorer tests.
- ie:
- plain_names:
- - "IE"
- - "Internet Explorer"
-```
-
-### `include_tags`
-
-This field causes the runner to only run tests whose tags match the given
-[boolean selector][]. If both `include_tags` and [`exclude_tags`](#exclude_tags)
-are used, the exclusions take precedence.
-
-[boolean selector]: https://github.com/dart-lang/boolean_selector/blob/master/README.md
-
-This is usually used in a [preset](#configuration-preset) to make it possible to
-quickly select a set of tests.
-
-```yaml
-presets:
- # Pass "-P windowless" to run tests that don't open browser windows.
- windowless:
- include_tags: !browser || content-shell
-```
-
-### `exclude_tags`
-
-This field causes the runner not to run tests whose tags match the given
-[boolean selector][]. If both [`include_tags`](#include_tags) and `exclude_tags`
-are used, the exclusions take precedence.
-
-This is usually used in a [preset](#configuration-preset) to make it possible to
-quickly select a set of tests.
-
-```yaml
-presets:
- # Pass "-P windowless" to run tests that don't open browser windows.
- windowless:
- exclude_tags: browser && !content-shell
-```
-
-### `platforms`
-
-This field indicates which platforms tests should run on by default. It allows
-the same platform identifiers that can be passed to `--platform`. If multiple
-platforms are included, the test runner will default to running tests on all of
-them. This defaults to `[vm]`.
-
-```yaml
-platforms: [content_shell]
-
-platforms:
-- chrome
-- firefox
-```
-
-### `concurrency`
-
-This field indicates the default number of test suites to run in parallel. More
-parallelism can improve the overall speed of running tests up to a point, but
-eventually it just adds more memory overhead without any performance gain. This
-defaults to approximately half the number of processors on the current machine.
-If it's set to 1, only one test suite will run at a time.
-
-```yaml
-concurrency: 3
-```
-
-### `pause_after_load`
-
-This field indicates that the test runner should pause for debugging after each
-test suite is loaded but before its tests are executed. If it's set,
-[`concurrency`](#concurrency) is automatically set to 1 and
-[`timeout`](#timeout) is automatically set to `none`.
-
-This is usually used in a [preset](#configuration-presets).
-
-```yaml
-presets:
- # Pass "-P debug" to enable debugging configuration
- debug:
- pause_after_load: true
- exclude_tags: undebuggable
- reporter: expanded
-```
-
-### `pub_serve`
-
-This field indicates that the test runner should run against a `pub serve`
-instance by default, and provides the port number for that instance. Note that
-if there is no `pub serve` instance running at that port, running the tests will
-fail by default.
-
-```yaml
-pub_serve: 8081
-```
-
-### `reporter`
-
-This field indicates the default reporter to use. It may be set to "compact",
-"expanded", or "json" (although why anyone would want to default to JSON is
-beyond me). It defaults to "expanded" on Windows and "compact" everywhere else.
-
-```yaml
-reporter: expanded
-```
-
-## Configuring Tags
-
-### `tags`
-
-The `tag` field can be used to apply [test configuration](#test-configuration)
-to all tests [with a given tag][tagging tests] or set of tags. It takes a map
-from tag selectors to configuration maps. These configuration maps are just like
-the top level of the configuration file, except that they may not contain
-[runner configuration](#runner-configuration).
-
-[tagging tests]: https://github.com/dart-lang/test/blob/master/README.md#tagging-tests
-
-```yaml
-tags:
- # Integration tests need more time to run.
- integration:
- timeout: 1m
-```
-
-Tags may also have no configuration associated with them. The test runner prints
-a warning whenever it encounters a tag it doesn't recognize, and this the best
-way to tell it that a tag exists.
-
-```yaml
-# We occasionally want to use --tags or --exclude-tags on these tags.
-tags:
- # A test that spawns a browser.
- browser:
-
- # A test that needs Ruby installed.
- ruby:
-```
-
-You can also use [boolean selector syntax][] to define configuration that
-involves multiple tags. For example:
-
-[boolean selector syntax]: https://github.com/dart-lang/boolean_selector/blob/master/README.md
-
-```yaml
-tags:
- # Tests that invoke sub-processes tend to be a little slower.
- ruby || python:
- timeout: 1.5x
-```
-
-Tag configuration is applied at whatever level the tag appears—so if a group is
-tagged as `integration`, its timeout will take precedence over the suite's
-timeout but not any tests'. If the group itself had a timeout declared, the
-group's explicit timeout would take precedence over the tag.
-
-If multiple tags appear at the same level, and they have conflicting
-configurations, the test runner *does not guarantee* what order they'll be
-resolved in. In practice, conflicting configuration is pretty unlikely and it's
-easy to just explicitly specify what you want on the test itself.
-
-This field counts as [test configuration](#test-configuration).
-
-### `add_tags`
-
-This field adds additional tags. It's technically
-[test configuration](#test-configuration), but it's usually used in more
-specific contexts. For example, when included in a tag's configuration, it can
-be used to enable tag inheritance, where adding one tag implicitly adds another
-as well. It takes a list of tag name strings.
-
-```yaml
-tags:
- # Any test that spawns a browser.
- browser:
- timeout: 2x
-
- # Tests that spawn specific browsers. These automatically get the browser tag
- # as well.
- chrome: {add_tags: [browser]}
- firefox: {add_tags: [browser]}
- safari: {add_tags: [browser]}
- ie: {add_tags: [browser]}
-```
-
-## Configuring Platforms
-
-There are two different kinds of platform configuration.
-[Operating system configuration](#on_os) cares about the operating system on
-which test runner is running. It sets global configuration for the runner on
-particular OSes. [Test platform configuration](#on_platform), on the other hand,
-cares about the platform the *test* is running on (like the
-[`@OnPlatform` annotation][@OnPlatform]). It sets configuration for particular
-tests that are running on matching platforms.
-
-[@OnPlatform]: https://github.com/dart-lang/test/blob/master/README.md#platform-specific-configuration
-
-### `on_os`
-
-This field applies configuration when specific operating systems are being used.
-It takes a map from operating system identifiers (the same ones that are used in
-[platform selectors][]) to configuration maps that are applied on those
-operating systems. These configuration maps are just like the top level of the
-configuration file, and allow any fields that may be used in the context where
-`on_os` was used.
-
-```yaml
-on_os:
- windows:
- # Both of these are the defaults anyway, but let's be explicit about it.
- color: false
- runner: expanded
-
- # My Windows machine is real slow.
- timeout: 2x
-
- # My Linux machine has SO MUCH RAM.
- linux:
- concurrency: 500
-```
-
-This field counts as [test configuration](#test-configuration). If it's used in
-a context that only allows test configuration, it may only contain test
-configuration.
-
-### `on_platform`
-
-This field applies configuration to tests that are run on specific platforms. It
-takes a map from [platform selectors][] to configuration maps that are applied
-to tests run on those platforms. These configuration maps are just like the top
-level of the configuration file, except that they may not contain
-[runner configuration](#runner-configuration).
-
-```yaml
-# Our code is kind of slow on Blink and WebKit.
-on_platform:
- chrome || safari: {timeout: 2x}
-```
-
-**Note**: operating system names that appear in `on_platform` refer to tests
-that are run on the Dart VM under that operating system. To configure all tests
-when running on a particular operating system, use [`on_os`](#on_os) instead.
-
-This field counts as [test configuration](#test-configuration).
-
-## Configuration Presets
-
-*Presets* are collections of configuration that can be explicitly selected on
-the command-line. They're useful for quickly selecting options that are
-frequently used together, for providing special configuration for continuous
-integration systems, and for defining more complex logic than can be expressed
-directly using command-line arguments.
-
-Presets can be selected on the command line using the `--preset` or `-P` flag.
-Any number of presets can be selected this way; if they conflict, the last one
-selected wins. Only presets that are defined in the configuration file may be
-selected.
-
-### `presets`
-
-This field defines which presets are available to select. It takes a map from
-preset names to configuration maps that are applied when those presets are
-selected. These configuration maps are just like the top level of the
-configuration file, and allow any fields that may be used in the context where
-`presets` was used.
-
-```yaml
-presets:
- # Use this when you need completely un-munged stack traces.
- debug:
- verbose_trace: false
- js_trace: true
-
- # Shortcut for running only browser tests.
- browser:
- paths:
- - test/runner/browser
- - test/runner/pub_serve_test.dart
-```
-
-The `presets` field counts as [test configuration](#test-configuration). It can
-be useful to use it in combination with other fields for advanced preset
-behavior.
-
-```yaml
-tags:
- chrome:
- skip: "Our Chrome launcher is busted. See issue 1234."
-
- # Pass -P force to verify that the launcher is still busted.
- presets: {force: {skip: false}}
-```
-
-### `add_presets`
-
-This field selects additional presets. It's technically
-[runner configuration](#runner-configuration), but it's usually used in more
-specific contexts. For example, when included in a preset's configuration, it
-can be used to enable preset inheritance, where selecting one preset implicitly
-selects another as well. It takes a list of preset name strings.
-
-```yaml
-presets:
- # Shortcut for running only browser tests.
- browser:
- paths: [test/runner/browser]
-
- # Shortcut for running only Chrome tests.
- chrome:
- filename: "chrome_*_test.dart"
- add_presets: [browser]
-```
« no previous file with comments | « doc/configuration.md ('k') | lib/src/executable.dart » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698