Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(47)

Side by Side Diff: mojo/public/dart/third_party/test/README.md

Issue 1346773002: Stop running pub get at gclient sync time and fix build bugs (Closed) Base URL: git@github.com:domokit/mojo.git@master
Patch Set: Created 5 years, 3 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
(Empty)
1 `test` provides a standard way of writing and running tests in Dart.
2
3 ## Writing Tests
4
5 Tests are specified using the top-level [`test()`][test] function, and test
6 assertions are made using [`expect()`][expect]:
7
8 [test]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@i d_test
9 [expect]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test @id_expect
10
11 ```dart
12 import "package:test/test.dart";
13
14 void main() {
15 test("String.split() splits the string on the delimiter", () {
16 var string = "foo,bar,baz";
17 expect(string.split(","), equals(["foo", "bar", "baz"]));
18 });
19
20 test("String.trim() removes surrounding whitespace", () {
21 var string = " foo ";
22 expect(string.trim(), equals("foo"));
23 });
24 }
25 ```
26
27 Tests can be grouped together using the [`group()`] function. Each group's
28 description is added to the beginning of its test's descriptions.
29
30 ```dart
31 import "package:test/test.dart";
32
33 void main() {
34 group("String", () {
35 test(".split() splits the string on the delimiter", () {
36 var string = "foo,bar,baz";
37 expect(string.split(","), equals(["foo", "bar", "baz"]));
38 });
39
40 test(".trim() removes surrounding whitespace", () {
41 var string = " foo ";
42 expect(string.trim(), equals("foo"));
43 });
44 });
45
46 group("int", () {
47 test(".remainder() returns the remainder of division", () {
48 expect(11.remainder(3), equals(2));
49 });
50
51 test(".toRadixString() returns a hex string", () {
52 expect(11.toRadixString(16), equals("b"));
53 });
54 });
55 }
56 ```
57
58 Any matchers from the [`matcher`][matcher] package can be used with `expect()`
59 to do complex validations:
60
61 [matcher]: http://www.dartdocs.org/documentation/matcher/latest/index.html#match er/matcher
62
63 ```dart
64 import "package:test/test.dart";
65
66 void main() {
67 test(".split() splits the string on the delimiter", () {
68 expect("foo,bar,baz", allOf([
69 contains("foo"),
70 isNot(startsWith("bar")),
71 endsWith("baz")
72 ]));
73 });
74 }
75 ```
76
77 ## Running Tests
78
79 A single test file can be run just using `pub run test:test path/to/test.dart`
80 (on Dart 1.10, this can be shortened to `pub run test path/to/test.dart`).
81
82 ![Single file being run via pub run"](https://raw.githubusercontent.com/dart-lan g/test/master/image/test1.gif)
83
84 Many tests can be run at a time using `pub run test:test path/to/dir`.
85
86 ![Directory being run via "pub run".](https://raw.githubusercontent.com/dart-lan g/test/master/image/test2.gif)
87
88 It's also possible to run a test on the Dart VM only by invoking it using `dart
89 path/to/test.dart`, but this doesn't load the full test runner and will be
90 missing some features.
91
92 The test runner considers any file that ends with `_test.dart` to be a test
93 file. If you don't pass any paths, it will run all the test files in your
94 `test/` directory, making it easy to test your entire application at once.
95
96 By default, tests are run in the Dart VM, but you can run them in the browser as
97 well by passing `pub run test:test -p chrome path/to/test.dart`.
98 `test` will take care of starting the browser and loading the tests, and all
99 the results will be reported on the command line just like for VM tests. In
100 fact, you can even run tests on both platforms with a single command: `pub run
101 test:test -p "chrome,vm" path/to/test.dart`.
102
103 ### Restricting Tests to Certain Platforms
104
105 Some test files only make sense to run on particular platforms. They may use
106 `dart:html` or `dart:io`, they might test Windows' particular filesystem
107 behavior, or they might use a feature that's only available in Chrome. The
108 [`@TestOn`][TestOn] annotation makes it easy to declare exactly which platforms
109 a test file should run on. Just put it at the top of your file, before any
110 `library` or `import` declarations:
111
112 ```dart
113 @TestOn("vm")
114
115 import "dart:io";
116
117 import "package:test/test.dart";
118
119 void main() {
120 // ...
121 }
122 ```
123
124 [TestOn]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test .TestOn
125
126 The string you pass to `@TestOn` is what's called a "platform selector", and it
127 specifies exactly which platforms a test can run on. It can be as simple as the
128 name of a platform, or a more complex Dart-like boolean expression involving
129 these platform names.
130
131 ### Platform Selector Syntax
132
133 Platform selectors can contain identifiers, parentheses, and operators. When
134 loading a test, each identifier is set to `true` or `false` based on the current
135 platform, and the test is only loaded if the platform selector returns `true`.
136 The operators `||`, `&&`, `!`, and `? :` all work just like they do in Dart. The
137 valid identifiers are:
138
139 * `vm`: Whether the test is running on the command-line Dart VM.
140
141 * `dartium`: Whether the test is running on Dartium.
142
143 * `content-shell`: Whether the test is running on the headless Dartium content
144 shell.
145
146 * `chrome`: Whether the test is running on Google Chrome.
147
148 * `phantomjs`: Whether the test is running on
149 [PhantomJS](http://phantomjs.org/).
150
151 * `firefox`: Whether the test is running on Mozilla Firefox.
152
153 * `safari`: Whether the test is running on Apple Safari.
154
155 * `ie`: Whether the test is running on Microsoft Internet Explorer.
156
157 * `dart-vm`: Whether the test is running on the Dart VM in any context,
158 including Dartium. It's identical to `!js`.
159
160 * `browser`: Whether the test is running in any browser.
161
162 * `js`: Whether the test has been compiled to JS. This is identical to
163 `!dart-vm`.
164
165 * `blink`: Whether the test is running in a browser that uses the Blink
166 rendering engine.
167
168 * `windows`: Whether the test is running on Windows. If `vm` is false, this will
169 be `false` as well.
170
171 * `mac-os`: Whether the test is running on Mac OS. If `vm` is false, this will
172 be `false` as well.
173
174 * `linux`: Whether the test is running on Linux. If `vm` is false, this will be
175 `false` as well.
176
177 * `android`: Whether the test is running on Android. If `vm` is false, this will
178 be `false` as well, which means that this *won't* be true if the test is
179 running on an Android browser.
180
181 * `posix`: Whether the test is running on a POSIX operating system. This is
182 equivalent to `!windows`.
183
184 For example, if you wanted to run a test on every browser but Chrome, you would
185 write `@TestOn("browser && !chrome")`.
186
187 ### Running Tests on Dartium
188
189 Tests can be run on [Dartium][] by passing the `-p dartium` flag. If you're
190 using the Dart Editor, the test runner will be able to find Dartium
191 automatically. On Mac OS, you can also [install it using Homebrew][homebrew].
192 Otherwise, make sure there's an executable called `dartium` (on Mac OS or Linux)
193 or `dartium.exe` (on Windows) on your system path.
194
195 [Dartium]: https://www.dartlang.org/tools/dartium/
196 [homebrew]: https://github.com/dart-lang/homebrew-dart
197
198 Similarly, tests can be run on the headless Dartium content shell by passing `-p
199 content-shell`. The content shell is installed along with Dartium when using
200 Homebrew. Otherwise, you can downloaded it manually [from this
201 page][content_shell]; if you do, make sure the executable named `content_shell`
202 (on Mac OS or Linux) or `content_shell.exe` (on Windows) is on your system path.
203
204 [content_shell]: http://gsdview.appspot.com/dart-archive/channels/stable/release /latest/dartium/
205
206 [In the future][issue 63], there will be a more explicit way to configure the
207 location of both the Dartium and content shell executables.
208
209 [issue 63]: https://github.com/dart-lang/test/issues/63
210
211 ## Asynchronous Tests
212
213 Tests written with `async`/`await` will work automatically. The test runner
214 won't consider the test finished until the returned `Future` completes.
215
216 ```dart
217 import "dart:async";
218
219 import "package:test/test.dart";
220
221 void main() {
222 test("new Future.value() returns the value", () async {
223 var value = await new Future.value(10);
224 expect(value, equals(10));
225 });
226 }
227 ```
228
229 There are also a number of useful functions and matchers for more advanced
230 asynchrony. The [`completion()`][completion] matcher can be used to test
231 `Futures`; it ensures that the test doesn't finish until the `Future` completes,
232 and runs a matcher against that `Future`'s value.
233
234 [completion]: http://www.dartdocs.org/documentation/test/latest/index.html#test/ test@id_completion
235
236 ```dart
237 import "dart:async";
238
239 import "package:test/test.dart";
240
241 void main() {
242 test("new Future.value() returns the value", () {
243 expect(new Future.value(10), completion(equals(10)));
244 });
245 }
246 ```
247
248 The [`throwsA()`][throwsA] matcher and the various `throwsExceptionType`
249 matchers work with both synchronous callbacks and asynchronous `Future`s. They
250 ensure that a particular type of exception is thrown:
251
252 [throwsA]: http://www.dartdocs.org/documentation/test/latest/index.html#test/tes t@id_throwsA
253
254 ```dart
255 import "dart:async";
256
257 import "package:test/test.dart";
258
259 void main() {
260 test("new Future.error() throws the error", () {
261 expect(new Future.error("oh no"), throwsA(equals("oh no")));
262 expect(new Future.error(new StateError("bad state")), throwsStateError);
263 });
264 }
265 ```
266
267 The [`expectAsync()`][expectAsync] function wraps another function and has two
268 jobs. First, it asserts that the wrapped function is called a certain number of
269 times, and will cause the test to fail if it's called too often; second, it
270 keeps the test from finishing until the function is called the requisite number
271 of times.
272
273 ```dart
274 import "dart:async";
275
276 import "package:test/test.dart";
277
278 void main() {
279 test("Stream.fromIterable() emits the values in the iterable", () {
280 var stream = new Stream.fromIterable([1, 2, 3]);
281
282 stream.listen(expectAsync((number) {
283 expect(number, inInclusiveRange(1, 3));
284 }, count: 3));
285 });
286 }
287 ```
288
289 [expectAsync]: http://www.dartdocs.org/documentation/test/latest/index.html#test /test@id_expectAsync
290
291 ## Running Tests with Custom HTML
292
293 By default, the test runner will generate its own empty HTML file for browser
294 tests. However, tests that need custom HTML can create their own files. These
295 files have three requirements:
296
297 * They must have the same name as the test, with `.dart` replaced by `.html`.
298
299 * They must contain a `link` tag with `rel="x-dart-test"` and an `href`
300 attribute pointing to the test script.
301
302 * They must contain `<script src="packages/test/dart.js"></script>`.
303
304 For example, if you had a test called `custom_html_test.dart`, you might write
305 the following HTML file:
306
307 ```html
308 <!doctype html>
309 <!-- custom_html_test.html -->
310 <html>
311 <head>
312 <title>Custom HTML Test</title>
313 <link rel="x-dart-test" href="custom_html_test.dart">
314 <script src="packages/test/dart.js"></script>
315 </head>
316 <body>
317 // ...
318 </body>
319 </html>
320 ```
321
322 ## Configuring Tests
323
324 ### Skipping Tests
325
326 If a test, group, or entire suite isn't working yet and you just want it to stop
327 complaining, you can mark it as "skipped". The test or tests won't be run, and,
328 if you supply a reason why, that reason will be printed. In general, skipping
329 tests indicates that they should run but is temporarily not working. If they're
330 is fundamentally incompatible with a platform, [`@TestOn`/`testOn`][TestOn]
331 should be used instead.
332
333 [TestOn]: #restricting-tests-to-certain-platforms
334
335 To skip a test suite, put a `@Skip` annotation at the top of the file:
336
337 ```dart
338 @Skip("currently failing (see issue 1234)")
339
340 import "package:test/test.dart";
341
342 void main() {
343 // ...
344 }
345 ```
346
347 The string you pass should describe why the test is skipped. You don't have to
348 include it, but it's a good idea to document why the test isn't running.
349
350 Groups and individual tests can be skipped by passing the `skip` parameter. This
351 can be either `true` or a String describing why the test is skipped. For example :
352
353 ```dart
354 import "package:test/test.dart";
355
356 void main() {
357 group("complicated algorithm tests", () {
358 // ...
359 }, skip: "the algorithm isn't quite right");
360
361 test("error-checking test", () {
362 // ...
363 }, skip: "TODO: add error-checking.");
364 }
365 ```
366
367 ### Timeouts
368
369 By default, tests will time out after 30 seconds of inactivity. However, this
370 can be configured on a per-test, -group, or -suite basis. To change the timeout
371 for a test suite, put a `@Timeout` annotation at the top of the file:
372
373 ```dart
374 @Timeout(const Duration(seconds: 45))
375
376 import "package:test/test.dart";
377
378 void main() {
379 // ...
380 }
381 ```
382
383 In addition to setting an absolute timeout, you can set the timeout relative to
384 the default using `@Timeout.factor`. For example, `@Timeout.factor(1.5)` will
385 set the timeout to one and a half times as long as the default—45 seconds.
386
387 Timeouts can be set for tests and groups using the `timeout` parameter. This
388 parameter takes a `Timeout` object just like the annotation. For example:
389
390 ```dart
391 import "package:test/test.dart";
392
393 void main() {
394 group("slow tests", () {
395 // ...
396
397 test("even slower test", () {
398 // ...
399 }, timeout: new Timeout.factor(2))
400 }, timeout: new Timeout(new Duration(minutes: 1)));
401 }
402 ```
403
404 Nested timeouts apply in order from outermost to innermost. That means that
405 "even slower test" will take two minutes to time out, since it multiplies the
406 group's timeout by 2.
407
408 ### Platform-Specific Configuration
409
410 Sometimes a test may need to be configured differently for different platforms.
411 Windows might run your code slower than other platforms, or your DOM
412 manipulation might not work right on Safari yet. For these cases, you can use
413 the `@OnPlatform` annotation and the `onPlatform` named parameter to `test()`
414 and `group()`. For example:
415
416 ```dart
417 @OnPlatform(const {
418 // Give Windows some extra wiggle-room before timing out.
419 "windows": const Timeout.factor(2)
420 })
421
422 import "package:test/test.dart";
423
424 void main() {
425 test("do a thing", () {
426 // ...
427 }, onPlatform: {
428 "safari": new Skip("Safari is currently broken (see #1234)")
429 });
430 }
431 ```
432
433 Both the annotation and the parameter take a map. The map's keys are [platform
434 selectors](#platform-selector-syntax) which describe the platforms for which the
435 specialized configuration applies. Its values are instances of some of the same
436 annotation classes that can be used for a suite: `Skip` and `Timeout`. A value
437 can also be a list of these values.
438
439 If multiple platforms match, the configuration is applied in order from first to
440 last, just as they would in nested groups. This means that for configuration
441 like duration-based timeouts, the last matching value wins.
442
443 ## Testing With `barback`
444
445 Packages using the `barback` transformer system may need to test code that's
446 created or modified using transformers. The test runner handles this using the
447 `--pub-serve` option, which tells it to load the test code from a `pub serve`
448 instance rather than from the filesystem.
449
450 Before using the `--pub-serve` option, add the `test/pub_serve` transformer to
451 your `pubspec.yaml`. This transformer adds the necessary bootstrapping code that
452 allows the test runner to load your tests properly:
453
454 ```yaml
455 transformers:
456 - test/pub_serve:
457 $include: test/**_test{.*,}.dart
458 ```
459
460 Note that if you're using the test runner along with [`polymer`][polymer], you
461 have to make sure that the `test/pub_serve` transformer comes *after* the
462 `polymer` transformer:
463
464 [polymer]: https://www.dartlang.org/polymer/
465
466 ```yaml
467 transformers:
468 - polymer
469 - test/pub_serve:
470 $include: test/**_test{.*,}.dart
471 ```
472
473 Then, start up `pub serve`. Make sure to pay attention to which port it's using
474 to serve your `test/` directory:
475
476 ```shell
477 $ pub serve
478 Loading source assets...
479 Loading test/pub_serve transformers...
480 Serving my_app web on http://localhost:8080
481 Serving my_app test on http://localhost:8081
482 Build completed successfully
483 ```
484
485 In this case, the port is `8081`. In another terminal, pass this port to
486 `--pub-serve` and otherwise invoke `pub run test:test` as normal:
487
488 ```shell
489 $ pub run test:test --pub-serve=8081 -p chrome
490 "pub serve" is compiling test/my_app_test.dart...
491 "pub serve" is compiling test/utils_test.dart...
492 00:00 +42: All tests passed!
493 ```
OLDNEW
« no previous file with comments | « mojo/public/dart/third_party/test/LICENSE ('k') | mojo/public/dart/third_party/test/bin/test.dart » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698