OLD | NEW |
---|---|
(Empty) | |
1 # Test filter files | |
2 | |
3 ## Summary | |
4 | |
5 This directory contains files that list tests that are not yet ready to run in a | |
6 particular mode. For example - the `site-per-process.browser_tests.filter` file | |
7 lists tests that should be excluded when running `browser_tests` in | |
8 `--site-per-process` mode. | |
9 | |
10 ## File syntax | |
11 | |
12 Contents of test filter files follow the syntax below: | |
13 | |
14 - Empty lines are ignored | |
15 - Lines starting with the '#' character are treated as comments and ignored. | |
16 - All other lines specify a single [test name pattern][gtest_filter]. | |
17 - Patterns prefixed with the '-' character specify tests to exclude from a | |
18 test run. | |
19 - All other patterns specify tests to include in a test run. | |
alexmos
2016/11/03 21:30:58
nit: while we're at it, should we also warn about
Łukasz Anforowicz
2016/11/03 22:00:15
Done. Sort of (no warning, just copying the short
| |
20 | |
21 Example test filter file: | |
22 | |
23 ```test.filter | |
24 # crbug.com/417518: Get tests working w/ --site-per-process | |
25 -BrowserTest.OtherRedirectsDontForkProcess | |
26 -ChromeRenderProcessHostTest.* | |
27 -ReferrerPolicyTest.HttpsRedirect | |
28 | |
29 # crbug.com/448592: Get extension browsertests working w/ --site-per-process | |
30 -IsolatedAppTest.CookieIsolation | |
31 -IsolatedAppTest.CrossProcessClientRedirect | |
32 -IsolatedAppTest.IsolatedAppProcessModel | |
33 -IsolatedAppTest.SubresourceCookieIsolation | |
34 ``` | |
35 | |
36 ## Usage | |
37 | |
38 When running tests on desktop platforms, the test filter file can be specified | |
39 using `--test-launcher-filter-file` command line flag. Example test invocation: | |
40 | |
41 ```bash | |
42 $ out/dbg/content_browsertests \ | |
43 --site-per-process \ | |
44 --test-launcher-filter-file=testing/buildbot/filters/site-per-process.conten t_browsertests.filter | |
45 ``` | |
46 | |
47 When running tests on Android, the test filter file can be specified using | |
48 `--gtest-filter-file` command line flag. Example test invocation: | |
49 | |
50 ```bash | |
51 $ out/android/bin/run_content_browsertests \ | |
52 --test-arguments=--site-per-process \ | |
53 --gtest-filter-file=testing/buildbot/filters/site-per-process.content_browse rtests.filter | |
54 ``` | |
55 | |
56 ## Applicability | |
57 | |
58 Test filter files described here are currently only supported for gtest-based | |
59 tests. | |
60 | |
61 For excluding layout tests for specific command line flags of `content_shell`, | |
alexmos
2016/11/03 21:30:58
nit: maybe simplify to "For excluding layout tests
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
62 see `third_party/WebKit/LayoutTests/FlagExpectations/README.txt`. | |
63 | |
64 ## Adding new test filter files | |
65 | |
66 ### File names | |
67 | |
68 Please use the following suggestions when naming the new file: | |
alexmos
2016/11/03 21:30:58
how about s/suggestions/conventions/ :)
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
69 | |
70 - Please include the name of the test executable (e.g. | |
71 `content_browsertests`). | |
72 - Please use `.filter` suffix / extension. | |
alexmos
2016/11/03 21:30:58
nit: "suffix / extension" - just pick one?
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
73 - Feel free to add other relevant things into the file name (e.g. the mode the | |
74 file applies to - for example `site-per-process`). | |
75 | |
76 ### Build dependencies | |
77 | |
78 When adding a new file, please remember to update `data` dependencies of the | |
79 relevant test target. This will help ensure that the test filter file ends up in | |
80 the right place on the bot running the tests. | |
81 | |
82 For example - when adding a new test filter file for `content_browsertests` | |
83 please update `content/test/BUILD.gn` to cover the new test file as a `data` | |
84 dependency of `content_browsertests` target. | |
85 | |
86 Example - `content/test/BUILD.gn`: | |
87 | |
88 ```BUILD.gn | |
89 test("content_browsertests") { | |
90 ... | |
91 data = [] | |
92 ... | |
93 # Adding all //testing/buildbot/filters/*.content_browsertests.*filter files | |
94 # as a dependency. | |
95 # | |
96 # They might not be used on all platforms (e.g. there is no mac bot coverage | |
97 # for --site-per-process today), but for simplicity we just include all of | |
98 # them below (doing this should help avoid unpleasant surprises when a bot | |
99 # config is switched in testing/buildbot/chromium.fyi.json without | |
100 # corresponding dependency changes here - see https://crbug.com/661447). | |
101 # | |
102 # To refresh, run the following command in bash: | |
103 # $ for i in $( \ | |
104 # find testing/buildbot/filters -name '*.content_browsertests.*filter'); \ | |
105 # do echo " \"//$i\","; done | sort | |
106 data += [ | |
107 "//testing/buildbot/filters/browser-side-navigation.linux.content_browsertes ts.filter", | |
108 "//testing/buildbot/filters/cast-linux.content_browsertests.filter", | |
109 "//testing/buildbot/filters/isolate-extensions.content_browsertests.filter", | |
110 "//testing/buildbot/filters/site-per-process.content_browsertests.filter", | |
111 ] | |
112 ``` | |
113 | |
114 [gtest_filter]: https://github.com/google/googletest/blob/master/googletest/docs /AdvancedGuide.md#running-a-subset-of-the-tests | |
OLD | NEW |