Chromium Code Reviews

Side by Side Diff: testing/buildbot/filters/README.md

Issue 2472153002: Gathering information about test filter files into a README.md file. (Closed)
Patch Set: Self-review. Created 4 years, 1 month ago
Use n/p to move between diff chunks; N/P to move between comments.
Jump to:
View unified diff |
« no previous file with comments | « build/android/test_runner.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 # Test filter files
2
3 ## Summary
4
5 This directory contains files that list tests that are not yet ready to run in a
6 particular mode. For example - the `site-per-process.browser_tests.filter` file
7 lists tests that should be excluded when running `browser_tests` in
8 `--site-per-process` mode.
9
10 ## File syntax
11
12 Contents of test filter files follow the syntax below:
13
14 - Empty lines are ignored
15 - Lines starting with the '#' character are treated as comments and ignored.
16 - All other lines specify a single [test name pattern][gtest_filter].
17 - Patterns prefixed with the '-' character specify tests to exclude from a
18 test run.
19 - All other patterns specify tests to include in a test run.
alexmos 2016/11/03 21:30:58 nit: while we're at it, should we also warn about
Łukasz Anforowicz 2016/11/03 22:00:15 Done. Sort of (no warning, just copying the short
20
21 Example test filter file:
22
23 ```test.filter
24 # crbug.com/417518: Get tests working w/ --site-per-process
25 -BrowserTest.OtherRedirectsDontForkProcess
26 -ChromeRenderProcessHostTest.*
27 -ReferrerPolicyTest.HttpsRedirect
28
29 # crbug.com/448592: Get extension browsertests working w/ --site-per-process
30 -IsolatedAppTest.CookieIsolation
31 -IsolatedAppTest.CrossProcessClientRedirect
32 -IsolatedAppTest.IsolatedAppProcessModel
33 -IsolatedAppTest.SubresourceCookieIsolation
34 ```
35
36 ## Usage
37
38 When running tests on desktop platforms, the test filter file can be specified
39 using `--test-launcher-filter-file` command line flag. Example test invocation:
40
41 ```bash
42 $ out/dbg/content_browsertests \
43 --site-per-process \
44 --test-launcher-filter-file=testing/buildbot/filters/site-per-process.conten t_browsertests.filter
45 ```
46
47 When running tests on Android, the test filter file can be specified using
48 `--gtest-filter-file` command line flag. Example test invocation:
49
50 ```bash
51 $ out/android/bin/run_content_browsertests \
52 --test-arguments=--site-per-process \
53 --gtest-filter-file=testing/buildbot/filters/site-per-process.content_browse rtests.filter
54 ```
55
56 ## Applicability
57
58 Test filter files described here are currently only supported for gtest-based
59 tests.
60
61 For excluding layout tests for specific command line flags of `content_shell`,
alexmos 2016/11/03 21:30:58 nit: maybe simplify to "For excluding layout tests
Łukasz Anforowicz 2016/11/03 22:00:15 Done.
62 see `third_party/WebKit/LayoutTests/FlagExpectations/README.txt`.
63
64 ## Adding new test filter files
65
66 ### File names
67
68 Please use the following suggestions when naming the new file:
alexmos 2016/11/03 21:30:58 how about s/suggestions/conventions/ :)
Łukasz Anforowicz 2016/11/03 22:00:15 Done.
69
70 - Please include the name of the test executable (e.g.
71 `content_browsertests`).
72 - Please use `.filter` suffix / extension.
alexmos 2016/11/03 21:30:58 nit: "suffix / extension" - just pick one?
Łukasz Anforowicz 2016/11/03 22:00:15 Done.
73 - Feel free to add other relevant things into the file name (e.g. the mode the
74 file applies to - for example `site-per-process`).
75
76 ### Build dependencies
77
78 When adding a new file, please remember to update `data` dependencies of the
79 relevant test target. This will help ensure that the test filter file ends up in
80 the right place on the bot running the tests.
81
82 For example - when adding a new test filter file for `content_browsertests`
83 please update `content/test/BUILD.gn` to cover the new test file as a `data`
84 dependency of `content_browsertests` target.
85
86 Example - `content/test/BUILD.gn`:
87
88 ```BUILD.gn
89 test("content_browsertests") {
90 ...
91 data = []
92 ...
93 # Adding all //testing/buildbot/filters/*.content_browsertests.*filter files
94 # as a dependency.
95 #
96 # They might not be used on all platforms (e.g. there is no mac bot coverage
97 # for --site-per-process today), but for simplicity we just include all of
98 # them below (doing this should help avoid unpleasant surprises when a bot
99 # config is switched in testing/buildbot/chromium.fyi.json without
100 # corresponding dependency changes here - see https://crbug.com/661447).
101 #
102 # To refresh, run the following command in bash:
103 # $ for i in $( \
104 # find testing/buildbot/filters -name '*.content_browsertests.*filter'); \
105 # do echo " \"//$i\","; done | sort
106 data += [
107 "//testing/buildbot/filters/browser-side-navigation.linux.content_browsertes ts.filter",
108 "//testing/buildbot/filters/cast-linux.content_browsertests.filter",
109 "//testing/buildbot/filters/isolate-extensions.content_browsertests.filter",
110 "//testing/buildbot/filters/site-per-process.content_browsertests.filter",
111 ]
112 ```
113
114 [gtest_filter]: https://github.com/google/googletest/blob/master/googletest/docs /AdvancedGuide.md#running-a-subset-of-the-tests
OLDNEW
« no previous file with comments | « build/android/test_runner.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine