Chromium Code Reviews| OLD | NEW |
|---|---|
| (Empty) | |
| 1 # Test filter files | |
| 2 | |
| 3 ## Summary | |
| 4 | |
| 5 This directory contains files that list tests that are not yet ready to run in a | |
| 6 particular mode. For example - the `site-per-process.browser_tests.filter` file | |
| 7 lists tests that should be excluded when running `browser_tests` in | |
| 8 `--site-per-process` mode. | |
| 9 | |
| 10 ## File syntax | |
| 11 | |
| 12 Contents of test filter files follow the syntax below: | |
| 13 | |
| 14 - Empty lines are ignored | |
| 15 - Lines starting with the '#' character are treated as comments and ignored. | |
| 16 - All other lines specify a single [test name pattern][gtest_filter]. | |
| 17 - Patterns prefixed with the '-' character specify tests to exclude from a | |
| 18 test run. | |
| 19 - All other patterns specify tests to include in a test run. | |
|
alexmos
2016/11/03 21:30:58
nit: while we're at it, should we also warn about
Łukasz Anforowicz
2016/11/03 22:00:15
Done. Sort of (no warning, just copying the short
| |
| 20 | |
| 21 Example test filter file: | |
| 22 | |
| 23 ```test.filter | |
| 24 # crbug.com/417518: Get tests working w/ --site-per-process | |
| 25 -BrowserTest.OtherRedirectsDontForkProcess | |
| 26 -ChromeRenderProcessHostTest.* | |
| 27 -ReferrerPolicyTest.HttpsRedirect | |
| 28 | |
| 29 # crbug.com/448592: Get extension browsertests working w/ --site-per-process | |
| 30 -IsolatedAppTest.CookieIsolation | |
| 31 -IsolatedAppTest.CrossProcessClientRedirect | |
| 32 -IsolatedAppTest.IsolatedAppProcessModel | |
| 33 -IsolatedAppTest.SubresourceCookieIsolation | |
| 34 ``` | |
| 35 | |
| 36 ## Usage | |
| 37 | |
| 38 When running tests on desktop platforms, the test filter file can be specified | |
| 39 using `--test-launcher-filter-file` command line flag. Example test invocation: | |
| 40 | |
| 41 ```bash | |
| 42 $ out/dbg/content_browsertests \ | |
| 43 --site-per-process \ | |
| 44 --test-launcher-filter-file=testing/buildbot/filters/site-per-process.conten t_browsertests.filter | |
| 45 ``` | |
| 46 | |
| 47 When running tests on Android, the test filter file can be specified using | |
| 48 `--gtest-filter-file` command line flag. Example test invocation: | |
| 49 | |
| 50 ```bash | |
| 51 $ out/android/bin/run_content_browsertests \ | |
| 52 --test-arguments=--site-per-process \ | |
| 53 --gtest-filter-file=testing/buildbot/filters/site-per-process.content_browse rtests.filter | |
| 54 ``` | |
| 55 | |
| 56 ## Applicability | |
| 57 | |
| 58 Test filter files described here are currently only supported for gtest-based | |
| 59 tests. | |
| 60 | |
| 61 For excluding layout tests for specific command line flags of `content_shell`, | |
|
alexmos
2016/11/03 21:30:58
nit: maybe simplify to "For excluding layout tests
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
| 62 see `third_party/WebKit/LayoutTests/FlagExpectations/README.txt`. | |
| 63 | |
| 64 ## Adding new test filter files | |
| 65 | |
| 66 ### File names | |
| 67 | |
| 68 Please use the following suggestions when naming the new file: | |
|
alexmos
2016/11/03 21:30:58
how about s/suggestions/conventions/ :)
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
| 69 | |
| 70 - Please include the name of the test executable (e.g. | |
| 71 `content_browsertests`). | |
| 72 - Please use `.filter` suffix / extension. | |
|
alexmos
2016/11/03 21:30:58
nit: "suffix / extension" - just pick one?
Łukasz Anforowicz
2016/11/03 22:00:15
Done.
| |
| 73 - Feel free to add other relevant things into the file name (e.g. the mode the | |
| 74 file applies to - for example `site-per-process`). | |
| 75 | |
| 76 ### Build dependencies | |
| 77 | |
| 78 When adding a new file, please remember to update `data` dependencies of the | |
| 79 relevant test target. This will help ensure that the test filter file ends up in | |
| 80 the right place on the bot running the tests. | |
| 81 | |
| 82 For example - when adding a new test filter file for `content_browsertests` | |
| 83 please update `content/test/BUILD.gn` to cover the new test file as a `data` | |
| 84 dependency of `content_browsertests` target. | |
| 85 | |
| 86 Example - `content/test/BUILD.gn`: | |
| 87 | |
| 88 ```BUILD.gn | |
| 89 test("content_browsertests") { | |
| 90 ... | |
| 91 data = [] | |
| 92 ... | |
| 93 # Adding all //testing/buildbot/filters/*.content_browsertests.*filter files | |
| 94 # as a dependency. | |
| 95 # | |
| 96 # They might not be used on all platforms (e.g. there is no mac bot coverage | |
| 97 # for --site-per-process today), but for simplicity we just include all of | |
| 98 # them below (doing this should help avoid unpleasant surprises when a bot | |
| 99 # config is switched in testing/buildbot/chromium.fyi.json without | |
| 100 # corresponding dependency changes here - see https://crbug.com/661447). | |
| 101 # | |
| 102 # To refresh, run the following command in bash: | |
| 103 # $ for i in $( \ | |
| 104 # find testing/buildbot/filters -name '*.content_browsertests.*filter'); \ | |
| 105 # do echo " \"//$i\","; done | sort | |
| 106 data += [ | |
| 107 "//testing/buildbot/filters/browser-side-navigation.linux.content_browsertes ts.filter", | |
| 108 "//testing/buildbot/filters/cast-linux.content_browsertests.filter", | |
| 109 "//testing/buildbot/filters/isolate-extensions.content_browsertests.filter", | |
| 110 "//testing/buildbot/filters/site-per-process.content_browsertests.filter", | |
| 111 ] | |
| 112 ``` | |
| 113 | |
| 114 [gtest_filter]: https://github.com/google/googletest/blob/master/googletest/docs /AdvancedGuide.md#running-a-subset-of-the-tests | |
| OLD | NEW |