Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(19)

Side by Side Diff: docs/testing/layout_tests.md

Issue 2695963006: Fix some links in documentation related to layout tests. (Closed)
Patch Set: Created 3 years, 10 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 # Layout Tests 1 # Layout Tests
2 2
3 Layout tests are used by Blink to test many components, including but not 3 Layout tests are used by Blink to test many components, including but not
4 limited to layout and rendering. In general, layout tests involve loading pages 4 limited to layout and rendering. In general, layout tests involve loading pages
5 in a test renderer (`content_shell`) and comparing the rendered output or 5 in a test renderer (`content_shell`) and comparing the rendered output or
6 JavaScript output against an expected output file. 6 JavaScript output against an expected output file.
7 7
8 This document covers running and debugging existing layout tests. See the 8 This document covers running and debugging existing layout tests. See the
9 [Writing Layout Tests documentation](./writing_layout_tests.md) if you find 9 [Writing Layout Tests documentation](./writing_layout_tests.md) if you find
10 yourself writing layout tests. 10 yourself writing layout tests.
(...skipping 166 matching lines...) Expand 10 before | Expand all | Expand 10 after
177 *** 177 ***
178 178
179 A test that runs but produces the wrong output is marked as "failed", one that 179 A test that runs but produces the wrong output is marked as "failed", one that
180 causes the test shell to crash is marked as "crashed", and one that takes longer 180 causes the test shell to crash is marked as "crashed", and one that takes longer
181 than a certain amount of time to complete is aborted and marked as "timed out". 181 than a certain amount of time to complete is aborted and marked as "timed out".
182 A row of dots in the script's output indicates one or more tests that passed. 182 A row of dots in the script's output indicates one or more tests that passed.
183 183
184 ## Test expectations 184 ## Test expectations
185 185
186 The 186 The
187 [TestExpectations](../../WebKit/LayoutTests/TestExpectations) file (and related 187 [TestExpectations](../../third_party/WebKit/LayoutTests/TestExpectations) file ( and related
188 files, including 188 files) contains the list of all known layout test failures. See the
189 [skia_test_expectations.txt](../../skia/skia_test_expectations.txt))
190 contains the list of all known layout test failures. See the
191 [Layout Test Expectations documentation](./layout_test_expectations.md) for more 189 [Layout Test Expectations documentation](./layout_test_expectations.md) for more
192 on this. 190 on this.
193 191
194 ## Testing Runtime Flags 192 ## Testing Runtime Flags
195 193
196 There are two ways to run layout tests with additional command-line arguments: 194 There are two ways to run layout tests with additional command-line arguments:
197 195
198 * Using `--additional-driver-flag`: 196 * Using `--additional-driver-flag`:
199 197
200 ```bash 198 ```bash
201 run-webkit-tests --additional-driver-flag=--blocking-repaint 199 run-webkit-tests --additional-driver-flag=--blocking-repaint
202 ``` 200 ```
203 201
204 This tells the test harness to pass `--blocking-repaint` to the 202 This tells the test harness to pass `--blocking-repaint` to the
205 content_shell binary. 203 content_shell binary.
206 204
207 It will also look for flag-specific expectations in 205 It will also look for flag-specific expectations in
208 `LayoutTests/FlagExpectations/blocking-repaint`, if this file exists. The 206 `LayoutTests/FlagExpectations/blocking-repaint`, if this file exists. The
209 suppressions in this file override the main TestExpectations file. 207 suppressions in this file override the main TestExpectations file.
210 208
211 * Using a *virtual test suite* defined in 209 * Using a *virtual test suite* defined in
212 [LayoutTests/VirtualTestSuites](https://code.google.com/p/chromium/codesearch# chromium/src/third_party/WebKit/LayoutTests/VirtualTestSuites). 210 [LayoutTests/VirtualTestSuites](../../third_party/WebKit/LayoutTests/VirtualTe stSuites).
213 A virtual test suite runs a subset of layout tests under a specific path with 211 A virtual test suite runs a subset of layout tests under a specific path with
214 additional flags. For example, you could test a (hypothetical) new mode for 212 additional flags. For example, you could test a (hypothetical) new mode for
215 repainting using the following virtual test suite: 213 repainting using the following virtual test suite:
216 214
217 ```json 215 ```json
218 { 216 {
219 "prefix": "blocking_repaint", 217 "prefix": "blocking_repaint",
220 "base": "fast/repaint", 218 "base": "fast/repaint",
221 "args": ["--blocking-repaint"], 219 "args": ["--blocking-repaint"],
222 } 220 }
(...skipping 94 matching lines...) Expand 10 before | Expand all | Expand 10 after
317 produce a page including links to the expected result, actual result, and 315 produce a page including links to the expected result, actual result, and
318 diff. 316 diff.
319 * Add the `--sources` option to `run_webkit_tests.py` to see exactly which 317 * Add the `--sources` option to `run_webkit_tests.py` to see exactly which
320 expected result it's comparing to (a file next to the test, something in 318 expected result it's comparing to (a file next to the test, something in
321 platform/mac/, something in platform/chromium-win/, etc.) 319 platform/mac/, something in platform/chromium-win/, etc.)
322 * If you're still sure it's correct, rebaseline the test (see below). 320 * If you're still sure it's correct, rebaseline the test (see below).
323 Otherwise... 321 Otherwise...
324 * If you're lucky, your test is one that runs properly when you navigate to it 322 * If you're lucky, your test is one that runs properly when you navigate to it
325 in content_shell normally. In that case, build the Debug content_shell 323 in content_shell normally. In that case, build the Debug content_shell
326 project, fire it up in your favorite debugger, and load the test file either 324 project, fire it up in your favorite debugger, and load the test file either
327 from a file:// URL. 325 from a `file:` URL.
328 * You'll probably be starting and stopping the content_shell a lot. In VS, 326 * You'll probably be starting and stopping the content_shell a lot. In VS,
329 to save navigating to the test every time, you can set the URL to your 327 to save navigating to the test every time, you can set the URL to your
330 test (file: or http:) as the command argument in the Debugging section of 328 test (`file:` or `http:`) as the command argument in the Debugging section of
331 the content_shell project Properties. 329 the content_shell project Properties.
332 * If your test contains a JS call, DOM manipulation, or other distinctive 330 * If your test contains a JS call, DOM manipulation, or other distinctive
333 piece of code that you think is failing, search for that in the Chrome 331 piece of code that you think is failing, search for that in the Chrome
334 solution. That's a good place to put a starting breakpoint to start 332 solution. That's a good place to put a starting breakpoint to start
335 tracking down the issue. 333 tracking down the issue.
336 * Otherwise, you're running in a standard message loop just like in Chrome. 334 * Otherwise, you're running in a standard message loop just like in Chrome.
337 If you have no other information, set a breakpoint on page load. 335 If you have no other information, set a breakpoint on page load.
338 * If your test only works in full layout-test mode, or if you find it simpler to 336 * If your test only works in full layout-test mode, or if you find it simpler to
339 debug without all the overhead of an interactive session, start the 337 debug without all the overhead of an interactive session, start the
340 content_shell with the command-line flag `--run-layout-test`, followed by the 338 content_shell with the command-line flag `--run-layout-test`, followed by the
341 URL (file: or http:) to your test. More information about running layout tests 339 URL (`file:` or `http:`) to your test. More information about running layout t ests
342 in content_shell can be found [here](./layout_tests_in_content_shell.md). 340 in content_shell can be found [here](./layout_tests_in_content_shell.md).
343 * In VS, you can do this in the Debugging section of the content_shell 341 * In VS, you can do this in the Debugging section of the content_shell
344 project Properties. 342 project Properties.
345 * Now you're running with exactly the same API, theme, and other setup that 343 * Now you're running with exactly the same API, theme, and other setup that
346 the layout tests use. 344 the layout tests use.
347 * Again, if your test contains a JS call, DOM manipulation, or other 345 * Again, if your test contains a JS call, DOM manipulation, or other
348 distinctive piece of code that you think is failing, search for that in 346 distinctive piece of code that you think is failing, search for that in
349 the Chrome solution. That's a good place to put a starting breakpoint to 347 the Chrome solution. That's a good place to put a starting breakpoint to
350 start tracking down the issue. 348 start tracking down the issue.
351 * If you can't find any better place to set a breakpoint, start at the 349 * If you can't find any better place to set a breakpoint, start at the
(...skipping 65 matching lines...) Expand 10 before | Expand all | Expand 10 after
417 `blink/tools/run_layout_tests.sh 415 `blink/tools/run_layout_tests.sh
418 --additional_driver_flag='--remote-debugging-port=9222' 416 --additional_driver_flag='--remote-debugging-port=9222'
419 --time-out-ms=6000000` 417 --time-out-ms=6000000`
420 * Option B) If you need to debug an http/tests/inspector test, start httpd 418 * Option B) If you need to debug an http/tests/inspector test, start httpd
421 as described above. Then, run content_shell: 419 as described above. Then, run content_shell:
422 `out/Default/content_shell --remote-debugging-port=9222 --run-layout-test 420 `out/Default/content_shell --remote-debugging-port=9222 --run-layout-test
423 http://127.0.0.1:8000/path/to/test.html` 421 http://127.0.0.1:8000/path/to/test.html`
424 * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single 422 * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single
425 link to open the devtools with the test loaded. 423 link to open the devtools with the test loaded.
426 * You may need to replace devtools.html with inspector.html in your URL (or you 424 * You may need to replace devtools.html with inspector.html in your URL (or you
427 can use local chrome inspection of content_shell from chrome://inspect 425 can use local chrome inspection of content_shell from `chrome://inspect`
428 instead) 426 instead)
429 * In the loaded devtools, set any required breakpoints and execute `test()` in 427 * In the loaded devtools, set any required breakpoints and execute `test()` in
430 the console to actually start the test. 428 the console to actually start the test.
431 429
432 ## Rebaselining Layout Tests 430 ## Rebaselining Layout Tests
433 431
434 *** promo 432 *** promo
435 To automatically re-baseline tests across all Chromium platforms, using the 433 To automatically re-baseline tests across all Chromium platforms, using the
436 buildbot results, see the 434 buildbot results, see the
437 [Rebaselining keywords in TestExpectations](./layout_test_expectations.md) 435 [Rebaselining keywords in TestExpectations](./layout_test_expectations.md)
(...skipping 36 matching lines...) Expand 10 before | Expand all | Expand 10 after
474 See 472 See
475 [bugs with the component Blink>Infra](https://bugs.chromium.org/p/chromium/issue s/list?can=2&q=component%3ABlink%3EInfra) 473 [bugs with the component Blink>Infra](https://bugs.chromium.org/p/chromium/issue s/list?can=2&q=component%3ABlink%3EInfra)
476 for issues related to Blink tools, include the layout test runner. 474 for issues related to Blink tools, include the layout test runner.
477 475
478 * Windows and Linux: Do not copy and paste while the layout tests are running, 476 * Windows and Linux: Do not copy and paste while the layout tests are running,
479 as it may interfere with the editing/pasteboard and other clipboard-related 477 as it may interfere with the editing/pasteboard and other clipboard-related
480 tests. (Mac tests swizzle NSClipboard to avoid any conflicts). 478 tests. (Mac tests swizzle NSClipboard to avoid any conflicts).
481 * If QuickTime is not installed, the plugin tests 479 * If QuickTime is not installed, the plugin tests
482 `fast/dom/object-embed-plugin-scripting.html` and 480 `fast/dom/object-embed-plugin-scripting.html` and
483 `plugins/embed-attributes-setting.html` are expected to fail. 481 `plugins/embed-attributes-setting.html` are expected to fail.
OLDNEW
« no previous file with comments | « docs/testing/layout_test_expectations.md ('k') | docs/testing/using_breakpad_with_content_shell.md » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698