OLD | NEW |
1 # Layout Tests | 1 # Layout Tests |
2 | 2 |
3 Layout tests are used by Blink to test many components, including but not | 3 Layout tests are used by Blink to test many components, including but not |
4 limited to layout and rendering. In general, layout tests involve loading pages | 4 limited to layout and rendering. In general, layout tests involve loading pages |
5 in a test renderer (`content_shell`) and comparing the rendered output or | 5 in a test renderer (`content_shell`) and comparing the rendered output or |
6 JavaScript output against an expected output file. | 6 JavaScript output against an expected output file. |
7 | 7 |
8 [TOC] | 8 [TOC] |
9 | 9 |
10 ## Running Layout Tests | 10 ## Running Layout Tests |
(...skipping 162 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
173 than a certain amount of time to complete is aborted and marked as "timed out". | 173 than a certain amount of time to complete is aborted and marked as "timed out". |
174 A row of dots in the script's output indicates one or more tests that passed. | 174 A row of dots in the script's output indicates one or more tests that passed. |
175 | 175 |
176 ## Test expectations | 176 ## Test expectations |
177 | 177 |
178 The | 178 The |
179 [TestExpectations](../../WebKit/LayoutTests/TestExpectations) file (and related | 179 [TestExpectations](../../WebKit/LayoutTests/TestExpectations) file (and related |
180 files, including | 180 files, including |
181 [skia_test_expectations.txt](../../skia/skia_test_expectations.txt)) | 181 [skia_test_expectations.txt](../../skia/skia_test_expectations.txt)) |
182 contains the list of all known layout test failures. See | 182 contains the list of all known layout test failures. See |
183 [Test Expectations](https://sites.google.com/a/chromium.org/dev/developers/testi
ng/webkit-layout-tests/testexpectations) | 183 [Test Expectations](./layout_test_expectations.md) |
184 for more on this. | 184 for more on this. |
185 | 185 |
186 ## Testing Runtime Flags | 186 ## Testing Runtime Flags |
187 | 187 |
188 There are two ways to run layout tests with additional command-line arguments: | 188 There are two ways to run layout tests with additional command-line arguments: |
189 | 189 |
190 * Using `--additional-driver-flag`: | 190 * Using `--additional-driver-flag`: |
191 | 191 |
192 ```bash | 192 ```bash |
193 run-webkit-tests --additional-driver-flag=--blocking-repaint | 193 run-webkit-tests --additional-driver-flag=--blocking-repaint |
(...skipping 219 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
413 * If your test contains a JS call, DOM manipulation, or other distinctive | 413 * If your test contains a JS call, DOM manipulation, or other distinctive |
414 piece of code that you think is failing, search for that in the Chrome | 414 piece of code that you think is failing, search for that in the Chrome |
415 solution. That's a good place to put a starting breakpoint to start | 415 solution. That's a good place to put a starting breakpoint to start |
416 tracking down the issue. | 416 tracking down the issue. |
417 * Otherwise, you're running in a standard message loop just like in Chrome. | 417 * Otherwise, you're running in a standard message loop just like in Chrome. |
418 If you have no other information, set a breakpoint on page load. | 418 If you have no other information, set a breakpoint on page load. |
419 * If your test only works in full layout-test mode, or if you find it simpler to | 419 * If your test only works in full layout-test mode, or if you find it simpler to |
420 debug without all the overhead of an interactive session, start the | 420 debug without all the overhead of an interactive session, start the |
421 content_shell with the command-line flag `--run-layout-test`, followed by the | 421 content_shell with the command-line flag `--run-layout-test`, followed by the |
422 URL (file: or http:) to your test. More information about running layout tests | 422 URL (file: or http:) to your test. More information about running layout tests |
423 in content_shell can be found | 423 in content_shell can be found [here](./layout_tests_in_content_shell.md). |
424 [here](https://www.chromium.org/developers/testing/webkit-layout-tests/content
-shell). | |
425 * In VS, you can do this in the Debugging section of the content_shell | 424 * In VS, you can do this in the Debugging section of the content_shell |
426 project Properties. | 425 project Properties. |
427 * Now you're running with exactly the same API, theme, and other setup that | 426 * Now you're running with exactly the same API, theme, and other setup that |
428 the layout tests use. | 427 the layout tests use. |
429 * Again, if your test contains a JS call, DOM manipulation, or other | 428 * Again, if your test contains a JS call, DOM manipulation, or other |
430 distinctive piece of code that you think is failing, search for that in | 429 distinctive piece of code that you think is failing, search for that in |
431 the Chrome solution. That's a good place to put a starting breakpoint to | 430 the Chrome solution. That's a good place to put a starting breakpoint to |
432 start tracking down the issue. | 431 start tracking down the issue. |
433 * If you can't find any better place to set a breakpoint, start at the | 432 * If you can't find any better place to set a breakpoint, start at the |
434 `TestShell::RunFileTest()` call in `content_shell_main.cc`, or at | 433 `TestShell::RunFileTest()` call in `content_shell_main.cc`, or at |
(...skipping 71 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
506 * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single | 505 * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single |
507 link to open the devtools with the test loaded. | 506 link to open the devtools with the test loaded. |
508 * You may need to replace devtools.html with inspector.html in your URL (or you | 507 * You may need to replace devtools.html with inspector.html in your URL (or you |
509 can use local chrome inspection of content_shell from chrome://inspect | 508 can use local chrome inspection of content_shell from chrome://inspect |
510 instead) | 509 instead) |
511 * In the loaded devtools, set any required breakpoints and execute `test()` in | 510 * In the loaded devtools, set any required breakpoints and execute `test()` in |
512 the console to actually start the test. | 511 the console to actually start the test. |
513 | 512 |
514 ## Rebaselining Layout Tests | 513 ## Rebaselining Layout Tests |
515 | 514 |
516 _To automatically re-baseline tests across all Chromium platforms, using the | 515 *** promo |
| 516 To automatically re-baseline tests across all Chromium platforms, using the |
517 buildbot results, see the | 517 buildbot results, see the |
518 [Rebaselining keywords in TestExpectations](https://www.chromium.org/developers/
testing/webkit-layout-tests/testexpectations#TOC-Rebaselining) | 518 [Rebaselining keywords in TestExpectations](./layout_test_expectations.md) |
519 and | 519 and the |
520 [Rebaselining Tool](https://trac.webkit.org/wiki/Rebaseline). | 520 [Rebaselining Tool](https://trac.webkit.org/wiki/Rebaseline). |
521 Alternatively, to manually run and test and rebaseline it on your workstation, | 521 Alternatively, to manually run and test and rebaseline it on your workstation, |
522 read on._ | 522 read on. |
| 523 *** |
523 | 524 |
524 By default, text-only tests (ones that call `testRunner.dumpAsText()`) produce | 525 By default, text-only tests (ones that call `testRunner.dumpAsText()`) produce |
525 only text results. Other tests produce both new text results and new image | 526 only text results. Other tests produce both new text results and new image |
526 results (the image baseline comprises two files, `-expected.png` and | 527 results (the image baseline comprises two files, `-expected.png` and |
527 `-expected.checksum`). So you'll need either one or three `-expected.\*` files | 528 `-expected.checksum`). So you'll need either one or three `-expected.\*` files |
528 in your new baseline, depending on whether you have a text-only test or not. If | 529 in your new baseline, depending on whether you have a text-only test or not. If |
529 you enable `--no-pixel-tests`, only new text results will be produced, even for | 530 you enable `--no-pixel-tests`, only new text results will be produced, even for |
530 tests that do image comparisons. | 531 tests that do image comparisons. |
531 | 532 |
532 ```bash | 533 ```bash |
(...skipping 22 matching lines...) Expand all Loading... |
555 See | 556 See |
556 [bugs with the component Blink>Infra](https://bugs.chromium.org/p/chromium/issue
s/list?can=2&q=component%3ABlink%3EInfra) | 557 [bugs with the component Blink>Infra](https://bugs.chromium.org/p/chromium/issue
s/list?can=2&q=component%3ABlink%3EInfra) |
557 for issues related to Blink tools, include the layout test runner. | 558 for issues related to Blink tools, include the layout test runner. |
558 | 559 |
559 * Windows and Linux: Do not copy and paste while the layout tests are running, | 560 * Windows and Linux: Do not copy and paste while the layout tests are running, |
560 as it may interfere with the editing/pasteboard and other clipboard-related | 561 as it may interfere with the editing/pasteboard and other clipboard-related |
561 tests. (Mac tests swizzle NSClipboard to avoid any conflicts). | 562 tests. (Mac tests swizzle NSClipboard to avoid any conflicts). |
562 * If QuickTime is not installed, the plugin tests | 563 * If QuickTime is not installed, the plugin tests |
563 `fast/dom/object-embed-plugin-scripting.html` and | 564 `fast/dom/object-embed-plugin-scripting.html` and |
564 `plugins/embed-attributes-setting.html` are expected to fail. | 565 `plugins/embed-attributes-setting.html` are expected to fail. |
OLD | NEW |