| Index: docs/testing/writing_layout_tests.md
|
| diff --git a/docs/testing/writing_layout_tests.md b/docs/testing/writing_layout_tests.md
|
| index f2767c206dedb34307ae5b5972ed16a7271b5d22..37e5e2ab7afa861739aaf50c766437d173b43780 100644
|
| --- a/docs/testing/writing_layout_tests.md
|
| +++ b/docs/testing/writing_layout_tests.md
|
| @@ -51,23 +51,23 @@ There are four broad types of layout tests, listed in the order of preference.
|
| JavaScript tests, and are only used when JavaScript tests are insufficient,
|
| such as when testing paint code.
|
| * *Pixel Tests* render a test page and compare the result against a pre-rendered
|
| - baseline image in the repository. Pixel tests are less robust than all
|
| - alternatives listed above, because the rendering of a page is influenced by
|
| + baseline image in the repository. Pixel tests are less robust than the
|
| + first two types, because the rendering of a page is influenced by
|
| many factors such as the host computer's graphics card and driver, the
|
| platform's text rendering system, and various user-configurable operating
|
| system settings. For this reason, it is common for a pixel test to have a
|
| - different reference image for each platform that Blink is tested on. Pixel
|
| - tests are least preferred, because the reference images are
|
| - [quite cumbersome to manage](./layout_test_expectations.md).
|
| -* *Dump Render Tree (DRT) Tests* output a textual representation of the render
|
| + different reference image for each platform that Blink is tested on, and
|
| + the reference images are
|
| + [quite cumbersome to manage](./layout_test_expectations.md). You
|
| + should only write a pixel test if you cannot use a reference test. By default
|
| + a pixel test will also dump the layout tree as text output, so they are
|
| + similar to ...
|
| +* *Layout tree tests*, which output a textual representation of the layout
|
| tree, which is the key data structure in Blink's page rendering system. The
|
| - test passes if the output matches a baseline text file in the repository. In
|
| - addition to their text result, DRT tests can also produce an image result
|
| - which is compared to an image baseline, similarly to pixel tests (described
|
| - above). A DRT test with two results (text and image) passes if _both_ results
|
| - match the baselines in the repository. DRT tests are less desirable than all
|
| - the alternatives, because they depend on a browser implementation detail.
|
| -
|
| + test passes if the output matches a baseline text file in the repository.
|
| + Layout tree tests are used as a last resort to test the internal quirks of
|
| + the implementation, and they should be avoided in favor of one of the earlier
|
| + options.
|
| ## General Principles
|
|
|
| The principles below are adapted from
|
| @@ -760,6 +760,9 @@ contain useful guidance. The most relevant pieces of advice are below.
|
| apply when testing text, text flow, font selection, font fallback, font
|
| features, or other typographic information.
|
|
|
| +TODO: Document how to opt out of generating a layout tree when generating
|
| +pixel results.
|
| +
|
| *** promo
|
| When using `window.testRunner.dumpAsTextWithPixelResults()`, the image result
|
| will always be 800x600px, because test pages are rendered in an 800x600px
|
| @@ -798,34 +801,46 @@ frame, then call the passed callback. There is also a library at
|
| `fast/repaint/resources/text-based-repaint.js` to help with writing paint
|
| invalidation and repaint tests.
|
|
|
| -## Dump Render Tree (DRT) Tests
|
| +## Layout tree tests
|
|
|
| -A Dump Render Tree test renders a web page and produces up to two results, which
|
| +A layout tree test renders a web page and produces up to two results, which
|
| are compared against baseline files:
|
|
|
| * All tests output a textual representation of Blink's
|
| - [render tree](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction),
|
| + [layout tree](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction) (called the render tree on that page),
|
| which is compared against an `-expected.txt` text baseline.
|
| * Some tests also output the image of the rendered page, which is compared
|
| against an `-expected.png` image baseline, using the same method as pixel
|
| tests.
|
|
|
| -TODO: Document the API used by DRT tests to opt out of producing image results.
|
| -
|
| -A DRT test passes if _all_ of its results match their baselines. Like pixel
|
| -tests, the output of DRT tests depends on platform-specific mechanisms, so DRT
|
| -tests often require per-platform baselines. Furthermore, DRT tests depend on the
|
| -render tree data structure, which means that if we replace the render tree data
|
| -structure, we will need to look at each DRT test and consider whether it is
|
| -still meaningful.
|
| -
|
| -For these reasons, DRT tests should **only** be used to cover aspects of the
|
| -layout code that can only be tested by looking at the render tree. Any
|
| -combination of the other test types is preferable to a DRT test. DRT tests are
|
| +Whether you want a pixel test or a layout tree test depends on whether
|
| +you care about the visual image, the details of how that image was
|
| +constructed, or both. It is possible for multiple layout trees to produce
|
| +the same pixel output, so it is important to make it clear in the test
|
| +which outputs you really care about.
|
| +
|
| +TODO: Document the API used by layout tree tests to opt out of producing image
|
| +results.
|
| +
|
| +A layout tree test passes if _all_ of its results match their baselines. Like pixel
|
| +tests, the output of layout tree tests depends on platform-specific details,
|
| +so layout tree tests often require per-platform baselines. Furthermore,
|
| +since the tests obviously depend on the layout tree structure,
|
| +that means that if we change the layout tree you have to rebaseline each
|
| +layout tree test to see if the results are still correct and whether the test
|
| +is still meaningful. There are actually many cases where the layout tree
|
| +output is misstated (i.e., wrong), because people didn't want to have to update
|
| +existing baselines and tests. This is really unfortunate and confusing.
|
| +
|
| +For these reasons, layout tree tests should **only** be used to cover aspects
|
| +of the layout code that can only be tested by looking at the layout tree. Any
|
| +combination of the other test types is preferable to a layout tree test.
|
| +Layout tree tests are
|
| [inherited from WebKit](https://webkit.org/blog/1456/layout-tests-practice/), so
|
| -the repository may have some unfortunate examples of DRT tests.
|
| +the repository may have some unfortunate examples of layout tree tests.
|
| +
|
|
|
| -The following page is an example of a DRT test.
|
| +The following page is an example of a layout tree test.
|
|
|
| ```html
|
| <!doctype html>
|
| @@ -866,11 +881,11 @@ test page uses the Ahem font (introduced above), whose main design goal is
|
| consistent cross-platform rendering. Had the test used another font, its text
|
| baseline would have depended on the fonts installed on the testing computer, and
|
| on the platform's font rendering system. Please follow the pixel tests
|
| -guidelines and write reliable DRT tests!
|
| +guidelines and write reliable layout tree tests!
|
|
|
| -WebKit's render tree is described in
|
| +WebKit's layout tree is described in
|
| [a series of posts](https://webkit.org/blog/114/webcore-rendering-i-the-basics/)
|
| -on WebKit's blog. Some of the concepts there still apply to Blink's render tree.
|
| +on WebKit's blog. Some of the concepts there still apply to Blink's layout tree.
|
|
|
| ## Directory Structure
|
|
|
|
|