Chromium Code Reviews| Index: docs/testing/writing_layout_tests.md |
| diff --git a/docs/testing/writing_layout_tests.md b/docs/testing/writing_layout_tests.md |
| index 6656762d1b294d89bf5643675bc1df43ccdf9a87..3cc004d2ad2b85248609cc2bfc1c340b82f0b448 100644 |
| --- a/docs/testing/writing_layout_tests.md |
| +++ b/docs/testing/writing_layout_tests.md |
| @@ -49,23 +49,23 @@ There are four broad types of layout tests, listed in the order of preference. |
| JavaScript tests, and are only used when JavaScript tests are insufficient, |
| such as when testing paint code. |
| * *Pixel Tests* render a test page and compare the result against a pre-rendered |
| - baseline image in the repository. Pixel tests are less robust than all |
| - alternatives listed above, because the rendering of a page is influenced by |
| + baseline image in the repository. Pixel tests are less robust than the |
| + first two types, because the rendering of a page is influenced by |
| many factors such as the host computer's graphics card and driver, the |
| platform's text rendering system, and various user-configurable operating |
| system settings. For this reason, it is common for a pixel test to have a |
| - different reference image for each platform that Blink is tested on. Pixel |
| - tests are least preferred, because the reference images are |
| - [quite cumbersome to manage](./layout_test_expectations.md). |
| -* *Dump Render Tree (DRT) Tests* output a textual representation of the render |
| + different reference image for each platform that Blink is tested on, and |
| + the reference images are |
| + [quite cumbersome to manage](./layout_test_expectations.md). You |
| + should only write a pixel test if you cannot use a reference test. By default |
| + a pixel test will also dump the render tree as text output, so they are |
| + similar to ... |
| +* *Render tree tests*, which output a textual representation of the render |
|
cbiesinger
2016/12/02 22:15:22
We call it Layout Tree now, fwiw
|
| tree, which is the key data structure in Blink's page rendering system. The |
| - test passes if the output matches a baseline text file in the repository. In |
| - addition to their text result, DRT tests can also produce an image result |
| - which is compared to an image baseline, similarly to pixel tests (described |
| - above). A DRT test with two results (text and image) passes if _both_ results |
| - match the baselines in the repository. DRT tests are less desirable than all |
| - the alternatives, because they depend on a browser implementation detail. |
| - |
| + test passes if the output matches a baseline text file in the repository. |
| + Render tree tests are used as a last resort to test the internal quirks of |
| + the implementation, and they should be avoided in favor of one of the earlier |
| + options. |
| ## General Principles |
| The principles below are adapted from |
| @@ -695,9 +695,9 @@ frame, then call the passed callback. There is also a library at |
| `fast/repaint/resources/text-based-repaint.js` to help with writing paint |
| invalidation and repaint tests. |
| -## Dump Render Tree (DRT) Tests |
| +## Render tree Tests |
| -A Dump Render Tree test renders a web page and produces up to two results, which |
| +A render tree test renders a web page and produces up to two results, which |
| are compared against baseline files: |
| * All tests output a textual representation of Blink's |
| @@ -707,22 +707,34 @@ are compared against baseline files: |
| against an `-expected.png` image baseline, using the same method as pixel |
| tests. |
| -TODO: Document the API used by DRT tests to opt out of producing image results. |
| - |
| -A DRT test passes if _all_ of its results match their baselines. Like pixel |
| -tests, the output of DRT tests depends on platform-specific mechanisms, so DRT |
| -tests often require per-platform baselines. Furthermore, DRT tests depend on the |
| -render tree data structure, which means that if we replace the render tree data |
| -structure, we will need to look at each DRT test and consider whether it is |
| -still meaningful. |
| - |
| -For these reasons, DRT tests should **only** be used to cover aspects of the |
| -layout code that can only be tested by looking at the render tree. Any |
| -combination of the other test types is preferable to a DRT test. DRT tests are |
| +Whether you want a pixel test or a render tree test depends on whether |
| +you care about the visual image, the details of how that image was |
| +constructed, or both. It is possible for multiple render trees to produce |
| +the same pixel output, so it is important to make it clear in the test |
| +which outputs you really care about. |
| + |
| +TODO: Document the API used by render tree tests to opt out of producing image |
| +results. |
|
m_gsnedders
2016/12/02 14:16:44
Given the intro above says that by default "a pixe
|
| + |
| +A render tree test passes if _all_ of its results match their baselines. Like pixel |
| +tests, the output of render tree tests depends on platform-specific details, |
| +so render tree tests often require per-platform baselines. Furthermore, |
| +since the tests obviously depend on the render tree structure, |
| +that means that if we change the render tree you have to rebaseline each |
| +render tree test to see if the results are still correct and whether the test |
| +is still meaningful. There are actually many cases where the render tree |
|
pwnall
2016/12/02 03:20:48
Could "many cases" link to a crbug that tracks rem
|
| +output is misstated (i.e., wrong), because people didn't want to have to update |
| +existing baselines and tests. This is really unfortunate and confusing. |
| + |
| +For these reasons, render tree tests should **only** be used to cover aspects |
| +of the layout code that can only be tested by looking at the render tree. Any |
| +combination of the other test types is preferable to a render tree test. |
| +render tree tests are |
|
m_gsnedders
2016/12/02 14:16:44
(grammar): needs a capital R
|
| [inherited from WebKit](https://webkit.org/blog/1456/layout-tests-practice/), so |
| -the repository may have some unfortunate examples of DRT tests. |
| +the repository may have some unfortunate examples of render tree tests. |
| + |
| -The following page is an example of a DRT test. |
| +The following page is an example of a render tree test. |
| ```html |
| <!doctype html> |
| @@ -763,7 +775,7 @@ test page uses the Ahem font (introduced above), whose main design goal is |
| consistent cross-platform rendering. Had the test used another font, its text |
| baseline would have depended on the fonts installed on the testing computer, and |
| on the platform's font rendering system. Please follow the pixel tests |
| -guidelines and write reliable DRT tests! |
| +guidelines and write reliable render tree tests! |
| WebKit's render tree is described in |
| [a series of posts](https://webkit.org/blog/114/webcore-rendering-i-the-basics/) |