Index: docs/testing/writing_layout_tests.md |
diff --git a/docs/testing/writing_layout_tests.md b/docs/testing/writing_layout_tests.md |
index 6656762d1b294d89bf5643675bc1df43ccdf9a87..3cc004d2ad2b85248609cc2bfc1c340b82f0b448 100644 |
--- a/docs/testing/writing_layout_tests.md |
+++ b/docs/testing/writing_layout_tests.md |
@@ -49,23 +49,23 @@ There are four broad types of layout tests, listed in the order of preference. |
JavaScript tests, and are only used when JavaScript tests are insufficient, |
such as when testing paint code. |
* *Pixel Tests* render a test page and compare the result against a pre-rendered |
- baseline image in the repository. Pixel tests are less robust than all |
- alternatives listed above, because the rendering of a page is influenced by |
+ baseline image in the repository. Pixel tests are less robust than the |
+ first two types, because the rendering of a page is influenced by |
many factors such as the host computer's graphics card and driver, the |
platform's text rendering system, and various user-configurable operating |
system settings. For this reason, it is common for a pixel test to have a |
- different reference image for each platform that Blink is tested on. Pixel |
- tests are least preferred, because the reference images are |
- [quite cumbersome to manage](./layout_test_expectations.md). |
-* *Dump Render Tree (DRT) Tests* output a textual representation of the render |
+ different reference image for each platform that Blink is tested on, and |
+ the reference images are |
+ [quite cumbersome to manage](./layout_test_expectations.md). You |
+ should only write a pixel test if you cannot use a reference test. By default |
+ a pixel test will also dump the render tree as text output, so they are |
+ similar to ... |
+* *Render tree tests*, which output a textual representation of the render |
cbiesinger
2016/12/02 22:15:22
We call it Layout Tree now, fwiw
|
tree, which is the key data structure in Blink's page rendering system. The |
- test passes if the output matches a baseline text file in the repository. In |
- addition to their text result, DRT tests can also produce an image result |
- which is compared to an image baseline, similarly to pixel tests (described |
- above). A DRT test with two results (text and image) passes if _both_ results |
- match the baselines in the repository. DRT tests are less desirable than all |
- the alternatives, because they depend on a browser implementation detail. |
- |
+ test passes if the output matches a baseline text file in the repository. |
+ Render tree tests are used as a last resort to test the internal quirks of |
+ the implementation, and they should be avoided in favor of one of the earlier |
+ options. |
## General Principles |
The principles below are adapted from |
@@ -695,9 +695,9 @@ frame, then call the passed callback. There is also a library at |
`fast/repaint/resources/text-based-repaint.js` to help with writing paint |
invalidation and repaint tests. |
-## Dump Render Tree (DRT) Tests |
+## Render tree Tests |
-A Dump Render Tree test renders a web page and produces up to two results, which |
+A render tree test renders a web page and produces up to two results, which |
are compared against baseline files: |
* All tests output a textual representation of Blink's |
@@ -707,22 +707,34 @@ are compared against baseline files: |
against an `-expected.png` image baseline, using the same method as pixel |
tests. |
-TODO: Document the API used by DRT tests to opt out of producing image results. |
- |
-A DRT test passes if _all_ of its results match their baselines. Like pixel |
-tests, the output of DRT tests depends on platform-specific mechanisms, so DRT |
-tests often require per-platform baselines. Furthermore, DRT tests depend on the |
-render tree data structure, which means that if we replace the render tree data |
-structure, we will need to look at each DRT test and consider whether it is |
-still meaningful. |
- |
-For these reasons, DRT tests should **only** be used to cover aspects of the |
-layout code that can only be tested by looking at the render tree. Any |
-combination of the other test types is preferable to a DRT test. DRT tests are |
+Whether you want a pixel test or a render tree test depends on whether |
+you care about the visual image, the details of how that image was |
+constructed, or both. It is possible for multiple render trees to produce |
+the same pixel output, so it is important to make it clear in the test |
+which outputs you really care about. |
+ |
+TODO: Document the API used by render tree tests to opt out of producing image |
+results. |
m_gsnedders
2016/12/02 14:16:44
Given the intro above says that by default "a pixe
|
+ |
+A render tree test passes if _all_ of its results match their baselines. Like pixel |
+tests, the output of render tree tests depends on platform-specific details, |
+so render tree tests often require per-platform baselines. Furthermore, |
+since the tests obviously depend on the render tree structure, |
+that means that if we change the render tree you have to rebaseline each |
+render tree test to see if the results are still correct and whether the test |
+is still meaningful. There are actually many cases where the render tree |
pwnall
2016/12/02 03:20:48
Could "many cases" link to a crbug that tracks rem
|
+output is misstated (i.e., wrong), because people didn't want to have to update |
+existing baselines and tests. This is really unfortunate and confusing. |
+ |
+For these reasons, render tree tests should **only** be used to cover aspects |
+of the layout code that can only be tested by looking at the render tree. Any |
+combination of the other test types is preferable to a render tree test. |
+render tree tests are |
m_gsnedders
2016/12/02 14:16:44
(grammar): needs a capital R
|
[inherited from WebKit](https://webkit.org/blog/1456/layout-tests-practice/), so |
-the repository may have some unfortunate examples of DRT tests. |
+the repository may have some unfortunate examples of render tree tests. |
+ |
-The following page is an example of a DRT test. |
+The following page is an example of a render tree test. |
```html |
<!doctype html> |
@@ -763,7 +775,7 @@ test page uses the Ahem font (introduced above), whose main design goal is |
consistent cross-platform rendering. Had the test used another font, its text |
baseline would have depended on the fonts installed on the testing computer, and |
on the platform's font rendering system. Please follow the pixel tests |
-guidelines and write reliable DRT tests! |
+guidelines and write reliable render tree tests! |
WebKit's render tree is described in |
[a series of posts](https://webkit.org/blog/114/webcore-rendering-i-the-basics/) |