Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(3)

Unified Diff: docs/testing/writing_layout_tests.md

Issue 2547463003: Tweak the descriptions of the various types of layout tests. (Closed)
Patch Set: rework, remove other DRT references Created 4 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
Index: docs/testing/writing_layout_tests.md
diff --git a/docs/testing/writing_layout_tests.md b/docs/testing/writing_layout_tests.md
index 6656762d1b294d89bf5643675bc1df43ccdf9a87..3cc004d2ad2b85248609cc2bfc1c340b82f0b448 100644
--- a/docs/testing/writing_layout_tests.md
+++ b/docs/testing/writing_layout_tests.md
@@ -49,23 +49,23 @@ There are four broad types of layout tests, listed in the order of preference.
JavaScript tests, and are only used when JavaScript tests are insufficient,
such as when testing paint code.
* *Pixel Tests* render a test page and compare the result against a pre-rendered
- baseline image in the repository. Pixel tests are less robust than all
- alternatives listed above, because the rendering of a page is influenced by
+ baseline image in the repository. Pixel tests are less robust than the
+ first two types, because the rendering of a page is influenced by
many factors such as the host computer's graphics card and driver, the
platform's text rendering system, and various user-configurable operating
system settings. For this reason, it is common for a pixel test to have a
- different reference image for each platform that Blink is tested on. Pixel
- tests are least preferred, because the reference images are
- [quite cumbersome to manage](./layout_test_expectations.md).
-* *Dump Render Tree (DRT) Tests* output a textual representation of the render
+ different reference image for each platform that Blink is tested on, and
+ the reference images are
+ [quite cumbersome to manage](./layout_test_expectations.md). You
+ should only write a pixel test if you cannot use a reference test. By default
+ a pixel test will also dump the render tree as text output, so they are
+ similar to ...
+* *Render tree tests*, which output a textual representation of the render
cbiesinger 2016/12/02 22:15:22 We call it Layout Tree now, fwiw
tree, which is the key data structure in Blink's page rendering system. The
- test passes if the output matches a baseline text file in the repository. In
- addition to their text result, DRT tests can also produce an image result
- which is compared to an image baseline, similarly to pixel tests (described
- above). A DRT test with two results (text and image) passes if _both_ results
- match the baselines in the repository. DRT tests are less desirable than all
- the alternatives, because they depend on a browser implementation detail.
-
+ test passes if the output matches a baseline text file in the repository.
+ Render tree tests are used as a last resort to test the internal quirks of
+ the implementation, and they should be avoided in favor of one of the earlier
+ options.
## General Principles
The principles below are adapted from
@@ -695,9 +695,9 @@ frame, then call the passed callback. There is also a library at
`fast/repaint/resources/text-based-repaint.js` to help with writing paint
invalidation and repaint tests.
-## Dump Render Tree (DRT) Tests
+## Render tree Tests
-A Dump Render Tree test renders a web page and produces up to two results, which
+A render tree test renders a web page and produces up to two results, which
are compared against baseline files:
* All tests output a textual representation of Blink's
@@ -707,22 +707,34 @@ are compared against baseline files:
against an `-expected.png` image baseline, using the same method as pixel
tests.
-TODO: Document the API used by DRT tests to opt out of producing image results.
-
-A DRT test passes if _all_ of its results match their baselines. Like pixel
-tests, the output of DRT tests depends on platform-specific mechanisms, so DRT
-tests often require per-platform baselines. Furthermore, DRT tests depend on the
-render tree data structure, which means that if we replace the render tree data
-structure, we will need to look at each DRT test and consider whether it is
-still meaningful.
-
-For these reasons, DRT tests should **only** be used to cover aspects of the
-layout code that can only be tested by looking at the render tree. Any
-combination of the other test types is preferable to a DRT test. DRT tests are
+Whether you want a pixel test or a render tree test depends on whether
+you care about the visual image, the details of how that image was
+constructed, or both. It is possible for multiple render trees to produce
+the same pixel output, so it is important to make it clear in the test
+which outputs you really care about.
+
+TODO: Document the API used by render tree tests to opt out of producing image
+results.
m_gsnedders 2016/12/02 14:16:44 Given the intro above says that by default "a pixe
+
+A render tree test passes if _all_ of its results match their baselines. Like pixel
+tests, the output of render tree tests depends on platform-specific details,
+so render tree tests often require per-platform baselines. Furthermore,
+since the tests obviously depend on the render tree structure,
+that means that if we change the render tree you have to rebaseline each
+render tree test to see if the results are still correct and whether the test
+is still meaningful. There are actually many cases where the render tree
pwnall 2016/12/02 03:20:48 Could "many cases" link to a crbug that tracks rem
+output is misstated (i.e., wrong), because people didn't want to have to update
+existing baselines and tests. This is really unfortunate and confusing.
+
+For these reasons, render tree tests should **only** be used to cover aspects
+of the layout code that can only be tested by looking at the render tree. Any
+combination of the other test types is preferable to a render tree test.
+render tree tests are
m_gsnedders 2016/12/02 14:16:44 (grammar): needs a capital R
[inherited from WebKit](https://webkit.org/blog/1456/layout-tests-practice/), so
-the repository may have some unfortunate examples of DRT tests.
+the repository may have some unfortunate examples of render tree tests.
+
-The following page is an example of a DRT test.
+The following page is an example of a render tree test.
```html
<!doctype html>
@@ -763,7 +775,7 @@ test page uses the Ahem font (introduced above), whose main design goal is
consistent cross-platform rendering. Had the test used another font, its text
baseline would have depended on the fonts installed on the testing computer, and
on the platform's font rendering system. Please follow the pixel tests
-guidelines and write reliable DRT tests!
+guidelines and write reliable render tree tests!
WebKit's render tree is described in
[a series of posts](https://webkit.org/blog/114/webcore-rendering-i-the-basics/)
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698