OLD | NEW |
1 # How to Extend the Layout Test Framework | 1 # How to Extend the Layout Test Framework |
2 | 2 |
3 The Layout Test Framework that Blink uses is a regression testing tool that is | 3 The Layout Test Framework that Blink uses is a regression testing tool that is |
4 multi-platform and it has a large amount of tools that help test varying types | 4 multi-platform and it has a large amount of tools that help test varying types |
5 of regression, such as pixel diffs, text diffs, etc. The framework is mainly | 5 of regression, such as pixel diffs, text diffs, etc. The framework is mainly |
6 used by Blink, however it was made to be extensible so that other projects can | 6 used by Blink, however it was made to be extensible so that other projects can |
7 use it test different parts of chrome (such as Print Preview). This is a guide | 7 use it test different parts of chrome (such as Print Preview). This is a guide |
8 to help people who want to actually the framework to test whatever they want. | 8 to help people who want to actually the framework to test whatever they want. |
9 | 9 |
10 [TOC] | 10 [TOC] |
11 | 11 |
12 ## Background | 12 ## Background |
13 | 13 |
14 Before you can start actually extending the framework, you should be familiar | 14 Before you can start actually extending the framework, you should be familiar |
15 with how to use it. See the | 15 with how to use it. See the |
16 [layout tests documentation](testing/layout_tests.md). | 16 [layout tests documentation](testing/layout_tests.md). |
17 | 17 |
18 ## How to Extend the Framework | 18 ## How to Extend the Framework |
19 | 19 |
20 There are two parts to actually extending framework to test a piece of software. | 20 There are two parts to actually extending framework to test a piece of software. |
21 The first part is extending certain files in: | 21 The first part is extending certain files in: |
22 [/third_party/Webkit/Tools/Scripts/webkitpy/layout_tests/](/third_party/Webkit/T
ools/Scripts/webkitpy/layout_tests/) | 22 [/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/](/third_party/WebKit/T
ools/Scripts/webkitpy/layout_tests/) |
23 The code in `webkitpy/layout_tests` is the layout test framework itself | 23 The code in `webkitpy/layout_tests` is the layout test framework itself |
24 | 24 |
25 The second part is creating a driver (program) to actually communicate the | 25 The second part is creating a driver (program) to actually communicate the |
26 layout test framework. This part is significantly more tricky and dependant on | 26 layout test framework. This part is significantly more tricky and dependent on |
27 what exactly exactly is being tested. | 27 what exactly exactly is being tested. |
28 | 28 |
29 ### Part 1 | 29 ### Part 1 |
30 | 30 |
31 This part isn’t too difficult. There are basically two classes that need to be | 31 This part isn’t too difficult. There are basically two classes that need to be |
32 extended (ideally, just inherited from). These classes are: | 32 extended (ideally, just inherited from). These classes are: |
33 | 33 |
34 * `Driver`. Located in `layout_tests/port/driver.py`. Each instance of this is | 34 * `Driver`. Located in `layout_tests/port/driver.py`. Each instance of this is |
35 the class that will actually an instance of the program that produces the | 35 the class that will actually an instance of the program that produces the |
36 test data (program in Part 2). | 36 test data (program in Part 2). |
(...skipping 65 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
102 | 102 |
103 This class is responsible for providing functionality such as where to look for | 103 This class is responsible for providing functionality such as where to look for |
104 tests, where to store test results, what driver to run, what timeout to use, | 104 tests, where to store test results, what driver to run, what timeout to use, |
105 what kind of files can be run, etc. It provides a lot of functionality, however | 105 what kind of files can be run, etc. It provides a lot of functionality, however |
106 it isn’t really sufficient because it doesn’t account of platform specific | 106 it isn’t really sufficient because it doesn’t account of platform specific |
107 problems, therefore port itself shouldn’t be extend. Instead LinuxPort, WinPort, | 107 problems, therefore port itself shouldn’t be extend. Instead LinuxPort, WinPort, |
108 and MacPort (and maybe the android port class) should be extended as they | 108 and MacPort (and maybe the android port class) should be extended as they |
109 provide platform specific overrides/extensions that implement most of the | 109 provide platform specific overrides/extensions that implement most of the |
110 important functionality. While there are many functions in Port, overriding one | 110 important functionality. While there are many functions in Port, overriding one |
111 function will affect most of the other ones to get the desired behavior. For | 111 function will affect most of the other ones to get the desired behavior. For |
112 example, if `layout_tests_dir()` is overriden, not only will the code look for | 112 example, if `layout_tests_dir()` is overridden, not only will the code look for |
113 tests in that directory, but it will find the correct TestExpectations file, the | 113 tests in that directory, but it will find the correct TestExpectations file, the |
114 platform specific expected files, etc. | 114 platform specific expected files, etc. |
115 | 115 |
116 Here are some of the functions that most likely need to be overridden. | 116 Here are some of the functions that most likely need to be overridden. |
117 | 117 |
118 * `driver_class` | 118 * `driver_class` |
119 * This should be overridden to allow the testing program to actually run. | 119 * This should be overridden to allow the testing program to actually run. |
120 By default the code will run content_shell, which might or might not be | 120 By default the code will run content_shell, which might or might not be |
121 what you want. | 121 what you want. |
122 * It should be overridden to return the driver extension class created | 122 * It should be overridden to return the driver extension class created |
(...skipping 129 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
252 * In this case this is how your output should look. | 252 * In this case this is how your output should look. |
253 * “Content-type: image/png\n” | 253 * “Content-type: image/png\n” |
254 * “ActualHash: hashData\n” | 254 * “ActualHash: hashData\n” |
255 * “Content-Length: lengthOfPng\n” | 255 * “Content-Length: lengthOfPng\n” |
256 * “pngdata” | 256 * “pngdata” |
257 * This part doesn’t need a header specifying that you are | 257 * This part doesn’t need a header specifying that you are |
258 sending png data, just send it | 258 sending png data, just send it |
259 * “#EOF\n” on both stdout and stderr | 259 * “#EOF\n” on both stdout and stderr |
260 * To see the structure of the data required, look at the | 260 * To see the structure of the data required, look at the |
261 `read_block` functions in Driver.py | 261 `read_block` functions in Driver.py |
OLD | NEW |