OLD | NEW |
1 Correctness Testing | 1 Correctness Testing |
2 =================== | 2 =================== |
3 | 3 |
4 Skia correctness testing is primarily served by a tool named DM. | 4 Skia correctness testing is primarily served by a tool named DM. |
5 This is a quickstart to building and running DM. | 5 This is a quickstart to building and running DM. |
6 | 6 |
7 ~~~ | 7 ~~~ |
8 $ ./gyp_skia | 8 $ ./gyp_skia |
9 $ ninja -C out/Debug dm | 9 $ ninja -C out/Debug dm |
10 $ out/Debug/dm -v -w dm_output | 10 $ out/Debug/dm -v -w dm_output |
(...skipping 109 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
120 dm_output/565/gm/bezier_quad_effects.png | 120 dm_output/565/gm/bezier_quad_effects.png |
121 ... | 121 ... |
122 ~~~ | 122 ~~~ |
123 | 123 |
124 The directories are nested first by sink type (`--config`), then by source type
(`--src`). | 124 The directories are nested first by sink type (`--config`), then by source type
(`--src`). |
125 The image from the task we just looked at, "8888 image mandrill_132x132_12x12.as
tc-5-subsets", | 125 The image from the task we just looked at, "8888 image mandrill_132x132_12x12.as
tc-5-subsets", |
126 can be found at dm_output/8888/image/mandrill_132x132_12x12.astc-5-subsets.png. | 126 can be found at dm_output/8888/image/mandrill_132x132_12x12.astc-5-subsets.png. |
127 | 127 |
128 dm.json is used by our automated testing system, so you can ignore it if you | 128 dm.json is used by our automated testing system, so you can ignore it if you |
129 like. It contains a listing of each test run and a checksum of the image | 129 like. It contains a listing of each test run and a checksum of the image |
130 generated for that run. (Boring technical detail: it is not a checksum of the | 130 generated for that run. |
131 .png file, but rather a checksum of the raw pixels used to create that .png.) | 131 |
| 132 ### Detail <a name="digests"></a> |
| 133 Boring technical detail: The checksum is not a checksum of the |
| 134 .png file, but rather a checksum of the raw pixels used to create that .png. |
| 135 That means it is possible for two different configurations to produce |
| 136 the same exact .png, but have their checksums differ. |
132 | 137 |
133 Unit tests don't generally output anything but a status update when they pass. | 138 Unit tests don't generally output anything but a status update when they pass. |
134 If a test fails, DM will print out its assertion failures, both at the time | 139 If a test fails, DM will print out its assertion failures, both at the time |
135 they happen and then again all together after everything is done running. | 140 they happen and then again all together after everything is done running. |
136 These failures are also included in the dm.json file. | 141 These failures are also included in the dm.json file. |
137 | 142 |
138 DM has a simple facility to compare against the results of a previous run: | 143 DM has a simple facility to compare against the results of a previous run: |
139 ~~~ | 144 ~~~ |
140 $ ./gyp_skia | 145 $ ./gyp_skia |
141 $ ninja -C out/Debug dm | 146 $ ninja -C out/Debug dm |
(...skipping 30 matching lines...) Expand all Loading... |
172 That's the basics of DM. DM supports many other modes and flags. Here are a | 177 That's the basics of DM. DM supports many other modes and flags. Here are a |
173 few examples you might find handy. | 178 few examples you might find handy. |
174 ~~~ | 179 ~~~ |
175 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. | 180 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. |
176 $ out/Debug/dm --src tests # Run only unit tests. | 181 $ out/Debug/dm --src tests # Run only unit tests. |
177 $ out/Debug/dm --nocpu # Test only GPU-backed work. | 182 $ out/Debug/dm --nocpu # Test only GPU-backed work. |
178 $ out/Debug/dm --nogpu # Test only CPU-backed work. | 183 $ out/Debug/dm --nogpu # Test only CPU-backed work. |
179 $ out/Debug/dm --match blur # Run only work with "blur" in its name. | 184 $ out/Debug/dm --match blur # Run only work with "blur" in its name. |
180 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. | 185 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. |
181 ~~~ | 186 ~~~ |
OLD | NEW |