OLD | NEW |
1 Correctness Testing | 1 Correctness Testing |
2 =================== | 2 =================== |
3 | 3 |
4 Skia correctness testing is primarily served by a tool named DM. | 4 Skia correctness testing is primarily served by a tool named DM. |
5 This is a quickstart to building and running DM. | 5 This is a quickstart to building and running DM. |
6 | 6 |
7 ~~~ | 7 ~~~ |
8 $ ./gyp_skia | 8 $ ./gyp_skia |
9 $ ninja -C out/Debug dm | 9 $ ninja -C out/Debug dm |
10 $ out/Debug/dm -v -w dm_output | 10 $ out/Debug/dm -v -w dm_output |
11 ~~~ | 11 ~~~ |
12 | 12 |
13 When you run this, you may notice your CPU peg to 100% for a while, then taper | 13 When you run this, you may notice your CPU peg to 100% for a while, then taper |
14 off to 1 or 2 active cores as the run finishes. This is intentional. DM is | 14 off to 1 or 2 active cores as the run finishes. This is intentional. DM is |
15 very multithreaded, but some of the work, particularly GPU-backed work, is | 15 very multithreaded, but some of the work, particularly GPU-backed work, is |
16 still forced to run on a single thread. You can use --threads N to limit DM to | 16 still forced to run on a single thread. You can use `--threads N` to limit DM t
o |
17 N threads if you like. This can sometimes be helpful on machines that have | 17 N threads if you like. This can sometimes be helpful on machines that have |
18 relatively more CPU available than RAM. | 18 relatively more CPU available than RAM. |
19 | 19 |
20 As DM runs, you ought to see a giant spew of output that looks something like th
is. | 20 As DM runs, you ought to see a giant spew of output that looks something like th
is. |
21 ~~~ | 21 ~~~ |
22 Skipping nonrendering: Don't understand 'nonrendering'. | 22 Skipping nonrendering: Don't understand 'nonrendering'. |
23 Skipping angle: Don't understand 'angle'. | 23 Skipping angle: Don't understand 'angle'. |
24 Skipping nvprmsaa4: Could not create a surface. | 24 Skipping nvprmsaa4: Could not create a surface. |
25 492 srcs * 3 sinks + 382 tests == 1858 tasks | 25 492 srcs * 3 sinks + 382 tests == 1858 tasks |
26 | 26 |
(...skipping 22 matching lines...) Expand all Loading... |
49 machines. These lines are a sort of FYI, mostly in case DM can't run some | 49 machines. These lines are a sort of FYI, mostly in case DM can't run some |
50 configuration you might be expecting it to run. | 50 configuration you might be expecting it to run. |
51 | 51 |
52 The next line is an overview of the work DM is about to do. | 52 The next line is an overview of the work DM is about to do. |
53 ~~~ | 53 ~~~ |
54 492 srcs * 3 sinks + 382 tests == 1858 tasks | 54 492 srcs * 3 sinks + 382 tests == 1858 tasks |
55 ~~~ | 55 ~~~ |
56 | 56 |
57 DM has found 382 unit tests (code linked in from tests/), and 492 other drawing | 57 DM has found 382 unit tests (code linked in from tests/), and 492 other drawing |
58 sources. These drawing sources may be GM integration tests (code linked in | 58 sources. These drawing sources may be GM integration tests (code linked in |
59 from gm/), image files (from --images, which defaults to "resources") or .skp | 59 from gm/), image files (from `--images`, which defaults to "resources") or .skp |
60 files (from --skps, which defaults to "skps"). You can control the types of | 60 files (from `--skps`, which defaults to "skps"). You can control the types of |
61 sources DM will use with --src (default, "tests gm image skp"). | 61 sources DM will use with `--src` (default, "tests gm image skp"). |
62 | 62 |
63 DM has found 3 usable ways to draw those 492 sources. This is controlled by | 63 DM has found 3 usable ways to draw those 492 sources. This is controlled by |
64 --config, which today defaults to "565 8888 gpu nonrendering angle nvprmsaa4". | 64 `--config`, which today defaults to "565 8888 gpu nonrendering angle nvprmsaa4". |
65 DM has skipped nonrendering, angle, and nvprmssa4, leaving three usable configs: | 65 DM has skipped nonrendering, angle, and nvprmssa4, leaving three usable configs: |
66 565, 8888, and gpu. These three name different ways to draw using Skia: | 66 565, 8888, and gpu. These three name different ways to draw using Skia: |
67 | 67 |
68 - 565: draw using the software backend into a 16-bit RGB bitmap | 68 - 565: draw using the software backend into a 16-bit RGB bitmap |
69 - 8888: draw using the software backend into a 32-bit RGBA bitmap | 69 - 8888: draw using the software backend into a 32-bit RGBA bitmap |
70 - gpu: draw using the GPU backend (Ganesh) into a 32-bit RGBA bitmap | 70 - gpu: draw using the GPU backend (Ganesh) into a 32-bit RGBA bitmap |
71 | 71 |
72 Sometimes DM calls these configs, sometimes sinks. Sorry. There are many | 72 Sometimes DM calls these configs, sometimes sinks. Sorry. There are many |
73 possible configs but generally we pay most attention to 8888 and gpu. | 73 possible configs but generally we pay most attention to 8888 and gpu. |
74 | 74 |
(...skipping 39 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
114 dm_output/565/gm/alphagradients.png | 114 dm_output/565/gm/alphagradients.png |
115 dm_output/565/gm/arcofzorro.png | 115 dm_output/565/gm/arcofzorro.png |
116 dm_output/565/gm/arithmode.png | 116 dm_output/565/gm/arithmode.png |
117 dm_output/565/gm/astcbitmap.png | 117 dm_output/565/gm/astcbitmap.png |
118 dm_output/565/gm/bezier_conic_effects.png | 118 dm_output/565/gm/bezier_conic_effects.png |
119 dm_output/565/gm/bezier_cubic_effects.png | 119 dm_output/565/gm/bezier_cubic_effects.png |
120 dm_output/565/gm/bezier_quad_effects.png | 120 dm_output/565/gm/bezier_quad_effects.png |
121 ... | 121 ... |
122 ~~~ | 122 ~~~ |
123 | 123 |
124 The directories are nested first by sink type (--config), then by source type (-
-src). | 124 The directories are nested first by sink type (`--config`), then by source type
(`--src`). |
125 The image from the task we just looked at, "8888 image mandrill_132x132_12x12.as
tc-5-subsets", | 125 The image from the task we just looked at, "8888 image mandrill_132x132_12x12.as
tc-5-subsets", |
126 can be found at dm_output/8888/image/mandrill_132x132_12x12.astc-5-subsets.png. | 126 can be found at dm_output/8888/image/mandrill_132x132_12x12.astc-5-subsets.png. |
127 | 127 |
128 dm.json is used by our automated testing system, so you can ignore it if you | 128 dm.json is used by our automated testing system, so you can ignore it if you |
129 like. It contains a listing of each test run and a checksum of the image | 129 like. It contains a listing of each test run and a checksum of the image |
130 generated for that run. (Boring technical detail: it is not a checksum of the | 130 generated for that run. (Boring technical detail: it is not a checksum of the |
131 .png file, but rather a checksum of the raw pixels used to create that .png.) | 131 .png file, but rather a checksum of the raw pixels used to create that .png.) |
132 | 132 |
133 Unit tests don't generally output anything but a status update when they pass. | 133 Unit tests don't generally output anything but a status update when they pass. |
134 If a test fails, DM will print out its assertion failures, both at the time | 134 If a test fails, DM will print out its assertion failures, both at the time |
(...skipping 24 matching lines...) Expand all Loading... |
159 That's the basics of DM. DM supports many other modes and flags. Here are a | 159 That's the basics of DM. DM supports many other modes and flags. Here are a |
160 few examples you might find handy. | 160 few examples you might find handy. |
161 ~~~ | 161 ~~~ |
162 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. | 162 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. |
163 $ out/Debug/dm --src tests # Run only unit tests. | 163 $ out/Debug/dm --src tests # Run only unit tests. |
164 $ out/Debug/dm --nocpu # Test only GPU-backed work. | 164 $ out/Debug/dm --nocpu # Test only GPU-backed work. |
165 $ out/Debug/dm --nogpu # Test only CPU-backed work. | 165 $ out/Debug/dm --nogpu # Test only CPU-backed work. |
166 $ out/Debug/dm --match blur # Run only work with "blur" in its name. | 166 $ out/Debug/dm --match blur # Run only work with "blur" in its name. |
167 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. | 167 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. |
168 ~~~ | 168 ~~~ |
OLD | NEW |