OLD | NEW |
1 Correctness Testing | 1 Correctness Testing |
2 =================== | 2 =================== |
3 | 3 |
4 Skia correctness testing is primarily served by a tool named DM. | 4 Skia correctness testing is primarily served by a tool named DM. |
5 This is a quickstart to building and running DM. | 5 This is a quickstart to building and running DM. |
6 | 6 |
7 ~~~ | 7 ~~~ |
8 $ ./gyp_skia | 8 $ python bin/sync-and-gyp |
9 $ ninja -C out/Debug dm | 9 $ ninja -C out/Debug dm |
10 $ out/Debug/dm -v -w dm_output | 10 $ out/Debug/dm -v -w dm_output |
11 ~~~ | 11 ~~~ |
12 | 12 |
13 When you run this, you may notice your CPU peg to 100% for a while, then taper | 13 When you run this, you may notice your CPU peg to 100% for a while, then taper |
14 off to 1 or 2 active cores as the run finishes. This is intentional. DM is | 14 off to 1 or 2 active cores as the run finishes. This is intentional. DM is |
15 very multithreaded, but some of the work, particularly GPU-backed work, is | 15 very multithreaded, but some of the work, particularly GPU-backed work, is |
16 still forced to run on a single thread. You can use `--threads N` to limit DM t
o | 16 still forced to run on a single thread. You can use `--threads N` to limit DM t
o |
17 N threads if you like. This can sometimes be helpful on machines that have | 17 N threads if you like. This can sometimes be helpful on machines that have |
18 relatively more CPU available than RAM. | 18 relatively more CPU available than RAM. |
(...skipping 116 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
135 That means it is possible for two different configurations to produce | 135 That means it is possible for two different configurations to produce |
136 the same exact .png, but have their checksums differ. | 136 the same exact .png, but have their checksums differ. |
137 | 137 |
138 Unit tests don't generally output anything but a status update when they pass. | 138 Unit tests don't generally output anything but a status update when they pass. |
139 If a test fails, DM will print out its assertion failures, both at the time | 139 If a test fails, DM will print out its assertion failures, both at the time |
140 they happen and then again all together after everything is done running. | 140 they happen and then again all together after everything is done running. |
141 These failures are also included in the dm.json file. | 141 These failures are also included in the dm.json file. |
142 | 142 |
143 DM has a simple facility to compare against the results of a previous run: | 143 DM has a simple facility to compare against the results of a previous run: |
144 ~~~ | 144 ~~~ |
145 $ ./gyp_skia | 145 $ python bin/sync-and-gyp |
146 $ ninja -C out/Debug dm | 146 $ ninja -C out/Debug dm |
147 $ out/Debug/dm -w good | 147 $ out/Debug/dm -w good |
148 | 148 |
149 (do some work) | 149 # do some work |
150 | 150 |
151 $ ./gyp_skia | 151 $ python bin/sync-and-gyp |
152 $ ninja -C out/Debug dm | 152 $ ninja -C out/Debug dm |
153 $ out/Debug/dm -r good -w bad | 153 $ out/Debug/dm -r good -w bad |
154 ~~~ | 154 ~~~ |
155 When using `-r`, DM will display a failure for any test that didn't produce the | 155 When using `-r`, DM will display a failure for any test that didn't produce the |
156 same image as the `good` run. | 156 same image as the `good` run. |
157 | 157 |
158 For anything fancier, I suggest using skdiff: | 158 For anything fancier, I suggest using skdiff: |
159 ~~~ | 159 ~~~ |
160 $ ./gyp_skia | 160 $ python bin/sync-and-gyp |
161 $ ninja -C out/Debug dm | 161 $ ninja -C out/Debug dm |
162 $ out/Debug/dm -w good | 162 $ out/Debug/dm -w good |
163 | 163 |
164 (do some work) | 164 # do some work |
165 | 165 |
166 $ ./gyp_skia | 166 $ python bin/sync-and-gyp |
167 $ ninja -C out/Debug dm | 167 $ ninja -C out/Debug dm |
168 $ out/Debug/dm -w bad | 168 $ out/Debug/dm -w bad |
169 | 169 |
170 $ ninja -C out/Debug skdiff | 170 $ ninja -C out/Debug skdiff |
171 $ mkdir diff | 171 $ mkdir diff |
172 $ out/Debug/skdiff good bad diff | 172 $ out/Debug/skdiff good bad diff |
173 | 173 |
174 (open diff/index.html in your web browser) | 174 # open diff/index.html in your web browser |
175 ~~~ | 175 ~~~ |
176 | 176 |
177 That's the basics of DM. DM supports many other modes and flags. Here are a | 177 That's the basics of DM. DM supports many other modes and flags. Here are a |
178 few examples you might find handy. | 178 few examples you might find handy. |
179 ~~~ | 179 ~~~ |
180 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. | 180 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl
anation of each. |
181 $ out/Debug/dm --src tests # Run only unit tests. | 181 $ out/Debug/dm --src tests # Run only unit tests. |
182 $ out/Debug/dm --nocpu # Test only GPU-backed work. | 182 $ out/Debug/dm --nocpu # Test only GPU-backed work. |
183 $ out/Debug/dm --nogpu # Test only CPU-backed work. | 183 $ out/Debug/dm --nogpu # Test only CPU-backed work. |
184 $ out/Debug/dm --match blur # Run only work with "blur" in its name. | 184 $ out/Debug/dm --match blur # Run only work with "blur" in its name. |
185 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. | 185 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we'
d do. |
186 ~~~ | 186 ~~~ |
OLD | NEW |