Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(62)

Side by Side Diff: site/dev/contrib/testing.md

Issue 923543002: DM: document -r (Closed) Base URL: https://skia.googlesource.com/skia.git@master
Patch Set: Created 5 years, 10 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 Correctness Testing 1 Correctness Testing
2 =================== 2 ===================
3 3
4 Skia correctness testing is primarily served by a tool named DM. 4 Skia correctness testing is primarily served by a tool named DM.
5 This is a quickstart to building and running DM. 5 This is a quickstart to building and running DM.
6 6
7 ~~~ 7 ~~~
8 $ ./gyp_skia 8 $ ./gyp_skia
9 $ ninja -C out/Debug dm 9 $ ninja -C out/Debug dm
10 $ out/Debug/dm -v -w dm_output 10 $ out/Debug/dm -v -w dm_output
(...skipping 117 matching lines...) Expand 10 before | Expand all | Expand 10 after
128 dm.json is used by our automated testing system, so you can ignore it if you 128 dm.json is used by our automated testing system, so you can ignore it if you
129 like. It contains a listing of each test run and a checksum of the image 129 like. It contains a listing of each test run and a checksum of the image
130 generated for that run. (Boring technical detail: it is not a checksum of the 130 generated for that run. (Boring technical detail: it is not a checksum of the
131 .png file, but rather a checksum of the raw pixels used to create that .png.) 131 .png file, but rather a checksum of the raw pixels used to create that .png.)
132 132
133 Unit tests don't generally output anything but a status update when they pass. 133 Unit tests don't generally output anything but a status update when they pass.
134 If a test fails, DM will print out its assertion failures, both at the time 134 If a test fails, DM will print out its assertion failures, both at the time
135 they happen and then again all together after everything is done running. 135 they happen and then again all together after everything is done running.
136 These failures are also included in the dm.json file. 136 These failures are also included in the dm.json file.
137 137
138 DM does not currently have a built-in image comparison facility, though we 138 DM has a simple facility to compare against the results of a previous run:
139 might add one. For now, you can use the skdiff tool to compare corresponding
140 images from two runs of DM.
141 ~~~ 139 ~~~
142 $ ./gyp_skia 140 $ ./gyp_skia
143 $ ninja -C out/Debug dm 141 $ ninja -C out/Debug dm
142 $ out/Debug/dm -w good
143
144 (do some work)
145
146 $ ./gyp_skia
147 $ ninja -C out/Debug dm
148 $ out/Debug/dm -r good -w bad
149 ~~~
150 When using `-r`, DM will display a failure for any test that didn't produce the
151 same image as the `good` run.
152
153 For anything fancier, I suggest using skdiff:
154 ~~~
155 $ ./gyp_skia
156 $ ninja -C out/Debug dm
144 $ out/Debug/dm -w good 157 $ out/Debug/dm -w good
145 158
146 (do some work) 159 (do some work)
147 160
148 $ ./gyp_skia 161 $ ./gyp_skia
149 $ ninja -C out/Debug dm 162 $ ninja -C out/Debug dm
150 $ out/Debug/dm -w bad 163 $ out/Debug/dm -w bad
151 164
152 $ ninja -C out/Debug skdiff 165 $ ninja -C out/Debug skdiff
153 $ mkdir diff 166 $ mkdir diff
154 $ out/Debug/skdiff good bad diff 167 $ out/Debug/skdiff good bad diff
155 168
156 (open diff/index.html in your web browser) 169 (open diff/index.html in your web browser)
157 ~~~ 170 ~~~
158 171
159 That's the basics of DM. DM supports many other modes and flags. Here are a 172 That's the basics of DM. DM supports many other modes and flags. Here are a
160 few examples you might find handy. 173 few examples you might find handy.
161 ~~~ 174 ~~~
162 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl anation of each. 175 $ out/Debug/dm --help # Print all flags, their defaults, and a brief expl anation of each.
163 $ out/Debug/dm --src tests # Run only unit tests. 176 $ out/Debug/dm --src tests # Run only unit tests.
164 $ out/Debug/dm --nocpu # Test only GPU-backed work. 177 $ out/Debug/dm --nocpu # Test only GPU-backed work.
165 $ out/Debug/dm --nogpu # Test only CPU-backed work. 178 $ out/Debug/dm --nogpu # Test only CPU-backed work.
166 $ out/Debug/dm --match blur # Run only work with "blur" in its name. 179 $ out/Debug/dm --match blur # Run only work with "blur" in its name.
167 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we' d do. 180 $ out/Debug/dm --dryRun # Don't really do anything, just print out what we' d do.
168 ~~~ 181 ~~~
OLDNEW
« no previous file with comments | « no previous file | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698