Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(49)

Side by Side Diff: gm/tests/run.sh

Issue 26294004: GM: Allow ignored-tests.txt to list configs as well as tests (and combinations too) (Closed) Base URL: http://skia.googlecode.com/svn/trunk/
Patch Set: clarify Created 7 years, 2 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « gm/tests/outputs/ignoring-one-test/output-expected/json-summary.txt ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/bin/bash 1 #!/bin/bash
2 2
3 # Self-tests for gm, based on tools/tests/run.sh 3 # Self-tests for gm, based on tools/tests/run.sh
4 # 4 #
5 # These tests are run by the Skia_PerCommit_House_Keeping bot at every commit, 5 # These tests are run by the Skia_PerCommit_House_Keeping bot at every commit,
6 # so make sure that they still pass when you make changes to gm! 6 # so make sure that they still pass when you make changes to gm!
7 # 7 #
8 # To generate new baselines when gm behavior changes, run gm/tests/rebaseline.sh 8 # To generate new baselines when gm behavior changes, run gm/tests/rebaseline.sh
9 # 9 #
10 # TODO: because this is written as a shell script (instead of, say, Python) 10 # TODO: because this is written as a shell script (instead of, say, Python)
(...skipping 184 matching lines...) Expand 10 before | Expand all | Expand 10 after
195 195
196 # Compare generated image against an input image file with identical pixels but different PNG encoding. 196 # Compare generated image against an input image file with identical pixels but different PNG encoding.
197 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/i dentical-pixels" "$GM_OUTPUTS/compared-against-identical-pixels-images" 197 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/i dentical-pixels" "$GM_OUTPUTS/compared-against-identical-pixels-images"
198 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/ide ntical-pixels.json" "$GM_OUTPUTS/compared-against-identical-pixels-json" 198 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/ide ntical-pixels.json" "$GM_OUTPUTS/compared-against-identical-pixels-json"
199 199
200 # Compare generated image against an input image file with different pixels. 200 # Compare generated image against an input image file with different pixels.
201 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/d ifferent-pixels" "$GM_OUTPUTS/compared-against-different-pixels-images" 201 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/d ifferent-pixels" "$GM_OUTPUTS/compared-against-different-pixels-images"
202 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/dif ferent-pixels.json" "$GM_OUTPUTS/compared-against-different-pixels-json" 202 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/dif ferent-pixels.json" "$GM_OUTPUTS/compared-against-different-pixels-json"
203 203
204 # Exercise --ignoreTests flag. 204 # Exercise --ignoreTests flag.
205 gm_test "--verbose --hierarchy --match selftest1 --ignoreTests selftest1 $CONFIG S -r $GM_INPUTS/json/different-pixels.json" "$GM_OUTPUTS/ignoring-one-test" 205 gm_test "--verbose --hierarchy --match selftest1 --ignoreTests 8888/selfte $CONF IGS -r $GM_INPUTS/json/different-pixels.json" "$GM_OUTPUTS/ignoring-one-test"
206 206
207 # Compare different pixels, but with a SUBSET of the expectations marked as 207 # Compare different pixels, but with a SUBSET of the expectations marked as
208 # ignore-failure. 208 # ignore-failure.
209 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/dif ferent-pixels-ignore-some-failures.json" "$GM_OUTPUTS/ignoring-some-failures" 209 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/json/dif ferent-pixels-ignore-some-failures.json" "$GM_OUTPUTS/ignoring-some-failures"
210 210
211 # Compare generated image against an empty "expected image" dir. 211 # Compare generated image against an empty "expected image" dir.
212 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/e mpty-dir" "$GM_OUTPUTS/compared-against-empty-dir" 212 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r $GM_INPUTS/images/e mpty-dir" "$GM_OUTPUTS/compared-against-empty-dir"
213 213
214 # Compare generated image against a nonexistent "expected image" dir. 214 # Compare generated image against a nonexistent "expected image" dir.
215 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r ../path/to/nowhere" "$GM_OUTPUTS/compared-against-nonexistent-dir" 215 gm_test "--verbose --hierarchy --match selftest1 $CONFIGS -r ../path/to/nowhere" "$GM_OUTPUTS/compared-against-nonexistent-dir"
(...skipping 33 matching lines...) Expand 10 before | Expand all | Expand 10 after
249 for CASE in $FAILING_CASES; do 249 for CASE in $FAILING_CASES; do
250 assert_fails "python gm/display_json_results.py $GM_OUTPUTS/$CASE/$OUTPUT_EXPE CTED_SUBDIR/json-summary.txt" 250 assert_fails "python gm/display_json_results.py $GM_OUTPUTS/$CASE/$OUTPUT_EXPE CTED_SUBDIR/json-summary.txt"
251 done 251 done
252 252
253 if [ $ENCOUNTERED_ANY_ERRORS == 0 ]; then 253 if [ $ENCOUNTERED_ANY_ERRORS == 0 ]; then
254 echo "All tests passed." 254 echo "All tests passed."
255 exit 0 255 exit 0
256 else 256 else
257 exit 1 257 exit 1
258 fi 258 fi
OLDNEW
« no previous file with comments | « gm/tests/outputs/ignoring-one-test/output-expected/json-summary.txt ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698