gm/tests/outputs/compared-against-different-pixels-images/output-expected/stdout - Issue 12851022: gm: display summary of all results by ErrorType

Side by Side Diff: gm/tests/outputs/compared-against-different-pixels-images/output-expected/stdout

Issue 12851022: gm: display summary of all results by ErrorType (Closed) Base URL: http://skia.googlecode.com/svn/trunk/

Patch Set: sync_to_r8399 Created 7 years, 8 months ago

Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.

Jump to:

View unified diff | Download patch | Annotate | Revision Log

« gm/gmmain.cpp ('K') | « gm/tests/outputs/compared-against-different-pixels-images/output-expected/stderr ('k') | gm/tests/outputs/compared-against-different-pixels-json/output-expected/stderr » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Hide Comments ('s')

OLD	NEW
1 GM: reading from gm/tests/inputs/images/different-pixels	1 GM: reading from gm/tests/inputs/images/different-pixels

2 GM: drawing... selftest1 [300 200]	2 GM: drawing... selftest1 [300 200]

3 GM: Ran 1 tests: 0 passed, 1 failed, 0 missing reference images	3 GM: Ran 1 GMs, each with 2 configs ["8888", "565"] and 7 modes ["replay", "seria lize", "rtree", "tileGrid1.000000", "pipe", "pipe cross-process", "pipe cross-pr ocess, shared address"], so there should be a total of 9 tests
	borenet 2013/03/29 05:55:14 This is very wordy... This is very wordy... epoger 2013/04/02 16:00:14 Agreed. What do you think it should look like? S Show quoted text On 2013/03/29 05:55:14, borenet wrote: > This is very wordy... Agreed. What do you think it should look like? Should we rearrange the information, or leave some of it out? If you tell me what you think this output should look like, I'll make it happen in this CL. borenet 2013/04/02 16:17:56 I don't know how helpful the above information is. Show quoted text On 2013/04/02 16:00:14, epoger wrote: > On 2013/03/29 05:55:14, borenet wrote: > > This is very wordy... > > Agreed. What do you think it should look like? Should we rearrange the > information, or leave some of it out? > > If you tell me what you think this output should look like, I'll make it happen > in this CL. I don't know how helpful the above information is. I guess it's primarily for the self-test? It might help just to add some line breaks: GM: Ran 1 GMs GM: ... over 2 configs ["8888", "565"] GM: ... and 7 modes [...] GM: ... for a total of 9 tests. What is the value of the following line ("Ran 9 tests...")? It seems that the "9 tests" is given above, and the falures for each type is given below. epoger 2013/04/03 04:18:23 Actually, no... I wanted to make it easier to noti Show quoted text On 2013/04/02 16:17:56, borenet wrote: > I don't know how helpful the above information is. I guess it's primarily for > the self-test? Actually, no... I wanted to make it easier to notice problems such as https://code.google.com/p/skia/issues/detail?id=1180 ('image-surface gm test is failing in "deferred" mode, and gm is not reporting the failure'): types of tests that the caller ASSUMES are being run by gm, but may not be! (Looking at this output, you can now tell that gm is not running in deferred mode, which is a bug as noted above!) In patchset 6, I split up this single summary line as you suggested. Personally, I prefer it all in one line (so it's easier to filter it with grep), but I'm fine either way. Show quoted text > What is the value of the following line ("Ran 9 tests...")? It seems that the > "9 tests" is given above, and the falures for each type is given below. To help catch another type of bug: a particular test which runs for MOST gms but not ALL. See "Ultimately, we should signal an error" TODO in gmmain.cpp.
	4 GM: Ran 9 tests: NoGpuContext=0 ImageMismatch=2 MissingExpectations=0 WritingRef erenceImage=0

	5 GM: [*] 0 NoGpuContext:

	6 GM: [*] 2 ImageMismatch: 8888/selftest1 565/selftest1

	7 GM: [ ] 0 MissingExpectations:

	8 GM: [*] 0 WritingReferenceImage:

	9 GM: (results marked with [*] will cause nonzero return value)

OLD	NEW