gm self-test: make sure we report failures in individual rendering modes

gm/tests/outputs/pipe-playback-failure/output-expected/json-summary.txt

 {
    "actual-results" : {
       "failed" : null,
       "failure-ignored" : null,
       "no-comparison" : null,
       "succeeded" : {

[epoger, 2013/03/14 17:22:19]

Hmm, note that the error in pipe-playback mode is NOT reported in the JSON
summary!  I think this means that our current plan in
https://goto.google.com/ChecksumRemainingSteps won't quite work.

Specifically, our idea that we will run gm in a --noFailOnMismatch mode, and
then examine the JSON summary in a later step to see if there were any failures.

How should we remedy that?  Maybe we should include all rendering modes in the
JSON summary?  Although that would make the file a lot bigger...

[epoger, 2013/03/14 18:16:21]

Here's a proposal:

- If rendering the same gm test in two different modes yields different bitmaps,
that will ALWAYS be treated as an error in gm.  (As if gm had thrown a
segfault... it should NEVER happen, and if it does, the JSON summary should say
in effect "we suffered massive failure while trying to run these tests, so we do
not have any results to report")

- Thus, appearance of a test in the "actual-results" portion of the JSON summary
implies that *all* rendering modes generated *exactly* the same results.

- Thus, we don't need to list the actual checksum of every rendering mode... we
list a single actual checksum, and readers know that it was the checksum of
every mode of that test.

[borenet, 2013/03/14 18:48:21]

I agree with all of this (it's pretty much what we already do), except I think
we should just leave the failed test out of the json file, as if it wasn't
generated.

[epoger, 2013/03/14 18:57:12]

In a way it seems like there would be value in the broken test showing up in the
"failed" portion of "actual-results"... but for now, I think it's reasonable to
just leave it out of the JSON summary and rely on the caller seeing the failure
while running gm...

I'll work on a change that does exactly that.

          "565/selftest1" : {
             "checksum" : 9512553915271796906
          },
          "8888/selftest1" : {
             "checksum" : 14022967492765711532
          }
       }
    },
    "expected-results" : {
       "565/selftest1" : {
          "checksums" : [ 9512553915271796906 ],
          "ignore-failure" : false
       },
       "8888/selftest1" : {
          "checksums" : [ 14022967492765711532 ],
          "ignore-failure" : false
       }
    }
 }