GM: stop recording all *_pdf.png results as "INVALID" on non-Mac platforms

GM: stop recording all *_pdf.png results as "INVALID" on non-Mac platforms

BUG=https://code.google.com/p/skia/issues/detail?id=1352
R=edisonn@google.com

Committed: https://code.google.com/p/skia/source/detail?r=9912

[epoger, 2013-06-18 04:22:34]

https://codereview.chromium.org/17366003/diff/1/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/17366003/diff/1/gm/tests/run.sh#newcode203
gm/tests/run.sh:203: gm_test "--verbose --hierarchy --match selftest1 $CONFIGS
-r $GM_INPUTS/json/identical-bytes.json" "$GM_OUTPUTS/add-config-pdf"
Patchset 1 adds a new "add-config-pdf" self-test, but this self-test doesn't
actually add pdf to the --configs argument yet.  That will be next...

[epoger, 2013-06-18 04:32:27]

https://codereview.chromium.org/17366003/diff/8001/gm/tests/outputs/add-confi...
File gm/tests/outputs/add-config-pdf/output-expected/command_line (right):

https://codereview.chromium.org/17366003/diff/8001/gm/tests/outputs/add-confi...
gm/tests/outputs/add-config-pdf/output-expected/command_line:1: out/Debug/gm
--verbose --hierarchy --match selftest1 --config 8888 565 pdf -r
gm/tests/inputs/json/identical-bytes.json --writeJsonSummaryPath
gm/tests/outputs/add-config-pdf/output-actual/json-summary.txt --writePath
gm/tests/outputs/add-config-pdf/output-actual/writePath --mismatchPath
gm/tests/outputs/add-config-pdf/output-actual/mismatchPath
In patchset 2, we make the new self-test actually add "pdf" to the list of
configs.

https://codereview.chromium.org/17366003/diff/8001/gm/tests/outputs/add-confi...
File gm/tests/outputs/add-config-pdf/output-expected/json-summary.txt (right):

https://codereview.chromium.org/17366003/diff/8001/gm/tests/outputs/add-confi...
gm/tests/outputs/add-config-pdf/output-expected/json-summary.txt:6:
"pdf/selftest1.png" : [ "INVALID" ]
... and here we see the "INVALID" line!

I think what's happening here is that the JSON-writing code in gm is attempting
to get a BitmapHash of the PDF output.  (It shouldn't be doing that, because
it's not a bitmap.)

https://codereview.chromium.org/17366003/diff/8001/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/17366003/diff/8001/gm/tests/run.sh#newcode111
gm/tests/run.sh:111: for IMAGEFILE in $(find $ACTUAL_OUTPUT_DIR -name *.pdf); do
I had to add this special treatment for the PDF files; otherwise, I run into the
same upload.py problem as mentioned here.

[epoger, 2013-06-18 05:20:10]

Ready for review at patchset 3.

https://codereview.chromium.org/17366003/diff/15003/gm/gmmain.cpp
File gm/gmmain.cpp (right):

https://codereview.chromium.org/17366003/diff/15003/gm/gmmain.cpp#newcode886
gm/gmmain.cpp:886:
RecordTestResults(ErrorCombination(kIntentionallySkipped_ErrorType),
Patchset 3 gives us better behavior... when rendering PDF tests on a platform
that doesn't support generation of SkBitmaps from PDFs for comparison, just note
that we can't validate this particular output mode.

https://codereview.chromium.org/17366003/diff/15003/gm/gmmain.cpp#newcode890
gm/gmmain.cpp:890: if (expectationsSource && (gRec.fFlags & kRead_ConfigFlag)) {
This section is just moved down and tab-indented.

https://codereview.chromium.org/17366003/diff/15003/gm/tests/outputs/add-conf...
File gm/tests/outputs/add-config-pdf/output-expected/json-summary.txt (right):

https://codereview.chromium.org/17366003/diff/15003/gm/tests/outputs/add-conf...
gm/tests/outputs/add-config-pdf/output-expected/json-summary.txt:5:
"no-comparison" : null,
The JSON summary looks better now.  No INVALID, just no record of the pdf
rendering (which is fine, because we can't compare PDFs on this platform)

https://codereview.chromium.org/17366003/diff/15003/gm/tests/outputs/add-conf...
File gm/tests/outputs/add-config-pdf/output-expected/stdout (right):

https://codereview.chromium.org/17366003/diff/15003/gm/tests/outputs/add-conf...
gm/tests/outputs/add-config-pdf/output-expected/stdout:12: GM: [ ] 1
IntentionallySkipped: pdf/selftest1
Even if we HAD image expectations for the PDF version, we wouldn't have been
able to compare them (because this platform doesn't know how to convert PDFs to
SkBitmaps).  So IntentionallySkipped makes more sense than MissingExpectations.

[epoger, 2013-06-18 05:26:10]

P.S. Have I mentioned lately that I wish our PDF generator yielded deterministic
output, so that....

1. We could watch for PDF regressions, even on platforms for which we can't
generate SkBitmaps from PDFs
2. Even on platforms where we *do* have that ability, we could skip that
time-consuming PDF-to-SkBitmap translation in the 99% of runs where the PDF file
didn't change by a single bit

https://code.google.com/p/skia/issues/detail?id=1277 ('pdf output includes
nondeterministic contents')

[epoger, 2013-06-21 13:33:25]

ping?

[edisonn, 2013-06-21 14:04:26]

LGTM

https://codereview.chromium.org/17366003/diff/15003/gm/tests/run.sh
File gm/tests/run.sh (right):

https://codereview.chromium.org/17366003/diff/15003/gm/tests/run.sh#newcode113
gm/tests/run.sh:113: done
Just a general comment: The thing that I am worried here, is that this kind of
pattern (rewriting content of a file) forces us to replay the whole test, and we
can't simply retry a certain step, we must rewind back from setup. But I don't
know how much valuable it is in practice, through.

[epoger, 2013-07-08 19:13:39]

Committed patchset #5 manually as r9912 (presubmit successful).