rebaseline.py: if expectations dir contains JSON format results, update those instead of image files

gm/gm_json.py

Index: gm/gm_json.py
===================================================================
--- gm/gm_json.py	(revision 9897)
+++ gm/gm_json.py	(working copy)
@@ -15,18 +15,37 @@
 import json
 
 
+# Key strings used in GM results JSON files (both expected-results.json and
+# actual-results.json).
+#
 # These constants must be kept in sync with the kJsonKey_ constants in
 # gm_expectations.cpp !
+
 JSONKEY_ACTUALRESULTS = 'actual-results'
+# Tests whose results failed to match expectations.
 JSONKEY_ACTUALRESULTS_FAILED = 'failed'
+# Tests whose results failed to match expectations, but IGNOREFAILURE causes
+# us to take them less seriously.
 JSONKEY_ACTUALRESULTS_FAILUREIGNORED = 'failure-ignored'
+# Tests for which we do not have any expectations.  They may be new tests that
+# we haven't had a chance to check in expectations for yet, or we may have
+# consciously decided to leave them without expectations because we are unhappy
+# with the results (although we should try to move away from that, and instead
+# check in expectations with the IGNOREFAILURE flag set).
 JSONKEY_ACTUALRESULTS_NOCOMPARISON = 'no-comparison'
+# Tests whose results matched their expectations.
 JSONKEY_ACTUALRESULTS_SUCCEEDED = 'succeeded'
 
 JSONKEY_EXPECTEDRESULTS = 'expected-results'
+# One or more [HashType/DigestValue] pairs representing valid results for this
+# test.  Typically, there will just be one pair, but we allow for multiple
+# expectations, and the test will pass if any one of them is matched.
 JSONKEY_EXPECTEDRESULTS_ALLOWEDDIGESTS = 'allowed-digests'
+# If the results of this test don't match expectations, should it be reported
+# in the FAILED section or the FAILUREIGNORED section?

[Stephen White, 2013/07/08 17:15:20]

Just to be braindead-clear, perhaps this should say 'If marked as
"ignore-failure", a test failure will be reported under the FAILUREINGORED
section instead of the FAILED section." (If this will will also cause the bot
not to go red, perhaps that should be documented here too?)

[epoger, 2013/07/08 17:52:29]

Good idea. Done.

 JSONKEY_EXPECTEDRESULTS_IGNOREFAILURE = 'ignore-failure'
 
+# Allowed hash types for test expectations.
 JSONKEY_HASHTYPE_BITMAP_64BITMD5 = 'bitmap-64bitMD5'
 
 def LoadFromString(file_contents):
@@ -45,3 +64,8 @@
      above."""
   file_contents = open(file_path, 'r').read()
   return LoadFromString(file_contents)
+
+def WriteToFile(json_dict, file_path):
+  """Writes the JSON summary in json_dict out to file_path."""
+  with open(file_path, 'w') as outfile:
+    json.dump(json_dict, outfile, sort_keys=True, indent=2)