Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(142)

Side by Side Diff: Tools/Scripts/webkitpy/layout_tests/models/test_failures.py

Issue 109763007: Reland r159735 - support for test harness (text-only) tests w/o expected results. (Closed) Base URL: svn://svn.chromium.org/blink/trunk
Patch Set: update testharness-based baselines Created 7 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
1 # Copyright (C) 2010 Google Inc. All rights reserved. 1 # Copyright (C) 2010 Google Inc. All rights reserved.
2 # 2 #
3 # Redistribution and use in source and binary forms, with or without 3 # Redistribution and use in source and binary forms, with or without
4 # modification, are permitted provided that the following conditions are 4 # modification, are permitted provided that the following conditions are
5 # met: 5 # met:
6 # 6 #
7 # * Redistributions of source code must retain the above copyright 7 # * Redistributions of source code must retain the above copyright
8 # notice, this list of conditions and the following disclaimer. 8 # notice, this list of conditions and the following disclaimer.
9 # * Redistributions in binary form must reproduce the above 9 # * Redistributions in binary form must reproduce the above
10 # copyright notice, this list of conditions and the following disclaimer 10 # copyright notice, this list of conditions and the following disclaimer
(...skipping 43 matching lines...) Expand 10 before | Expand all | Expand 10 after
54 elif FailureTimeout in failure_types: 54 elif FailureTimeout in failure_types:
55 return test_expectations.TIMEOUT 55 return test_expectations.TIMEOUT
56 elif FailureEarlyExit in failure_types: 56 elif FailureEarlyExit in failure_types:
57 return test_expectations.SKIP 57 return test_expectations.SKIP
58 elif (FailureMissingResult in failure_types or 58 elif (FailureMissingResult in failure_types or
59 FailureMissingImage in failure_types or 59 FailureMissingImage in failure_types or
60 FailureMissingImageHash in failure_types or 60 FailureMissingImageHash in failure_types or
61 FailureMissingAudio in failure_types): 61 FailureMissingAudio in failure_types):
62 return test_expectations.MISSING 62 return test_expectations.MISSING
63 else: 63 else:
64 is_text_failure = FailureTextMismatch in failure_types 64 is_text_failure = (FailureTextMismatch in failure_types or
65 FailureTestHarnessAssertion in failure_types)
65 is_image_failure = (FailureImageHashIncorrect in failure_types or 66 is_image_failure = (FailureImageHashIncorrect in failure_types or
66 FailureImageHashMismatch in failure_types) 67 FailureImageHashMismatch in failure_types)
67 is_audio_failure = (FailureAudioMismatch in failure_types) 68 is_audio_failure = (FailureAudioMismatch in failure_types)
68 if is_text_failure and is_image_failure: 69 if is_text_failure and is_image_failure:
69 return test_expectations.IMAGE_PLUS_TEXT 70 return test_expectations.IMAGE_PLUS_TEXT
70 elif is_text_failure: 71 elif is_text_failure:
71 return test_expectations.TEXT 72 return test_expectations.TEXT
72 elif is_image_failure or is_reftest_failure(failure_list): 73 elif is_image_failure or is_reftest_failure(failure_list):
73 return test_expectations.IMAGE 74 return test_expectations.IMAGE
74 elif is_audio_failure: 75 elif is_audio_failure:
(...skipping 59 matching lines...) Expand 10 before | Expand all | Expand 10 after
134 135
135 def driver_needs_restart(self): 136 def driver_needs_restart(self):
136 return True 137 return True
137 138
138 139
139 class FailureMissingResult(TestFailure): 140 class FailureMissingResult(TestFailure):
140 def message(self): 141 def message(self):
141 return "-expected.txt was missing" 142 return "-expected.txt was missing"
142 143
143 144
145 class FailureTestHarnessAssertion(TestFailure):
146 def message(self):
147 return "asserts failed"
148
149
144 class FailureTextMismatch(TestFailure): 150 class FailureTextMismatch(TestFailure):
145 def message(self): 151 def message(self):
146 return "text diff" 152 return "text diff"
147 153
154
148 class FailureMissingImageHash(TestFailure): 155 class FailureMissingImageHash(TestFailure):
149 def message(self): 156 def message(self):
150 return "-expected.png was missing an embedded checksum" 157 return "-expected.png was missing an embedded checksum"
151 158
152 159
153 class FailureMissingImage(TestFailure): 160 class FailureMissingImage(TestFailure):
154 def message(self): 161 def message(self):
155 return "-expected.png was missing" 162 return "-expected.png was missing"
156 163
157 164
(...skipping 45 matching lines...) Expand 10 before | Expand all | Expand 10 after
203 210
204 211
205 class FailureEarlyExit(TestFailure): 212 class FailureEarlyExit(TestFailure):
206 def message(self): 213 def message(self):
207 return "skipped due to early exit" 214 return "skipped due to early exit"
208 215
209 216
210 # Convenient collection of all failure classes for anything that might 217 # Convenient collection of all failure classes for anything that might
211 # need to enumerate over them all. 218 # need to enumerate over them all.
212 ALL_FAILURE_CLASSES = (FailureTimeout, FailureCrash, FailureMissingResult, 219 ALL_FAILURE_CLASSES = (FailureTimeout, FailureCrash, FailureMissingResult,
220 FailureTestHarnessAssertion,
213 FailureTextMismatch, FailureMissingImageHash, 221 FailureTextMismatch, FailureMissingImageHash,
214 FailureMissingImage, FailureImageHashMismatch, 222 FailureMissingImage, FailureImageHashMismatch,
215 FailureImageHashIncorrect, FailureReftestMismatch, 223 FailureImageHashIncorrect, FailureReftestMismatch,
216 FailureReftestMismatchDidNotOccur, FailureReftestNoImages Generated, 224 FailureReftestMismatchDidNotOccur, FailureReftestNoImages Generated,
217 FailureMissingAudio, FailureAudioMismatch, 225 FailureMissingAudio, FailureAudioMismatch,
218 FailureEarlyExit) 226 FailureEarlyExit)
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698