Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(217)

Side by Side Diff: Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py

Issue 109763007: Reland r159735 - support for test harness (text-only) tests w/o expected results. (Closed) Base URL: svn://svn.chromium.org/blink/trunk
Patch Set: update testharness-based baselines Created 7 years ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
OLDNEW
1 # Copyright (C) 2010 Google Inc. All rights reserved. 1 # Copyright (C) 2010 Google Inc. All rights reserved.
2 # Copyright (C) 2010 Gabor Rapcsanyi (rgabor@inf.u-szeged.hu), University of Sze ged 2 # Copyright (C) 2010 Gabor Rapcsanyi (rgabor@inf.u-szeged.hu), University of Sze ged
3 # 3 #
4 # Redistribution and use in source and binary forms, with or without 4 # Redistribution and use in source and binary forms, with or without
5 # modification, are permitted provided that the following conditions are 5 # modification, are permitted provided that the following conditions are
6 # met: 6 # met:
7 # 7 #
8 # * Redistributions of source code must retain the above copyright 8 # * Redistributions of source code must retain the above copyright
9 # notice, this list of conditions and the following disclaimer. 9 # notice, this list of conditions and the following disclaimer.
10 # * Redistributions in binary form must reproduce the above 10 # * Redistributions in binary form must reproduce the above
(...skipping 131 matching lines...) Expand 10 before | Expand all | Expand 10 after
142 # TestFailure type and test_expectations.EXPECTATION. 142 # TestFailure type and test_expectations.EXPECTATION.
143 if test_failures.FailureMissingAudio in failure_types: 143 if test_failures.FailureMissingAudio in failure_types:
144 test_dict['is_missing_audio'] = True 144 test_dict['is_missing_audio'] = True
145 145
146 if test_failures.FailureMissingResult in failure_types: 146 if test_failures.FailureMissingResult in failure_types:
147 test_dict['is_missing_text'] = True 147 test_dict['is_missing_text'] = True
148 148
149 if test_failures.FailureMissingImage in failure_types or test_failures.Failu reMissingImageHash in failure_types: 149 if test_failures.FailureMissingImage in failure_types or test_failures.Failu reMissingImageHash in failure_types:
150 test_dict['is_missing_image'] = True 150 test_dict['is_missing_image'] = True
151 151
152 if test_failures.FailureTestHarnessAssertion in failure_types:
153 test_dict['is_testharness_test'] = True
154
152 return test_dict 155 return test_dict
153 156
154 157
155 def summarize_results(port_obj, expectations, initial_results, retry_results, en abled_pixel_tests_in_retry, only_include_failing=False): 158 def summarize_results(port_obj, expectations, initial_results, retry_results, en abled_pixel_tests_in_retry, only_include_failing=False):
156 """Returns a dictionary containing a summary of the test runs, with the foll owing fields: 159 """Returns a dictionary containing a summary of the test runs, with the foll owing fields:
157 'version': a version indicator 160 'version': a version indicator
158 'fixable': The number of fixable tests (NOW - PASS) 161 'fixable': The number of fixable tests (NOW - PASS)
159 'skipped': The number of skipped tests (NOW & SKIPPED) 162 'skipped': The number of skipped tests (NOW & SKIPPED)
160 'num_regressions': The number of non-flaky failures 163 'num_regressions': The number of non-flaky failures
161 'num_flaky': The number of flaky failures 164 'num_flaky': The number of flaky failures
(...skipping 130 matching lines...) Expand 10 before | Expand all | Expand 10 after
292 except Exception, e: 295 except Exception, e:
293 _log.warn("Failed to determine svn revision for checkout (cwd: %s, webki t_base: %s), leaving 'revision' key blank in full_results.json.\n%s" % (port_obj ._filesystem.getcwd(), port_obj.path_from_webkit_base(), e)) 296 _log.warn("Failed to determine svn revision for checkout (cwd: %s, webki t_base: %s), leaving 'revision' key blank in full_results.json.\n%s" % (port_obj ._filesystem.getcwd(), port_obj.path_from_webkit_base(), e))
294 # Handle cases where we're running outside of version control. 297 # Handle cases where we're running outside of version control.
295 import traceback 298 import traceback
296 _log.debug('Failed to learn head svn revision:') 299 _log.debug('Failed to learn head svn revision:')
297 _log.debug(traceback.format_exc()) 300 _log.debug(traceback.format_exc())
298 results['chromium_revision'] = "" 301 results['chromium_revision'] = ""
299 results['blink_revision'] = "" 302 results['blink_revision'] = ""
300 303
301 return results 304 return results
OLDNEW
« no previous file with comments | « Tools/Scripts/webkitpy/layout_tests/models/test_failures.py ('k') | Tools/Scripts/webkitpy/layout_tests/port/test.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698