Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(151)

Side by Side Diff: tools/testrunner/local/verbose.py

Issue 22381003: Add flaky test classification feature to test suites. (Closed) Base URL: https://v8.googlecode.com/svn/branches/bleeding_edge
Patch Set: Review Created 7 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « tools/testrunner/local/testsuite.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2012 the V8 project authors. All rights reserved. 1 # Copyright 2012 the V8 project authors. All rights reserved.
2 # Redistribution and use in source and binary forms, with or without 2 # Redistribution and use in source and binary forms, with or without
3 # modification, are permitted provided that the following conditions are 3 # modification, are permitted provided that the following conditions are
4 # met: 4 # met:
5 # 5 #
6 # * Redistributions of source code must retain the above copyright 6 # * Redistributions of source code must retain the above copyright
7 # notice, this list of conditions and the following disclaimer. 7 # notice, this list of conditions and the following disclaimer.
8 # * Redistributions in binary form must reproduce the above 8 # * Redistributions in binary form must reproduce the above
9 # copyright notice, this list of conditions and the following 9 # copyright notice, this list of conditions and the following
10 # disclaimer in the documentation and/or other materials provided 10 # disclaimer in the documentation and/or other materials provided
(...skipping 36 matching lines...) Expand 10 before | Expand all | Expand 10 after
47 skipped = timeout = nocrash = passes = fail_ok = fail = 0 47 skipped = timeout = nocrash = passes = fail_ok = fail = 0
48 for t in tests: 48 for t in tests:
49 if "outcomes" not in dir(t) or not t.outcomes: 49 if "outcomes" not in dir(t) or not t.outcomes:
50 passes += 1 50 passes += 1
51 continue 51 continue
52 o = t.outcomes 52 o = t.outcomes
53 if statusfile.DoSkip(o): 53 if statusfile.DoSkip(o):
54 skipped += 1 54 skipped += 1
55 continue 55 continue
56 if statusfile.TIMEOUT in o: timeout += 1 56 if statusfile.TIMEOUT in o: timeout += 1
57 if statusfile.IsFlaky(o): nocrash += 1 57 if statusfile.IsPassOrFail(o): nocrash += 1
58 if list(o) == [statusfile.PASS]: passes += 1 58 if list(o) == [statusfile.PASS]: passes += 1
59 if statusfile.IsFailOk(o): fail_ok += 1 59 if statusfile.IsFailOk(o): fail_ok += 1
60 if list(o) == [statusfile.FAIL]: fail += 1 60 if list(o) == [statusfile.FAIL]: fail += 1
61 print REPORT_TEMPLATE % { 61 print REPORT_TEMPLATE % {
62 "total": total, 62 "total": total,
63 "skipped": skipped, 63 "skipped": skipped,
64 "timeout": timeout, 64 "timeout": timeout,
65 "nocrash": nocrash, 65 "nocrash": nocrash,
66 "pass": passes, 66 "pass": passes,
67 "fail_ok": fail_ok, 67 "fail_ok": fail_ok,
(...skipping 22 matching lines...) Expand all
90 print 90 print
91 sys.stderr.write("--- Total time: %s ---\n" % FormatTime(overall_time)) 91 sys.stderr.write("--- Total time: %s ---\n" % FormatTime(overall_time))
92 timed_tests = [ t for s in suites for t in s.tests 92 timed_tests = [ t for s in suites for t in s.tests
93 if t.duration is not None ] 93 if t.duration is not None ]
94 timed_tests.sort(lambda a, b: cmp(b.duration, a.duration)) 94 timed_tests.sort(lambda a, b: cmp(b.duration, a.duration))
95 index = 1 95 index = 1
96 for entry in timed_tests[:20]: 96 for entry in timed_tests[:20]:
97 t = FormatTime(entry.duration) 97 t = FormatTime(entry.duration)
98 sys.stderr.write("%4i (%s) %s\n" % (index, t, entry.GetLabel())) 98 sys.stderr.write("%4i (%s) %s\n" % (index, t, entry.GetLabel()))
99 index += 1 99 index += 1
OLDNEW
« no previous file with comments | « tools/testrunner/local/testsuite.py ('k') | no next file » | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698