Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(2459)

Unified Diff: Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py

Issue 1289163002: Improve flakiness logic for print-flaky-tests. (Closed) Base URL: svn://svn.chromium.org/blink/trunk
Patch Set: Created 5 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
Index: Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py
diff --git a/Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py b/Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py
index 90b47691056cc28d3292b95534837762bfee0469..8a675d8577d9c431ffa118eb7471ec5f1cdad4c9 100644
--- a/Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py
+++ b/Tools/Scripts/webkitpy/layout_tests/layout_package/bot_test_expectations_unittest.py
@@ -135,6 +135,12 @@ class BotTestExpectationsTest(unittest.TestCase):
'maybeflaky.html': self._results_from_string('FP'),
'notflakypass.html': self._results_from_string('P'),
'notflakyfail.html': self._results_from_string('F'),
+ # Even if there are no expected results, it's not very flaky if it didn't do mulitple retries.
+ # This accounts for the latest expectations not necessarily matching the expectations
+ # at the time of the given run.
+ 'notverflakynoexpected.html': self._results_from_string('FT'),
+ # If the test is flaky, but marked as such, it shouldn't get printed out.
+ 'notflakyexpected.html': {'results': [[2, 'FFFP']], 'expected': 'PASS FAIL'},
}
}
}
@@ -144,6 +150,7 @@ class BotTestExpectationsTest(unittest.TestCase):
self._assert_expectations(test_data, {
'foo/veryflaky.html': sorted(["TEXT", "PASS"]),
+ 'foo/notverflakynoexpected.html': ['TEXT', 'TIMEOUT'],
'foo/maybeflaky.html': sorted(["TEXT", "PASS"]),
}, only_ignore_very_flaky=False)

Powered by Google App Engine
This is Rietveld 408576698