Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(2108)

Unified Diff: Tools/Scripts/webkitpy/tool/commands/flakytests.py

Issue 1289163002: Improve flakiness logic for print-flaky-tests. (Closed) Base URL: svn://svn.chromium.org/blink/trunk
Patch Set: Created 5 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View side-by-side diff with in-line comments
Download patch
Index: Tools/Scripts/webkitpy/tool/commands/flakytests.py
diff --git a/Tools/Scripts/webkitpy/tool/commands/flakytests.py b/Tools/Scripts/webkitpy/tool/commands/flakytests.py
index 24835ebd4ff00e04925b2ea27e9fc8e491936758..f7e8a700d8ffb8c18cc7db5f00ca9a6384d60f91 100644
--- a/Tools/Scripts/webkitpy/tool/commands/flakytests.py
+++ b/Tools/Scripts/webkitpy/tool/commands/flakytests.py
@@ -40,10 +40,22 @@ class FlakyTests(AbstractDeclarativeCommand):
help_text = "Print out flaky lines from the flakiness dashboard"
show_in_main_help = True
- FLAKY_TEST_CONTENTS = '''%s
+ FLAKINESS_DASHBOARD_URL = 'https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=%s'
-Manually add bug numbers for these and then put the lines in LayoutTests/TestExpectations.
-TODO(ojan): Write a script to file/assign the bugs then create a bot to do this automatically.
+ BUG_TEMPLATE = 'https://code.google.com/p/chromium/issues/entry?owner=FILL_ME_IN&status=Assigned&labels=Pri-1,Cr-Blink,FlakyLayoutTest&summary=XXXXXXX%20is%20flaky&comment=XXXXXXX%20is%20flaky.%0A%0AIt%20failed%20twice%20and%20then%20passed%20on%20the%203rd%20or%204th%20retry.%20This%20is%20too%20flaky.%20The%20test%20will%20be%20skipped%20until%20it%27s%20fixed.%20If%20not%20fixed%20in%203%20months,%20it%20will%20be%20deleted%20or%20perma-skipped.%0A%0AIn%20the%20flakiness%20dashboard,%20the%20turquoise%20boxes%20are%20runs%20where%20the%20test%20failed%20and%20then%20passed%20on%20retry.%0A%0Ahttp://test-results.appspot.com/dashboards/flakiness_dashboard.html%23tests=XXXXXXX'
+
+ HEADER = '''Manually add bug numbers for these and then put the lines in LayoutTests/TestExpectations.
+Look up the test in the flakiness dashboard first to see if the the platform
+specifiers should be made more general.
+
+Bug template:
+%s
+''' % BUG_TEMPLATE
+
+ OUTPUT = '''%s
+%s
+
+Flakiness dashboard: %s
'''
def __init__(self):
@@ -93,5 +105,10 @@ TODO(ojan): Write a script to file/assign the bugs then create a bot to do this
fs = tool.filesystem
lines = filter(lambda line: fs.exists(fs.join(port.layout_tests_dir(), line.path)), lines)
- print self.FLAKY_TEST_CONTENTS % TestExpectations.list_to_string(lines) # pylint: disable=E1601
+ test_names = [line.name for line in lines]
+ flakiness_dashbord_url = self.FLAKINESS_DASHBOARD_URL % ','.join(test_names)
+ expectations_string = TestExpectations.list_to_string(lines)
+
+ # pylint: disable=E1601
+ print self.OUTPUT % (self.HEADER, expectations_string, flakiness_dashbord_url)

Powered by Google App Engine
This is Rietveld 408576698