Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(151)

Side by Side Diff: tools/bisect-perf-regression.py

Issue 288843002: Fix bisect script to match a results with {mean, stddev} format. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/src
Patch Set: Created 6 years, 7 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « no previous file | tools/bisect-perf-regression_test.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright (c) 2013 The Chromium Authors. All rights reserved. 2 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Performance Test Bisect Tool 6 """Performance Test Bisect Tool
7 7
8 This script bisects a series of changelists using binary search. It starts at 8 This script bisects a series of changelists using binary search. It starts at
9 a bad revision where a performance metric has regressed, and asks for a last 9 a bad revision where a performance metric has regressed, and asks for a last
10 known-good revision. It will then binary search across this revision range by 10 known-good revision. It will then binary search across this revision range by
(...skipping 1927 matching lines...) Expand 10 before | Expand all | Expand 10 after
1938 """Attempts to parse a metric in the format RESULT <graph>: <trace>= ... 1938 """Attempts to parse a metric in the format RESULT <graph>: <trace>= ...
1939 1939
1940 Args: 1940 Args:
1941 metric: The metric as a list of [<trace>, <value>] strings. 1941 metric: The metric as a list of [<trace>, <value>] strings.
1942 text: The text to parse the metric values from. 1942 text: The text to parse the metric values from.
1943 1943
1944 Returns: 1944 Returns:
1945 A list of floating point numbers found. 1945 A list of floating point numbers found.
1946 """ 1946 """
1947 # Format is: RESULT <graph>: <trace>= <value> <units> 1947 # Format is: RESULT <graph>: <trace>= <value> <units>
1948 metric_formatted = re.escape('RESULT %s: %s=' % (metric[0], metric[1])) 1948 metric_re = re.escape('RESULT %s: %s=' % (metric[0], metric[1]))
1949
1950 # The log will be parsed looking for format:
1951 # <*>RESULT <graph_name>: <trace_name>= <value>
1952 single_result_re = re.compile(
1953 metric_re + '\s*(?P<VALUE>[-]?\d*(\.\d*)?)')
1954
1955 # The log will be parsed looking for format:
1956 # <*>RESULT <graph_name>: <trace_name>= [<value>,value,value,...]
1957 multi_results_re = re.compile(
1958 metric_re + '\s*\[\s*(?P<VALUES>[-]?[\d\., ]+)\s*\]')
1959
1960 # The log will be parsed looking for format:
1961 # <*>RESULT <graph_name>: <trace_name>= {<mean>, <std deviation>}
1962 mean_stddev_re = re.compile(
1963 metric_re +
1964 '\s*\{\s*(?P<MEAN>[-]?\d*(\.\d*)?),\s*(?P<STDDEV>\d+(\.\d*)?)\s*\}')
1949 1965
1950 text_lines = text.split('\n') 1966 text_lines = text.split('\n')
1951 values_list = [] 1967 values_list = []
1952
1953 for current_line in text_lines: 1968 for current_line in text_lines:
1954 # Parse the output from the performance test for the metric we're 1969 # Parse the output from the performance test for the metric we're
1955 # interested in. 1970 # interested in.
1956 metric_re = metric_formatted +\ 1971 single_result_match = single_result_re.search(current_line)
1957 "(\s)*(?P<values>[0-9]+(\.[0-9]*)?)" 1972 multi_results_match = multi_results_re.search(current_line)
1958 metric_re = re.compile(metric_re) 1973 mean_stddev_match = mean_stddev_re.search(current_line)
1959 regex_results = metric_re.search(current_line) 1974 if (not single_result_match is None and
1960 1975 single_result_match.group('VALUE')):
1961 if not regex_results is None: 1976 values_list += [single_result_match.group('VALUE')]
1962 values_list += [regex_results.group('values')] 1977 elif (not multi_results_match is None and
1963 else: 1978 multi_results_match.group('VALUES')):
1964 metric_re = metric_formatted +\ 1979 metric_values = multi_results_match.group('VALUES')
1965 "(\s)*\[(\s)*(?P<values>[0-9,.]+)\]" 1980 values_list += metric_values.split(',')
1966 metric_re = re.compile(metric_re) 1981 elif (not mean_stddev_match is None and
1967 regex_results = metric_re.search(current_line) 1982 mean_stddev_match.group('MEAN')):
1968 1983 values_list += [mean_stddev_match.group('MEAN')]
1969 if not regex_results is None:
1970 metric_values = regex_results.group('values')
1971
1972 values_list += metric_values.split(',')
1973 1984
1974 values_list = [float(v) for v in values_list if IsStringFloat(v)] 1985 values_list = [float(v) for v in values_list if IsStringFloat(v)]
1975 1986
1976 # If the metric is times/t, we need to sum the timings in order to get 1987 # If the metric is times/t, we need to sum the timings in order to get
1977 # similar regression results as the try-bots. 1988 # similar regression results as the try-bots.
1978 metrics_to_sum = [['times', 't'], ['times', 'page_load_time'], 1989 metrics_to_sum = [['times', 't'], ['times', 'page_load_time'],
1979 ['cold_times', 'page_load_time'], ['warm_times', 'page_load_time']] 1990 ['cold_times', 'page_load_time'], ['warm_times', 'page_load_time']]
1980 1991
1981 if metric in metrics_to_sum: 1992 if metric in metrics_to_sum:
1982 if values_list: 1993 if values_list:
(...skipping 1910 matching lines...) Expand 10 before | Expand all | Expand 10 after
3893 # The perf dashboard scrapes the "results" step in order to comment on 3904 # The perf dashboard scrapes the "results" step in order to comment on
3894 # bugs. If you change this, please update the perf dashboard as well. 3905 # bugs. If you change this, please update the perf dashboard as well.
3895 bisect_utils.OutputAnnotationStepStart('Results') 3906 bisect_utils.OutputAnnotationStepStart('Results')
3896 print 'Error: %s' % e.message 3907 print 'Error: %s' % e.message
3897 if opts.output_buildbot_annotations: 3908 if opts.output_buildbot_annotations:
3898 bisect_utils.OutputAnnotationStepClosed() 3909 bisect_utils.OutputAnnotationStepClosed()
3899 return 1 3910 return 1
3900 3911
3901 if __name__ == '__main__': 3912 if __name__ == '__main__':
3902 sys.exit(main()) 3913 sys.exit(main())
OLDNEW
« no previous file with comments | « no previous file | tools/bisect-perf-regression_test.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698