Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(433)

Side by Side Diff: tools/auto_bisect/bisect_perf_regression.py

Issue 1205663002: Make bisect to abort early when the return codes for known good and known bad revisions are same. (Closed) Base URL: https://chromium.googlesource.com/chromium/src.git@master
Patch Set: Created 5 years, 5 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
1 #!/usr/bin/env python 1 #!/usr/bin/env python
2 # Copyright 2013 The Chromium Authors. All rights reserved. 2 # Copyright 2013 The Chromium Authors. All rights reserved.
3 # Use of this source code is governed by a BSD-style license that can be 3 # Use of this source code is governed by a BSD-style license that can be
4 # found in the LICENSE file. 4 # found in the LICENSE file.
5 5
6 """Chromium auto-bisect tool 6 """Chromium auto-bisect tool
7 7
8 This script bisects a range of commits using binary search. It starts by getting 8 This script bisects a range of commits using binary search. It starts by getting
9 reference values for the specified "good" and "bad" commits. Then, for revisions 9 reference values for the specified "good" and "bad" commits. Then, for revisions
10 in between, it will get builds, run tests and classify intermediate revisions as 10 in between, it will get builds, run tests and classify intermediate revisions as
(...skipping 2289 matching lines...) Expand 10 before | Expand all | Expand 10 after
2300 2300
2301 if good_results[1]: 2301 if good_results[1]:
2302 error = ('An error occurred while building and running the \'good\' ' 2302 error = ('An error occurred while building and running the \'good\' '
2303 'reference value. The bisect cannot continue without ' 2303 'reference value. The bisect cannot continue without '
2304 'a working \'good\' revision to start from.\n\nError: %s' % 2304 'a working \'good\' revision to start from.\n\nError: %s' %
2305 good_results[0]) 2305 good_results[0])
2306 return BisectResults(error=error) 2306 return BisectResults(error=error)
2307 2307
2308 # We need these reference values to determine if later runs should be 2308 # We need these reference values to determine if later runs should be
2309 # classified as pass or fail. 2309 # classified as pass or fail.
2310
2310 known_bad_value = bad_results[0] 2311 known_bad_value = bad_results[0]
2311 known_good_value = good_results[0] 2312 known_good_value = good_results[0]
2312 2313
2314 # Abort bisect early when the return codes for known good
2315 # and known bad revisions are same.
2316 if (self._IsBisectModeReturnCode() and
2317 known_bad_value['mean'] == known_good_value['mean']):
2318 return BisectResults(abort_reason=('known good and known bad revisions '
2319 'returned same return code (return code=%s). '
2320 'Continuing bisect might not yield any results.' %
2321 known_bad_value['mean']))
2313 # Check the direction of improvement only if the improvement_direction 2322 # Check the direction of improvement only if the improvement_direction
2314 # option is set to a specific direction (1 for higher is better or -1 for 2323 # option is set to a specific direction (1 for higher is better or -1 for
2315 # lower is better). 2324 # lower is better).
2316 improvement_dir = self.opts.improvement_direction 2325 improvement_dir = self.opts.improvement_direction
2317 if improvement_dir: 2326 if improvement_dir:
2318 higher_is_better = improvement_dir > 0 2327 higher_is_better = improvement_dir > 0
2319 if higher_is_better: 2328 if higher_is_better:
2320 message = "Expecting higher values to be better for this metric, " 2329 message = "Expecting higher values to be better for this metric, "
2321 else: 2330 else:
2322 message = "Expecting lower values to be better for this metric, " 2331 message = "Expecting lower values to be better for this metric, "
(...skipping 571 matching lines...) Expand 10 before | Expand all | Expand 10 after
2894 # bugs. If you change this, please update the perf dashboard as well. 2903 # bugs. If you change this, please update the perf dashboard as well.
2895 bisect_utils.OutputAnnotationStepStart('Results') 2904 bisect_utils.OutputAnnotationStepStart('Results')
2896 print 'Runtime Error: %s' % e 2905 print 'Runtime Error: %s' % e
2897 if opts.output_buildbot_annotations: 2906 if opts.output_buildbot_annotations:
2898 bisect_utils.OutputAnnotationStepClosed() 2907 bisect_utils.OutputAnnotationStepClosed()
2899 return 1 2908 return 1
2900 2909
2901 2910
2902 if __name__ == '__main__': 2911 if __name__ == '__main__':
2903 sys.exit(main()) 2912 sys.exit(main())
OLDNEW
« no previous file with comments | « no previous file | tools/auto_bisect/bisect_perf_regression_test.py » ('j') | tools/auto_bisect/bisect_perf_regression_test.py » ('J')

Powered by Google App Engine
This is Rietveld 408576698