Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(70)

Side by Side Diff: scripts/slave/recipe_modules/auto_bisect/bisector.py

Issue 1825993003: Making check_initial_confidence verify return_code bisects. (Closed) Base URL: https://chromium.googlesource.com/chromium/tools/build.git@master
Patch Set: Removing unnecessary mocks. Created 4 years, 9 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | scripts/slave/recipe_modules/auto_bisect/bisector_test.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 # Copyright 2015 The Chromium Authors. All rights reserved. 1 # Copyright 2015 The Chromium Authors. All rights reserved.
2 # Use of this source code is governed by a BSD-style license that can be 2 # Use of this source code is governed by a BSD-style license that can be
3 # found in the LICENSE file. 3 # found in the LICENSE file.
4 4
5 import json 5 import json
6 import re 6 import re
7 import time 7 import time
8 import urllib 8 import urllib
9 9
10 from . import depot_config 10 from . import depot_config
(...skipping 469 matching lines...) Expand 10 before | Expand all | Expand 10 after
480 """Checks that the initial range presents a clear enough regression. 480 """Checks that the initial range presents a clear enough regression.
481 481
482 We ensure that the good and bad revisions produce significantly different 482 We ensure that the good and bad revisions produce significantly different
483 results, increasing the sample size until MAX_REQUIRED_SAMPLES is reached 483 results, increasing the sample size until MAX_REQUIRED_SAMPLES is reached
484 or REGRESSION_CHECK_TIMEOUT seconds have elapsed. 484 or REGRESSION_CHECK_TIMEOUT seconds have elapsed.
485 485
486 Returns: True if the revisions produced results that differ from each 486 Returns: True if the revisions produced results that differ from each
487 other in a statistically significant manner. False if such difference could 487 other in a statistically significant manner. False if such difference could
488 not be established in the time or sample size allowed. 488 not be established in the time or sample size allowed.
489 """ 489 """
490 if self.test_type != 'perf': 490 if self.test_type == 'return_code':
491 return True 491 return (self.good_rev.overall_return_code !=
492 self.bad_rev.overall_return_code)
492 493
493 if self.bypass_stats_check: 494 if self.bypass_stats_check:
494 dummy_result = self.good_rev.values != self.bad_rev.values 495 dummy_result = self.good_rev.values != self.bad_rev.values
495 if not dummy_result: 496 if not dummy_result:
496 self._set_insufficient_confidence_warning() 497 self._set_insufficient_confidence_warning()
497 return dummy_result 498 return dummy_result
498 499
499 with self.api.m.step.nest('Re-testing reference range'): 500 with self.api.m.step.nest('Re-testing reference range'):
500 expiration_time = time.time() + REGRESSION_CHECK_TIMEOUT 501 expiration_time = time.time() + REGRESSION_CHECK_TIMEOUT
501 while time.time() < expiration_time: 502 while time.time() < expiration_time:
(...skipping 342 matching lines...) Expand 10 before | Expand all | Expand 10 after
844 }) 845 })
845 return revision_rows 846 return revision_rows
846 847
847 def _get_build_url(self): 848 def _get_build_url(self):
848 properties = self.api.m.properties 849 properties = self.api.m.properties
849 bot_url = properties.get('buildbotURL', 850 bot_url = properties.get('buildbotURL',
850 'http://build.chromium.org/p/chromium/') 851 'http://build.chromium.org/p/chromium/')
851 builder_name = urllib.quote(properties.get('buildername', '')) 852 builder_name = urllib.quote(properties.get('buildername', ''))
852 builder_number = str(properties.get('buildnumber', '')) 853 builder_number = str(properties.get('buildnumber', ''))
853 return '%sbuilders/%s/builds/%s' % (bot_url, builder_name, builder_number) 854 return '%sbuilders/%s/builds/%s' % (bot_url, builder_name, builder_number)
OLDNEW
« no previous file with comments | « no previous file | scripts/slave/recipe_modules/auto_bisect/bisector_test.py » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698