Making check_initial_confidence verify return_code bisects.

scripts/slave/recipe_modules/auto_bisect/bisector_test.py

Index: scripts/slave/recipe_modules/auto_bisect/bisector_test.py
diff --git a/scripts/slave/recipe_modules/auto_bisect/bisector_test.py b/scripts/slave/recipe_modules/auto_bisect/bisector_test.py
index ea7c53230a91b6761f44e83382d0f3cc2b2cb2ce..8f6ddf5669513c98b6c4af149b8df1cecac430f3 100755
--- a/scripts/slave/recipe_modules/auto_bisect/bisector_test.py
+++ b/scripts/slave/recipe_modules/auto_bisect/bisector_test.py
@@ -149,12 +149,16 @@ class BisectorTest(unittest.TestCase):  # pragma: no cover
 
   @mock.patch.object(auto_bisect.bisector.Bisector, 'significantly_different',
                      mock.MagicMock())
-  def test_check_initial_confidence_not_required(self):
+  def test_check_initial_confidence_return_code(self):
     return_code_config = self.bisect_config
     return_code_config['test_type'] = 'return_code'
     # When confidence is not required, confidence_score should not be called.

[qyearsley, 2016/03/23 20:54:40]

This comment is now confusing since this test method doesn't reference use or
check confidence_score (but it checks that significantly_different is not called
-- which could again be a separate little test method).

[RobertoCN, 2016/03/23 21:52:09]

Removed the check for not calling significantly different for two reasons:
- Bypass repro check is no longer a use case (the parameter is ignored)
- Significantly_different has been modified to support return_code bisects.

Split the test for return code confidence check into pass and fail.

     bisector = auto_bisect.bisector.Bisector(self.dummy_api, return_code_config,
                                              MockRevisionClass)
+    bisector.good_rev.mean_value = 0
+    bisector.bad_rev.mean_value = 0
+    self.assertFalse(bisector.check_initial_confidence())
+    bisector.bad_rev.mean_value = 1
     self.assertTrue(bisector.check_initial_confidence())
     self.assertFalse(bisector.significantly_different.called)
 

[qyearsley, 2016/03/23 20:54:40]

This test method might be a bit clearer if it's split into separate methods that
each test one simple behavior/case: there could be one case where
check_initial_confidence() returns True and another where it returns False.

[RobertoCN, 2016/03/23 21:52:09]

Done.