Iteratively increase sample size for good/bad classification.

scripts/slave/recipe_modules/auto_bisect/resources/significantly_different_test.py

Index: scripts/slave/recipe_modules/auto_bisect/resources/significantly_different_test.py
diff --git a/scripts/slave/recipe_modules/auto_bisect/resources/significantly_different_test.py b/scripts/slave/recipe_modules/auto_bisect/resources/significantly_different_test.py
new file mode 100755
index 0000000000000000000000000000000000000000..f427c39e040e00d7f0154c37e8ad72654bfd00d5
--- /dev/null
+++ b/scripts/slave/recipe_modules/auto_bisect/resources/significantly_different_test.py
@@ -0,0 +1,30 @@
+#!/usr/bin/python
+"""Tests for significantly_different."""
+
+import os
+import unittest
+
+# pylint: disable=relative-import
+import significantly_different
+
+
+class SignificantlyDifferentTest(unittest.TestCase):
+
+  def setUp(self):
+    self.conda_path = None
+    try:
+      # pylint: disable=unused-variable
+      import scipy
+      # pylint: enable=unused-variable

[qyearsley, 2016/01/26 19:26:00]

I think it also works to just put the pylint: disable=unused-variable on the
same line as `import scipy`, and then it should only affect that one line.

[RobertoCN, 2016/02/01 17:29:51]

Done.

+      self.conda_path = 'python'
+    except ImportError:
+      if os.path.exists(os.path.expanduser('~/conda-test/bin/python')):
+        self.conda_path = os.path.expanduser('~/conda-test/bin/python')
+
+  def test_basic_case(self):
+    significantly_different.main(
+        ['', '[1, 2, 3, 3, 2, 1]', '[1, 2, 2, 2, 5, 0]', 0.05], self.conda_path)

[qyearsley, 2016/01/26 19:26:00]

Does this actually run anaconda and run the statistical tests?

Since the main() function doesn't return values, and it just prints them, it's a
bit hard to test in a unit test; maybe it would be better to put the main logic
in another function which returns the results dict, and then main() just calls
that function and prints the results as JSON. Then, in the test, you could call
the function with the logic and assert that you get certain results.

[RobertoCN, 2016/02/01 17:29:51]

Done.

+
+
+if __name__ == '__main__':
+  unittest.main()