Re-applying reverted changes for regression confidence check + fix: ConfidenceScoretakes flat lists

Re-applying reverted changes plus fix. ConfidenceScore now takes flat
lists and the previously offending code now checks whether the lists
need to be flattened before being passed to the function.

BUG=422727
Review URL: https://codereview.chromium.org/644323002

Cr-Commit-Position: refs/heads/master@{#300196}

Committed: https://crrev.com/d0efba359b185fe140635b6269442a81e49e4c69
Cr-Commit-Position: refs/heads/master@{#300953}

[RobertoCN, 2014-10-20 23:12:11]

robertocn@chromium.org changed reviewers:
+ prasadv@chromium.org, qyearsley@chromium.org

[RobertoCN, 2014-10-20 23:12:11]

Tested this:
http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisec...

Failed expectedly

[RobertoCN, 2014-10-22 23:00:44]

Hey guys, I seem to have successfully rebased these changes after the
not-so-minor refactoring that took place. I tried a positive and a negative try
job and they both completed as expected.

PTAL

[qyearsley, 2014-10-23 00:38:39]

Hey, this looks good, as before, although I've got a couple more comments now.

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
File tools/auto_bisect/bisect_perf_regression.py (right):

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:101: \'Good\' revision: {}
Giving names to each of these template fields would make it more readable when
the template is filled out.

For example, if template = """
Good revision: {good_rev}
Mean: {mean}
Std. Err.: {std_err}
...
"""
then you can call template.format(good_rev=123, mean=12, std_err=1)

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:110: NOTE: There\'s still a chance
that this is actually a regression, but you may
Escaping single quotes isn't necessary insider a """-quoted string.

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:2259: return
BisectResults(error=error)
Might be a good idea to extract everything under the "if not
self.opts.debug_ignore_regression_confidence" into a separate function.

[RobertoCN, 2014-10-23 19:51:45]

Applied all recommended changes. Manually submitted bisect jobs with these
changes and all seem to work.

Committing.

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
File tools/auto_bisect/bisect_perf_regression.py (right):

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:101: \'Good\' revision: {}
On 2014/10/23 00:38:39, qyearsley wrote:
> Giving names to each of these template fields would make it more readable when
> the template is filled out.
> 
> For example, if template = """
> Good revision: {good_rev}
> Mean: {mean}
> Std. Err.: {std_err}
> ...
> """
> then you can call template.format(good_rev=123, mean=12, std_err=1)
> 

Done.

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:110: NOTE: There\'s still a chance
that this is actually a regression, but you may
On 2014/10/23 00:38:39, qyearsley wrote:
> Escaping single quotes isn't necessary insider a """-quoted string.

Done.

https://codereview.chromium.org/665893003/diff/60001/tools/auto_bisect/bisect...
tools/auto_bisect/bisect_perf_regression.py:2259: return
BisectResults(error=error)
On 2014/10/23 00:38:38, qyearsley wrote:
> Might be a good idea to extract everything under the "if not
> self.opts.debug_ignore_regression_confidence" into a separate function.

Done.

[RobertoCN, 2014-10-23 19:52:05]

The CQ bit was checked by robertocn@chromium.org

[commit-bot: I haz the power, 2014-10-23 19:53:48]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665893003/80001

[commit-bot: I haz the power, 2014-10-23 19:53:50]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-10-23 19:53:51]

No LGTM from a valid reviewer yet. Only full committers are accepted.
Even if an LGTM may have been provided, it was from a non-committer or
a provisional committer, _not_ a full super star committer.
See http://www.chromium.org/getting-involved/become-a-committer
Note that this has nothing to do with OWNERS files.

[qyearsley, 2014-10-23 20:25:08]

On 2014/10/23 19:53:51, I haz the power (commit-bot) wrote:
> No LGTM from a valid reviewer yet. Only full committers are accepted.
> Even if an LGTM may have been provided, it was from a non-committer or
> a provisional committer, _not_ a full super star committer.
> See http://www.chromium.org/getting-involved/become-a-committer
> Note that this has nothing to do with OWNERS files.

LGTM

[qyearsley, 2014-10-23 20:25:12]

The CQ bit was checked by qyearsley@chromium.org

[commit-bot: I haz the power, 2014-10-23 20:27:34]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665893003/80001

[commit-bot: I haz the power, 2014-10-23 21:33:35]

Committed patchset #5 (id:80001)

[commit-bot: I haz the power, 2014-10-23 21:34:15]

Patchset 5 (id:??) landed as
https://crrev.com/d0efba359b185fe140635b6269442a81e49e4c69
Cr-Commit-Position: refs/heads/master@{#300953}