[telemetry] Fix computation of standard deviations

[telemetry] Fix computation of standard deviations

Fixes computation of standard deviation in ListOfScalarValues so that:
- regular standard deviation is always used when merging values from
  the same page,
- pooled standard deviation is always used when mergning values from
  different pages.

Note: This means that metrics in a benchmark that only collect one value
per page will get a std of 0.0. (At least two samples per page are
needed for the pooled stdev to be defined.)

BUG=catapult:#2369

Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/dc09b776a2fe2c211d96f3e34dd906516f8f8b18

[perezju, 2016-06-20 16:13:17]

perezju@chromium.org changed reviewers:
+ eakuefner@chromium.org, nednguyen@google.com, petrcermak@chromium.org

[perezju, 2016-06-20 16:13:18]

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
File telemetry/telemetry/value/list_of_scalar_values_unittest.py (right):

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
telemetry/telemetry/value/list_of_scalar_values_unittest.py:109:
self.assertAlmostEqual(156.88849, vM.std, places=4)
I've left on purpose the original values (instead of making them more
"realistic", i.e. all close to the same mean) so that the computed ("regular")
stdev is clearly larger.

[petrcermak, 2016-06-20 16:52:25]

lgtm

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
File telemetry/telemetry/value/list_of_scalar_values_unittest.py (right):

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
telemetry/telemetry/value/list_of_scalar_values_unittest.py:221: # Values from
same page use regular standard deviation.
supernit: s/same/the same/

[nednguyen, 2016-06-20 17:23:02]

lgtm if Ethan is happy

[eakuefner, 2016-06-20 17:26:32]

lgtm

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
File telemetry/telemetry/value/list_of_scalar_values.py (right):

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
telemetry/telemetry/value/list_of_scalar_values.py:220: # ListOfScalarValues'
constructor.
supernit: s/s'/s's/

[perezju, 2016-06-21 09:48:07]

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
File telemetry/telemetry/value/list_of_scalar_values.py (right):

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
telemetry/telemetry/value/list_of_scalar_values.py:220: # ListOfScalarValues'
constructor.
On 2016/06/20 17:26:32, eakuefner wrote:
> supernit: s/s'/s's/

Grammar confuses me. I rephrased this to avoid the apostrophe.

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
File telemetry/telemetry/value/list_of_scalar_values_unittest.py (right):

https://codereview.chromium.org/2087463002/diff/1/telemetry/telemetry/value/l...
telemetry/telemetry/value/list_of_scalar_values_unittest.py:221: # Values from
same page use regular standard deviation.
On 2016/06/20 16:52:24, petrcermak wrote:
> supernit: s/same/the same/

Done.

[perezju, 2016-06-21 09:48:13]

The CQ bit was checked by perezju@chromium.org

[perezju, 2016-06-21 09:48:13]

The patchset sent to the CQ was uploaded after l-g-t-m from
eakuefner@chromium.org, nednguyen@google.com, petrcermak@chromium.org
Link to the patchset: https://codereview.chromium.org/2087463002/#ps20001
(title: "fixed nits")

[commit-bot: I haz the power, 2016-06-21 09:48:26]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2087463002/20001

[commit-bot: I haz the power, 2016-06-21 09:59:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-06-21 09:59:55]

Try jobs failed on following builders:
  Catapult Windows Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Wi...)

[perezju, 2016-06-21 10:05:23]

Some more unittests need updating. I'm sheriffing android bots today/tomorrow,
will get back to this on Thursday.

[perezju, 2016-06-23 11:07:24]

The CQ bit was checked by perezju@chromium.org

[perezju, 2016-06-23 11:07:24]

The patchset sent to the CQ was uploaded after l-g-t-m from
eakuefner@chromium.org, nednguyen@google.com, petrcermak@chromium.org
Link to the patchset: https://codereview.chromium.org/2087463002/#ps40001
(title: "fix more unit tests")

[commit-bot: I haz the power, 2016-06-23 11:07:26]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2087463002/40001

[commit-bot: I haz the power, 2016-06-23 11:41:47]

Description was changed from

==========
[telemetry] Fix computation of standard deviations

Fixes computation of standard deviation in ListOfScalarValues so that:
- regular standard deviation is always used when merging values from
  the same page,
- pooled standard deviation is always used when mergning values from
  different pages.

Note: This means that metrics in a benchmark that only collect one value
per page will get a std of 0.0. (At least two samples per page are
needed for the pooled stdev to be defined.)

BUG=catapult:#2369
==========

to

==========
[telemetry] Fix computation of standard deviations

Fixes computation of standard deviation in ListOfScalarValues so that:
- regular standard deviation is always used when merging values from
  the same page,
- pooled standard deviation is always used when mergning values from
  different pages.

Note: This means that metrics in a benchmark that only collect one value
per page will get a std of 0.0. (At least two samples per page are
needed for the pooled stdev to be defined.)

BUG=catapult:#2369

Committed:
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...
==========

[commit-bot: I haz the power, 2016-06-23 11:41:48]

Committed patchset #3 (id:40001) as
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...