Use only the steady gesture in frame rate tests.

Use only the steady gesture in frame rate tests.

Store frame intervals in an array rather than throwing them away. This is to
enable more-detailed study of frame rates.

Report error as sigma directly, instead of 60 - 1000/sigma. This should help
understand whether standard deviation is even the right metric to use for frame
rate instability.


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=112774

[nduca, 2011-11-11 18:49:12]

Thoughts?

The most controversial thing here is that I'm changing the reporting of stddev.
It used to be
  Error = 1000/mean - 1000/(sigma+mean)
Now I'm reporting it directly
  Error = sigma

[jbates, 2011-11-11 23:43:46]

http://codereview.chromium.org/8536008/diff/1/chrome/test/data/perf/frame_rat...
File chrome/test/data/perf/frame_rate/head.js (right):

http://codereview.chromium.org/8536008/diff/1/chrome/test/data/perf/frame_rat...
chrome/test/data/perf/frame_rate/head.js:243: var S = Math.sqrt(V / (N - 1));
(N - 1) should be N?

[Justin Novosad, 2011-11-14 16:08:42]

http://codereview.chromium.org/8536008/diff/1/chrome/test/data/perf/frame_rat...
File chrome/test/data/perf/frame_rate/head.js (right):

http://codereview.chromium.org/8536008/diff/1/chrome/test/data/perf/frame_rat...
chrome/test/data/perf/frame_rate/head.js:243: var S = Math.sqrt(V / (N - 1));
On 2011/11/11 23:43:46, jbates wrote:
> (N - 1) should be N?
Actually N-1 is correct for calculating unbiased sample variance. The unbiased
estimator is always preferable when N is variable.

http://codereview.chromium.org/8536008/diff/1/chrome/test/data/perf/frame_rat...
chrome/test/data/perf/frame_rate/head.js:249: // based on histogram-spikes or
percentiles.
I think the right way to do it would be to compute the variance in terms of fps.
To get sigma in fps, you would need to change line 239 to:
var v = 1000.0/__t_deltas[i] - 1000.0/M;
This the result of this is _not_ equivalent to 1/interval_sigma

[nduca, 2011-11-16 13:02:25]

PTAL

[Justin Novosad, 2011-11-16 15:10:53]

On 2011/11/16 13:02:25, nduca wrote:
> PTAL

LGTM

[jbates, 2011-11-16 18:06:00]

lgtm

[nduca, 2011-12-02 01:50:39]

Consider a typical run with ~199 frames all of which are 16.0ms and one frame
that is either :
 a) 100ms long
 b) 2ms long

For this:
 a) fps: 64.68  /- 30.85
 b) fps: 62.23  /- 3.7

Whereas if we consider intervals:
 a) interval: 15.93  /- 0.98
 b) interval: 16.42  /- 3.70

I hate both of these results, but I hate intervals a lot less.

[Justin Novosad, 2011-12-02 18:22:32]

I agree. I find the way the stats behave with intervals is more meaningful, from
a human interpretation point of view.

[commit-bot: I haz the power, 2011-12-02 18:42:29]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/nduca@chromium.org/8536008/12001

[commit-bot: I haz the power, 2011-12-02 20:44:10]

Change committed as 112774