telemetry: Add new metrics to smoothness benchmark.

tools/perf/metrics/smoothness.py

Index: tools/perf/metrics/smoothness.py
diff --git a/tools/perf/metrics/smoothness.py b/tools/perf/metrics/smoothness.py
index cfb19b093c372530fd0dee77365f7a69e65922b2..e228cd7fb693777fadafe3ff9dcdb8823afb3aca 100644
--- a/tools/perf/metrics/smoothness.py
+++ b/tools/perf/metrics/smoothness.py
@@ -4,6 +4,7 @@
 import os
 
 from telemetry.core import util
+from metrics import discrepancy
 
 TIMELINE_MARKER = 'smoothness_scroll'
 
@@ -88,6 +89,21 @@ def Average(numerator, denominator, scale = None, precision = None):
     avg = round(avg, precision)
   return avg
 
+def DivideIfPossibleOrZero(numerator, denominator):
+  if not denominator:
+    return 0.0
+  else:
+    return numerator / denominator
+
+def GeneralizedMean(values, exponent):
+  ''' http://en.wikipedia.org/wiki/Generalized_mean '''
+  if not values:
+    return 0.0
+  sum_of_powers = 0.0
+  for v in values:
+    sum_of_powers += v ** exponent
+  return (sum_of_powers / len(values)) ** (1.0/exponent)
+
 def CalcFirstPaintTimeResults(results, tab):
   if tab.browser.is_content_shell:
     results.Add('first_paint', 'ms', 'unsupported')
@@ -110,9 +126,24 @@ def CalcFirstPaintTimeResults(results, tab):
 def CalcResults(benchmark_stats, results):

[nduca, 2013/09/05 17:57:41]

you're saying screen frame throughout here but as far as i can tell this is impl
thread frames no? how do you reconcile these names with reality as we go toward
ubercomp etc?

[ernstm, 2013/09/05 18:38:07]

The name originates from the numFramesSentToScreen field in the stats returned
by GpuBenchmarkingExtension, used in the old smoothness implementation. It can
represent impl thread frames or frames submitted in
RenderWidget::DoDeferredUpdated in legacy software mode. Maybe we should just
drop the 'screen' and put some documentation into the code that explains what
exactly the time stamps, etc. represent?

   s = benchmark_stats
 
+  frame_times = []

[nduca, 2013/09/05 17:57:41]

screen_frame_xxx here and elsewhere

+  for i in xrange(1, len(s.screen_frame_timestamps)):
+    frame_times.append(
+        s.screen_frame_timestamps[i] - s.screen_frame_timestamps[i-1])
+
   # Scroll Results
   results.Add('mean_frame_time', 'ms',
               Average(s.total_time, s.screen_frame_count, 1000, 3))
+  results.Add('absolute_frame_discrepancy', '',
+              round(discrepancy.FrameDiscrepancy(s.screen_frame_timestamps,
+                                                 True), 4))
+  results.Add('relative_frame_discrepancy', '',

[nduca, 2013/09/05 17:57:41]

people are going to be very confused about the difference between absolute and
relative. can we pick one and stick with it?

[ernstm, 2013/09/05 18:38:07]

I've put in both, so that we can evaluate which one works best in practice. I
personally believe we should use the absolute discrepancy in combination with
the inverse RMS frame time. Absolute discrepancy quantifies the worst jank
(independent of the  number of good frames) and inverse RMS frame time
characterizes the average experience during the entire scroll. If we just use
the absolute discrepancy, we can drop the 'absolute' from the name.

+              round(discrepancy.FrameDiscrepancy(s.screen_frame_timestamps,
+                                                 False), 4))
+  results.Add('inverse_rms_frame_time', '',
+              round(DivideIfPossibleOrZero(1000.0,

[nduca, 2013/09/05 17:57:41]

this is the key metric, no? can we make this less about the way we compute it
and more about it being the frame time people should pay attention to?
frame_time perhaps.

And, can we deprecate mean_frame_time?

[ernstm, 2013/09/05 18:38:07]

The thing about this metric is that the exponent in the generalized mean allows
us to blend between the average frame time and the maximum frame time. The
inverse is similar to frame rate, but it is not the same (except when all frame
times are equal). We could call it generalized_frame_rate, or power_frame_rate
(generalized mean is also called power mean).

+                                           GeneralizedMean(frame_times, 2.0)),
+                    2))
   results.Add('dropped_percent', '%',
               Average(s.dropped_frame_count, s.screen_frame_count,
                       100, 1),