telemetry: add screenshot measurement

tools/perf/measurements/screenshot.py

Index: tools/perf/measurements/screenshot.py
diff --git a/tools/perf/measurements/screenshot.py b/tools/perf/measurements/screenshot.py
new file mode 100644
index 0000000000000000000000000000000000000000..87b2dbb50e76922f10a8d5ac08e5509bd6afb42d
--- /dev/null
+++ b/tools/perf/measurements/screenshot.py
@@ -0,0 +1,48 @@
+# Copyright 2014 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+import os
+import time
+
+from telemetry.page import page_measurement
+
+
+class Screenshot(page_measurement.PageMeasurement):
+  @classmethod
+  def AddCommandLineArgs(cls, parser):
+    parser.add_option('--png-outdir',
+                      help='Output directory for the PNG files')
+    parser.add_option('--screenshot-delay', type='float',
+                      default=3,
+                      help='Wait for this many seconds before taking the '
+                      'screenshot')
+    parser.add_option('--screenshot-timeout', type='float',
+                      default=5,
+                      help='Timeout after this many seconds when attempting to '
+                      'take the screenshot')
+
+  @classmethod
+  def ProcessCommandLineArgs(cls, parser, args):
+    if not args.png_outdir:
+      parser.error('Please specify --png-outdir')
+    cls._png_outdir = args.png_outdir
+
+  def MeasurePage(self, page, tab, results):
+    tab.WaitForDocumentReadyStateToBeComplete()
+    time.sleep(self.options.screenshot_delay)

[tonyg, 2014/05/21 09:36:57]

This seems like it'll be really flaky.

[ernstm, 2014/05/22 17:48:01]

This measurement will always be flaky on non-static pages, and that is okay. We
are not planning to run this measurement routinely on the bots. Instead, we want
to use it for one-off correctness tests of new features, e.g. GPU rasterization
in a semi-automated way. The plan is to use cluster telemetry to generate
screenshots for large page sets with and without the feature and then compare
them with e.g. Skia's image diff. I've done this locally on my machine and it
proved to be very useful. I was able to identify two correctness issues that
have not been detected by other tests.

+    screenshot = tab.Screenshot(self.options.screenshot_timeout)
+
+    outpath = os.path.abspath(
+        os.path.join(self._png_outdir, page.file_safe_name)) + '.png'
+
+    if os.path.exists(outpath):
+      previous_mtime = os.path.getmtime(outpath)
+    else:
+      previous_mtime = -1
+
+    screenshot.WritePngFile(outpath)
+
+    saved_picture_count = 0
+    if os.path.exists(outpath) and os.path.getmtime(outpath) > previous_mtime:
+      saved_picture_count = 1
+    results.Add('saved_picture_count', 'count', saved_picture_count)