[Telemetry] Refactoring in preparation for video-based Speed Index support.

tools/telemetry/telemetry/core/platform/android_platform_backend.py

Index: tools/telemetry/telemetry/core/platform/android_platform_backend.py
diff --git a/tools/telemetry/telemetry/core/platform/android_platform_backend.py b/tools/telemetry/telemetry/core/platform/android_platform_backend.py
index 9cd6f677bb4dbdcee2c1a41fe9612ccae855ec0a..33404777f04e672d947bde42ad8cf31a0ae94e49 100644
--- a/tools/telemetry/telemetry/core/platform/android_platform_backend.py
+++ b/tools/telemetry/telemetry/core/platform/android_platform_backend.py
@@ -3,6 +3,9 @@
 # found in the LICENSE file.
 
 import logging
+import os
+import subprocess
+import tempfile
 
 from telemetry.core import exceptions
 from telemetry.core import platform
@@ -39,6 +42,8 @@ class AndroidPlatformBackend(
     self._host_platform_backend = platform.CreatePlatformBackendForCurrentOS()
     self._can_access_protected_file_contents = \
         self._adb.CanAccessProtectedFileContents()
+    self._video_recorder = None
+    self._video_output = None
     if self._no_performance_mode:
       logging.warning('CPU governor will not be set!')
 
@@ -135,6 +140,9 @@ class AndroidPlatformBackend(
   def GetOSName(self):
     return 'android'
 
+  def GetOSVersionName(self):
+    return self._adb.GetBuildId()[0]
+
   def CanFlushIndividualFilesFromSystemCache(self):
     return False
 
@@ -170,6 +178,29 @@ class AndroidPlatformBackend(
     raise NotImplementedError(
         'Please teach Telemetry how to install ' + application)
 
+  def CanCaptureBrowserVideo(self):
+    return self.GetOSVersionName() >= 'K'
+
+  def StartBrowserVideoCapture(self, tab, max_bitrate_mbps):
+    assert not self._video_recorder, 'Already started video capture'
+    assert max_bitrate_mbps >= 0.1 and max_bitrate_mbps <= 100
+    self._video_output = tempfile.mkstemp()[1]
+    self._video_recorder = subprocess.Popen(
+        [os.path.join(util.GetChromiumSrcDir(), 'build', 'android',
+                      'screenshot.py'),
+         '--video', '--bitrate', str(max_bitrate_mbps), '--file',
+         self._video_output], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
+
+  def StopBrowserVideoCapture(self):
+    assert self._video_recorder, 'Must start video capture first'
+    self._video_recorder.communicate(input='\n')
+    self._video_recorder.wait()
+    self._video_recorder = None
+
+    # TODO(tonyg/szym): Decode the mp4 output and return the list of bitmaps.
+    raise NotImplementedError("mp4 video saved to %s, but Telemetry doesn't "

[bulach, 2013/11/28 11:05:23]

drive-by: no idea if this is what you need, but:
avconv -i foo.mp4 Foo%d.bmp would generate a gazillion huge bitmaps :)

avconv is part of libffmpeg but I'm not sure we're building it for host.. sudo
apt-get install libav-tools should also work.

[szym, 2013/11/29 11:05:12]

Like you said, it could easily reach 1-2GB.

It would be great to use ffmpeg with python bindings (like pyffmpeg), but if we
want to keep telemetry dependency-free, I think piping the output of ffmpeg is
going to work just fine.

If we really want to keep telemetry dependency-free, then we could pipe
rawvideo:

def ReadFrames(inputfile):
  size = None
  proc = subprocess.Popen(['avconv', '-i', inputfile],
                          stderr=subprocess.PIPE)
  for line in proc.stderr.readlines():
    if 'Video:' in line:
      size = line.split(',')[2]
      size = map(int, size.split()[0].split('x'))
      break
  assert size
  proc.wait()

  frame_length = size[0] * size[1] * 3
  frame_data = bytearray(frame_length)

  with open(os.devnull, 'w') as fnull:
    # Use rawvideo so that we don't need any external library to parse frames.
    proc = subprocess.Popen(['avconv', '-i', inputfile, '-vcodec', 'rawvideo',
                            '-pix_fmt', 'rgb24', '-f', 'rawvideo', '-'],
                            stderr=fnull,
                            stdout=subprocess.PIPE)
    while True:
      num_read = proc.stdout.readinto(frame_data)
      if not num_read:
        raise StopIteration
      assert num_read == len(frame_data)
      yield frame_data, size


On my laptop this runs at 110 fps.

However, computing the histogram in a python loop will slow things down a lot.
In experiments on my laptop, I got:

raw python: 4.5 fps
numpy.histogram: 9 fps
PIL.Image.histogram: 75 fps

I think PIL is available on all telemetry platforms.

[tonyg, 2013/11/29 20:42:07]

We accidentally checked in a PIL dep on the bots recently and it failed. That's
not to say we couldn't get it installed. Can we pull it into third_party?

[bulach, 2013/12/02 18:11:59]

I'm not volunteering myself but just throwing some ideas.. :)
ffmpeg is already in third_party.. I suppose we could write a small host tool in
c++ for all platforms that would do the keyframe decoding *and* calculate the
histograms from raw frames directly in c++ as well?

I'm not at all familiar with ffmpeg though, so have no idea how easy is to rip
"avconv" and hook up in an histogram calculator...

since ideas are cheap :) here's another one:
"imagemagick" package has a histogram calculator, not sure if this is what we're
looking for here.

$ convert chrome/test/data/dromaeo/images/dino7.png -define
histogram:unique-colors=true -format %c histogram:info:-
...
       102: (  8,  5,  3,110) #0805036E rgba(8,5,3,0.431373)
        19: (  8,  5,  3, 66) #08050342 rgba(8,5,3,0.258824)
        17: (  8,  5,  3, 88) #08050358 rgba(8,5,3,0.345098)
        10: (  8,  5,  3, 44) #0805032C rgba(8,5,3,0.172549)
         8: (  8,  5,  3, 22) #08050316 rgba(8,5,3,0.0862745)
         8: (  8,  5,  3,  0) #08050300 rgba(8,5,3,0)
         2: (  8,  5,  3,104) #08050368 rgba(8,5,3,0.407843)
         2: (  8,  5,  3,105) #08050369 rgba(8,5,3,0.411765)
         2: (  8,  5,  3, 97) #08050361 rgba(8,5,3,0.380392)
         2: (  8,  5,  3, 87) #08050357 rgba(8,5,3,0.341176)
...


it looks like imagemagick also work on all platforms and its apache-style
licensed... so perhaps we could pipe from avconv into imagemagick? szym, would
you be able to try that loop?

+                              "know how to decode it." % self._video_output)
+
   def _GetFileContents(self, fname):
     if not self._can_access_protected_file_contents:
       logging.warning('%s cannot be retrieved on non-rooted device.' % fname)