Make VR latency results compatible with perf dashboard, refactor into classes

chrome/test/vr/perf/latency/android_webvr_latency_test.py

Index: chrome/test/vr/perf/latency/android_webvr_latency_test.py
diff --git a/chrome/test/vr/perf/latency/android_webvr_latency_test.py b/chrome/test/vr/perf/latency/android_webvr_latency_test.py
new file mode 100644
index 0000000000000000000000000000000000000000..69e490be7b588a1b7b646fb5500e56c3e509cbd3
--- /dev/null
+++ b/chrome/test/vr/perf/latency/android_webvr_latency_test.py
@@ -0,0 +1,116 @@
+# Copyright 2017 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import webvr_latency_test
+
+import logging
+import os
+import time
+
+DEFAULT_SCREEN_WIDTH = 720
+DEFAULT_SCREEN_HEIGHT = 1280
+
+class AndroidWebVrLatencyTest(webvr_latency_test.WebVrLatencyTest):
+  """Android implementation of the WebVR latency test."""
+  def __init__(self, args):
+    super(AndroidWebVrLatencyTest, self).__init__(args)
+    self.device_name = self._Adb(['shell', 'getprop',
+                                  'ro.product.name']).strip()
+
+  def _Setup(self):
+    self._Adb(['root'])
+    # Install the latest VrCore APK
+    self._Adb(['install', '-r', '-d',
+               '../../third_party/gvr-android-sdk/test-apks/vr_services'
+               '/vr_services_current.apk'])
+    self._SaveInstalledVrCoreVersion()
+    # TODO(bsheedy): Make APK path configurable so usable with other channels
+    self._Adb(['install', '-r', 'apks/ChromePublic.apk'])
+    # Force WebVR support, remove open tabs, and don't have first run
+    # experience.
+    self._SetChromeCommandLineFlags(['--enable-webvr', '--no-restore-state',
+                                     '--disable-fre'])
+    # Wake up the device and sleep, otherwise WebGL can crash on startup.
+    self._Adb(['shell', 'input', 'keyevent', 'KEYCODE_WAKEUP'])
+    time.sleep(1)
+    # Start Chrome
+    # TODO(bsheedy): See about having versioned copies of the flicker app
+    # instead of using personal github.
+    self._Adb(['shell', 'am', 'start',
+               '-a', 'android.intent.action.MAIN',
+               '-n', 'org.chromium.chrome/com.google.android.apps.chrome.Main',
+               'https://weableandbob.github.io/Motopho/flicker_apps/webvr/webvr-flicker-app-klaus.html?polyfill=0\&canvasClickPresents=1'])

[Lei Lei, 2017/04/17 22:57:13]

This line is too long, can you split it into multiple lines?

[bsheedy, 2017/04/18 21:17:25]

Done.

+    time.sleep(10)
+    # Tap the center of the screen to start presenting.
+    (width, height) = self._GetScreenResolution()
+    self._Adb(['shell', 'input', 'touchscreen', 'tap', str(width/2),
+               str(height/2)])

[Lei Lei, 2017/04/17 22:57:13]

Is there any way to verify if Chrome is in VR mode just in case tapping doesn't
work?

[bsheedy, 2017/04/18 21:17:25]

There's no way to query it directly. However, VrCore does output some things to
logcat when entering VR, so we could clear logcat before entering VR, attempt to
enter, and check whether the logcat output contains the string we're looking
for.

I've gone ahead and added that to the script - I haven't had issues with it not
entering VR before, but adding this won't (shouldn't?) hurt.

[Lei Lei, 2017/04/20 04:56:20]

I am not sure if this will be an issue, but I remember we had this issue in
daydream tests, sometimes we got invalid results because Chrome never entered to
VR mode. It is better to add this check and exclude those invalid results. 

+    time.sleep(5)
+
+  def _Teardown(self):
+    # Exit VR and Close Chrome
+    self._Adb(['shell', 'input', 'keyevent', 'KEYCODE_BACK'])
+    self._Adb(['shell', 'am', 'force-stop', 'org.chromium.chrome'])
+    # Turn off the screen
+    self._Adb(['shell', 'input', 'keyevent', 'KEYCODE_POWER'])
+
+  def _Adb(self, cmd):
+    """Runs the given command via adb.
+
+    Returns:
+      A string containing the stdout and stderr of the adb command.
+    """
+    # TODO(bsheedy): Maybe migrate to use Devil (overkill?)

[Lei Lei, 2017/04/17 22:57:13]

I think it is better to migrate to use Devil, so we don't need to update adb
binary to the latest version manually.

[bsheedy, 2017/04/18 21:17:25]

I got it working with Devil, but its dependencies caused the isolated test to be
quite large (uploading to the isolation server stated a push state size of
1403267784, so over 1.4 GB?), which significantly impacted how long it took to
isolate, upload, and fully run the test. If we need Devil in the future, it's
still an option, but for now, I've started including and using the adb in
//third_party/android_tools/sdk/platform-tools, which is regularly updated.

+    return self._RunCommand([self.args.adb_path] + cmd)
+
+  def _SaveInstalledVrCoreVersion(self):
+    """Retrieves the VrCore version and saves it for dashboard uploading."""
+    output = self._Adb(['shell', 'dumpsys', 'package', 'com.google.vr.vrcore'])
+    version = None
+    for line in output.split('\n'):
+      if 'versionName' in line:
+        version = line.split('=')[1]
+        break
+    if version:
+      logging.info('VrCore version is %s', version)
+    else:
+      logging.info('VrCore version not retrieved')
+      version = '0'
+    if not (self.args.output_dir and os.path.isdir(self.args.output_dir)):
+      logging.warning('No output directory, not saving VrCore version')
+      return
+    with file(os.path.join(self.args.output_dir,
+                           self.args.vrcore_version_file), 'w') as outfile:
+      outfile.write(version)
+
+  def _SetChromeCommandLineFlags(self, flags):
+    """Sets the Chrome command line flags to the given list."""
+    self._Adb(['shell', "echo 'chrome " + ' '.join(flags) + "' > "
+               + '/data/local/tmp/chrome-command-line'])
+
+  def _GetScreenResolution(self):
+    """Retrieves the device's screen resolution, or a default if not found.
+
+    Returns:
+      A tuple (width, height).
+    """
+    output = self._Adb(['shell', 'dumpsys', 'display'])
+    width = None
+    height = None
+    for line in output.split('\n'):
+      if 'mDisplayWidth' in line:
+        width = int(line.split('=')[1])
+      elif 'mDisplayHeight' in line:
+        height = int(line.split('=')[1])
+      if width and height:
+        break
+    if not width:
+      logging.warning('Could not get actual screen width, defaulting to %d',
+                      DEFAULT_SCREEN_WIDTH)
+      width = DEFAULT_SCREEN_WIDTH
+    if not height:
+      logging.warning('Could not get actual screen height, defaulting to %d',
+                      DEFAULT_SCREEN_HEIGHT)
+      height = DEFAULT_SCREEN_HEIGHT
+    return (width, height)