Android: first step into making "perf tests" reuse the functional test infra.

build/android/pylib/perf/setup.py

Index: build/android/pylib/perf/setup.py
diff --git a/build/android/pylib/perf/setup.py b/build/android/pylib/perf/setup.py
new file mode 100644
index 0000000000000000000000000000000000000000..66f0433623926e541bbe3f3d6231c59d15b8753d
--- /dev/null
+++ b/build/android/pylib/perf/setup.py
@@ -0,0 +1,72 @@
+# Copyright 2013 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""Generates test runner factory and tests for performance tests."""
+
+import json
+import logging
+import os
+import psutil
+import signal
+import test_runner
+
+from pylib import android_commands
+from pylib import cmd_helper
+from pylib import forwarder
+from pylib import ports
+
+
+def _KillPendingServers():
+  for retry in range(5):
+    for server in ['lighttpd', 'web-page-replay']:
+      pids = [p.pid for p in psutil.process_iter() if server in p.name]
+      for pid in pids:
+        try:
+          logging.warning('Killing %s %s', server, pid)
+          os.kill(pid, signal.SIGQUIT)
+        except Exception as e:
+          logging.warning('Failed killing %s %s %s', server, pid, e)
+  # Restart the adb server with taskset to set a single CPU affinity.

[craigdh, 2013/08/13 16:58:46]

This seems widely applicable. Should this be done for all types of tests?

[bulach, 2013/08/13 17:06:53]

yes, just not yet. :)
the main goal of this patch is to start converging the perf tests with
functional tests, and at this stage, I'd like to keep changes to other harnesses
to a minimum, if that's ok?
as soon as this lands, there are a few refactorings that will be done, like this
one as well:
https://codereview.chromium.org/22854004/diff/1/build/android/pylib/perf/setu...

the main problem is that the perf tests will only be exercised downstream, so
I'd like this patch to stick :)

then it'll be much easier to mess around with other types of tests and change
their implementation, as long as we never need to revert the entire perf
runner... wdyt?

(oh, and while at it: ulan@ from the v8 team, says that this adb trick helped
his tests too, on a completely different environment / harness / buildbots /
etc.., so indeed, this should be shared across all our tests..).

[craigdh, 2013/08/13 19:03:46]

Sounds most excellent.

+  cmd_helper.RunCmd(['adb', 'kill-server'])
+  cmd_helper.RunCmd(['taskset', '-c', '0', 'adb', 'start-server'])
+  cmd_helper.RunCmd(['taskset', '-c', '0', 'adb', 'root'])
+  i = 1
+  while not android_commands.GetAttachedDevices():
+    time.sleep(i)
+    i *= 2
+    if i > 10:
+      break
+  # Reset the test port allocation. It's important to do it before starting
+  # to dispatch any step.
+  if not ports.ResetTestServerPortAllocation():
+    raise Exception('Failed to reset test server port.')
+
+  forwarder.Forwarder.UseMultiprocessing()
+
+
+def Setup(test_options):
+  """Create and return the test runner factory and tests.
+
+  Args:
+    test_options: A PerformanceOptions object.
+
+  Returns:
+    A tuple of (TestRunnerFactory, tests).
+  """
+  # Before running the tests, kill any leftover server.
+  _KillPendingServers()
+
+  with file(test_options.steps, 'r') as f:
+    tests = json.load(f)
+
+  flaky_steps = []
+  if test_options.flaky_steps:
+    with file(test_options.flaky_steps, 'r') as f:
+      flaky_steps = json.load(f)
+
+  def TestRunnerFactory(device, shard_index):
+    return test_runner.TestRunner(
+        test_options, device, tests, flaky_steps)
+
+  return (TestRunnerFactory, sorted(tests.keys()))