Add support for cc_perftests and other non-telemetry gtest based tests.

testing/scripts/perf_gtests.py

+#!/usr/bin/env python

[Paweł Hajdan Jr., 2015/01/30 12:16:53]

nit: Rename the file to start with "gtest_". We may have several gtest-related
scripts here, and it seems more elegant for all of them to share a common prefix
(as opposed to e.g. common suffix which may be slightly harder to notice).

[shatch, 2015/01/30 16:46:52]

Done.

+# Copyright 2014 The Chromium Authors. All rights reserved.

[Paweł Hajdan Jr., 2015/01/30 12:16:52]

nit: 2015

[shatch, 2015/01/30 16:46:52]

Done.

+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+import json
+import os
+import sys
+
+
+import common
+
+
+def main_run(args):
+  filter_tests = []
+  if args.filter_file:
+    filter_tests = json.load(args.filter_file)
+
+  perf_id = args.properties.get('perf-id')
+  script_args = args.args
+  test_suite = script_args[0]
+
+  gtest_args = [
+        '--target', args.build_config_fs,
+        '--annotate', 'graphing',
+        '--perf-id', perf_id,
+        '--perf-dashboard-id', test_suite,
+        '--results-url', args.properties.get('results-url'),
+        '--slave-name', args.properties.get('slavename'),
+        '--builder-name', args.properties.get('buildername'),
+        '--build-number', str(args.properties.get('buildnumber')),
+  ]
+
+  if 'android' in perf_id:
+    gtest_args.extend([
+        '--no-xvfb',
+        '--run-python-script', os.path.join(
+            args.paths['checkout'], 'build', 'android', 'test_runner.py'),
+        'gtest', '--release',
+        '--suite', test_suite,
+        '--verbose',
+    ])
+  else:
+    gtest_args.extend(['--xvfb', '--test-type', test_suite])
+    gtest_args.extend(script_args)
+
+  rc = common.run_runtest(args, gtest_args + filter_tests)
+
+  json.dump({
+      'valid': bool(rc <= common.MAX_FAILURES_EXIT_STATUS and rc == 0),

[Paweł Hajdan Jr., 2015/01/30 12:16:53]

Does gtest really use MAX_FAILURES_EXIT_STATUS?

[shatch, 2015/01/30 16:46:52]

Yeah you're right, was just a leftover from copying telemetry_unittests.

+      'failures': [],

[Paweł Hajdan Jr., 2015/01/30 12:16:53]

Sorry, this really *needs* to return actual list of failures.

[shatch, 2015/01/30 16:46:52]

There didn't seem to be any existing functionality to surface failures from
runtest, so I've added that in the other CL. PTAL

+  }, args.output)
+
+  return rc
+
+
+def main_compile_targets(args):
+  json.dump(['chrome'], args.output)

[Paweł Hajdan Jr., 2015/01/30 12:16:53]

This is suspicious - it seems this should request actual gtest target, rather
than 'chrome'.

[shatch, 2015/01/30 16:46:52]

The perf bots don't actually build anything, so I just made this empty.

[Paweł Hajdan Jr., 2015/01/30 21:22:39]

Sorry, I don't think it's correct. The chromium/chromium_trybot recipes can
ensure that only builders compile, but the info about targets needs to be
accurate.

[shatch, 2015/02/09 19:45:17]

Ok, I've included a list of supported gtests now.

[Paweł Hajdan Jr., 2015/02/10 08:27:03]

I believe we'll need a more sophisticated solution than this.

Let's see how this would apply to non-perf gtest tests. Since both
base_unittests and browser_tests would be "supported tests", I'd have to compile
browser_tests even if I'd only run base_unittests. The latter is very cheap to
compile, which is really not true for browser_tests.

My suggestion is to support variable substitutions in the result of compile
targets call. For example, returning '${name}' would result in variable
substituion inside ScriptTest
(build/scripts/slave/recipe_modules/chromium/steps.py). This can be done e.g.
using python template strings
(https://docs.python.org/2/library/string.html#template-strings). Furthermore,
this can handle os-specific cases like '${name}_apk' for Android.

+
+
+if __name__ == '__main__':
+  funcs = {
+    'run': main_run,
+    'compile_targets': main_compile_targets,
+  }
+  sys.exit(common.run_script(sys.argv[1:], funcs))