Logdog for logcats

build/android/test_wrapper/logdog_wrapper.py

+#!/usr/bin/env python
+# Copyright 2016 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""Wrapper for adding logdog streaming support to swarming tasks."""
+
+import argparse
+import sys
+import os
+import subprocess
+import logging
+
+
+def CommandParser():
+  # Parses the command line arguments being passed in
+  parser = argparse.ArgumentParser()
+  parser.add_argument('--logdog-bin-cmd', required=True,
+                      help='Command for running logdog butler binary')
+  parser.add_argument('--project', required=True,
+                      help='Name of logdog project')
+  parser.add_argument('--logdog-server', required=True,
+                      default='luci-logdog.appspot.com',

[dnj, 2016/07/29 22:35:34]

Please use "services-dot-luci-logdog.appspot.com" as the default. The "services"
module is a more efficient backend for stream registration with the same
behavior.

[nicholaslin, 2016/07/29 23:54:06]

Done, but clicking on this seems to show an empty link?

[dnj, 2016/07/30 15:47:48]

Yeah it's a services endpoint, not a user-facing endpoint.

[nicholaslin, 2016/08/02 22:05:13]

Got it. The streams go to services-dot-luci-logdog.appspot.com and the viewable
link is still luci-logdog.appspot.com/...

+                      help='URL of logdog server, https:// is assumed.')
+  parser.add_argument('--service-account-json', required=True,
+                      help='Location of authentication json')
+  parser.add_argument('--prefix', required=True,
+                      help='Prefix to be used for logdog stream')
+  parser.add_argument('--source', required=True,
+                      help='Location of file for logdog to stream')
+  parser.add_argument('--name', required=True,
+                      help='Name to be used for logdog stream')
+  return parser
+
+
+def CreateUrl(server, project, prefix, name):
+  url_prefix = prefix.replace('/', '%2F')
+  return ('https://%s/v/?s=%s%%2F%s%%2F%%2B%%2F%s'

[ghost stip (do not use), 2016/07/29 21:56:40]

lmao

[dnj, 2016/07/29 22:35:34]

Er can we use something like urllib.quote
(https://docs.python.org/2/library/urllib.html#urllib.quote) for the query
string part of this?

[nicholaslin, 2016/07/29 23:54:06]

Done.

+          % (server, project, url_prefix, name))
+
+
+def main():
+  parser = CommandParser()
+  args, test_cmd = parser.parse_known_args(sys.argv[1:])
+  if not test_cmd:
+    parser.error('Must specify command to run after the logdog flags')
+  subprocess.check_call(test_cmd)

[ghost stip (do not use), 2016/07/29 21:56:40]

we should do result = subprocess.call(test_cmd)

and then later return result

that way the test wrapper always returns the return value of the test, not of
the wrapper or logdog.

[nicholaslin, 2016/07/29 23:54:06]

Done.

+  if '${SWARMING_TASK_ID}' in args.prefix:
+    args.prefix = args.prefix.replace('${SWARMING_TASK_ID}',
+                                      os.environ.get('SWARMING_TASK_ID'))
+  url = CreateUrl(args.logdog_server, args.project, args.prefix, args.name)
+  logging.info('Logcats are located at: %s', url)
+  logdog_cmd = [args.logdog_bin_cmd, '-project', args.project,
+                '-output', 'logdog,host=%s' % args.logdog_server,
+                '-prefix', args.prefix,
+                '-service-account-json', args.service_accont_json,
+                'stream', '-source', args.source,

[dnj, 2016/07/29 22:35:34]

I see we gave up on the "serve" approach w/ the library. That's a shame :( We
lose all sorts of cool stuff, like:
* Accurate timestamps
* Live streaming of the logs

If you're not using either of those, maybe consider not using LogDog at all and
just uploading the files to Google Storage *shrug*

+                '-stream', '-name=%s' % args.name]
+  return subprocess.call(logdog_cmd)
+
+
+if __name__ == '__main__':
+  sys.exit(main())