Add a script that can discover *.isolated files in a directory.

build/find_isolated_tests.py

+#!/usr/bin/env python
+# Copyright 2014 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+"""
+Scans build output directory for .isolated files, calculates their SHA1
+hashes, stores final list in JSON document and then removes *.isolated files
+found (to ensure no stale *.isolated stay around on the next build).
+
+Used to figure out what tests were build in isolated mode to trigger these
+tests to run on swarming.
+
+For more info see:
+https://sites.google.com/a/chromium.org/dev/developers/testing/isolated-testing
+"""
+
+import glob
+import hashlib
+import json
+import optparse
+import os
+import re
+import sys
+
+
+def hash_file(filepath):
+  """Calculates the hash of a file without reading it all in memory at once."""
+  digest = hashlib.sha1()
+  with open(filepath, 'rb') as f:
+    while True:
+      chunk = f.read(1024*1024)
+      if not chunk:
+        break
+      digest.update(chunk)
+  return digest.hexdigest()
+
+
+def main():
+  parser = optparse.OptionParser(usage='%prog path [options]')

[M-A Ruel, 2014/02/28 18:56:04]

I like description=sys.modules[__name__].__doc__, as you prefer

s/path/<build dir>/

[Vadim Sh., 2014/02/28 19:12:18]

Done.

+  parser.add_option('--output-json', help='File to dump JSON results into.')
+  options, args = parser.parse_args()
+
+  if len(args) != 1:
+    parser.error('Missing path.')

[M-A Ruel, 2014/02/28 18:56:04]

s/path/build output directory/

[Vadim Sh., 2014/02/28 19:12:18]

Done.

+  outdir = args[0]
+
+  result = {}
+
+  # Get the file hash values and output the pair.
+  for filepath in sorted(glob.glob(os.path.join(outdir, '*.isolated'))):
+    test_name = os.path.splitext(os.path.basename(filepath))[0]
+    if re.match(r'^.+?\.\d$', test_name):
+      # It's a split .isolated file, e.g. foo.0.isolated. Ignore these.
+      continue
+
+    sha1_hash = hash_file(filepath)
+    os.remove(filepath)
+    if not options.output_json:

[M-A Ruel, 2014/02/28 18:56:04]

If you don't plan to use the --output_json flag, do not write it.

[Vadim Sh., 2014/02/28 19:12:18]

That is actually exactly what I'm going to use. Maybe remove stdout then and
change cmd line to %prod <build dir> <output json>?

[M-A Ruel, 2014/02/28 19:14:00]

Ok. Note that I do like flags in general, so you could use
find_isolated_tests.py --build-dir out/Release --output-json foo.json

This is much more self-explanative at call point than having to figure out which
parameter means what.

[Vadim Sh., 2014/02/28 19:23:20]

Good idea. Done.

+      print('%s %s' % (test_name, sha1_hash))
+      # TODO(csharp): Remove both the message and the deletion once the isolate
+      # tracked dependencies are inputs for the isolated files.
+      print('Deleted %s to ensure it is regenerated by the next build.' %
+            filepath)
+    else:
+      result[test_name] = sha1_hash
+
+  if options.output_json:
+    with open(options.output_json, 'wb') as f:
+      json.dump(result, f)
+
+  return 0
+
+
+if __name__ == '__main__':
+  sys.exit(main())