[Android] Add support for external sharding to the test runner.

build/android/pylib/local/device/local_device_test_run.py

 # Copyright 2014 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 import fnmatch
 import imp
 import logging
 import posixpath
 import signal
 import thread
@@ skipping 159 matching lines @@
       # See local_device_gtest_run._ExtractTestsFromFilter()
       if name.endswith('*'):
         return any(fnmatch.fnmatch(n, name) and is_failure_result(t)
                    for n, t in all_test_results.iteritems())
       return is_failure_result(all_test_results.get(name))
 
     failed_tests = (t for t in tests if test_failed(self._GetUniqueTestName(t)))
 
     return [t for t in failed_tests if self._ShouldRetry(t)]
 
+  def _ApplyExternalSharding(self, tests, shard_index, total_shards):

[bpastene, 2017/03/14 01:03:46]

It might be helpful to log the shard args here. I know the base test launcher
prints something like:
"This is shard 1/5"

[jbudorick, 2017/03/14 15:36:01]

Done.

+    return [

[bpastene, 2017/03/14 01:03:46]

Any interest in complaining and/or exploding if shard_index > total_shards?

[jbudorick, 2017/03/14 15:36:00]

Done.

+        t for t in tests
+        if hash(self._GetUniqueTestName(t)) % total_shards == shard_index]

[mikecase (-- gone --), 2017/03/13 18:44:50]

I was curious how uniform the distribution of "hash(<typical test names>) %
SHARDS" would be. Here are the results I got for running against a set of Chrome
test names.

2 shards: [830, 797]
3 shards: [528, 534, 565]
4 shards: [403, 391, 427, 406]
5 shards: [333, 312, 328, 313, 341]
6 shards: [275, 257, 278, 253, 277, 287]
7 shards: [237, 226, 217, 232, 230, 249, 236]

The shard sizes are pretty even. But you could get them perfectly even by just
alphabetizing the test names, then just modding the array indices. Your approach
I guess is more robust if some shards for some reason have a different set of
tests?

[jbudorick, 2017/03/13 19:02:20]

What I have here is also:
 - consistent with how base_test_launcher handles it (% hash algorithm, which I
didn't bother investigating):
https://codesearch.chromium.org/chromium/src/base/test/launcher/test_launcher...
 - stable -- holding hash method and total number of shards constant, a given
test will always get mapped to the same shard

+
   def GetTool(self, device):
     if not str(device) in self._tools:
       self._tools[str(device)] = valgrind_tools.CreateTool(
           self._env.tool, device)
     return self._tools[str(device)]
 
   def _CreateShards(self, tests):
     raise NotImplementedError
 
   def _GetUniqueTestName(self, test):
     # pylint: disable=no-self-use
     return test
 
   def _ShouldRetry(self, test):
     # pylint: disable=no-self-use,unused-argument
     return True
 
   def _GetTests(self):
     raise NotImplementedError
 
   def _RunTest(self, device, test):
     raise NotImplementedError
 
   def _ShouldShard(self):
     raise NotImplementedError
 
 
 class NoTestsError(Exception):
   """Error for when no tests are found."""