Support filling in baselines from other platforms in rebaseline-cl.

third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py

Index: third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
diff --git a/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py b/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
index 71349d22c8602344f4363c4e5685f84e6b9f6872..85834fe84a5c7ac0d321617c116c5cc52ca57b5c 100644
--- a/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
+++ b/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline_cl.py
@@ -36,6 +36,9 @@ class RebaselineCL(AbstractParallelRebaselineCommand):
             optparse.make_option(
                 '--no-trigger-jobs', dest='trigger_jobs', action='store_false', default=True,
                 help='Do not trigger any try jobs.'),
+            optparse.make_option(
+                '--fill-missing', dest='fill_missing', action='store_true', default=False,
+                help='If some builds have no results available, use results from other builds.'),

[wkorman, 2017/04/10 20:12:27]

This reads as if the port will be the same and the only thing we are potentially
doing is falling back to an earlier build if it had suitable baseline. But I
think we could in fact switch ports? Perhaps worth clarifying documentation.

[qyearsley, 2017/04/10 23:52:09]

Ah, yes, not just "use results from other builds", but "use results from other
platforms' try jobs".

Done.

             self.no_optimize_option,
             self.results_directory_option,
         ])
@@ -73,7 +76,7 @@ class RebaselineCL(AbstractParallelRebaselineCommand):
             _log.info('Please re-run webkit-patch rebaseline-cl once all pending try jobs have finished.')
             return 1
 
-        if builders_with_no_results:
+        if builders_with_no_results and not options.fill_missing:
             # TODO(qyearsley): Support trying to continue as long as there are
             # some results from some builder; see http://crbug.com/673966.
             _log.error('The following builders have no results:')
@@ -96,6 +99,9 @@ class RebaselineCL(AbstractParallelRebaselineCommand):
                 builds_to_results,
                 only_changed_tests=options.only_changed_tests)
 
+        if options.fill_missing:
+            self.fill_in_missing_results(test_baseline_set)
+
         _log.debug('Rebaselining: %s', test_baseline_set)
 
         if not options.dry_run:
@@ -220,3 +226,22 @@ class RebaselineCL(AbstractParallelRebaselineCommand):
         except (ValueError, KeyError):
             _log.warning('Unexpected retry summary content:\n%s', content)
             return None
+
+    def fill_in_missing_results(self, test_baseline_set):
+        """Modifies test_baseline_set to guarantee that there are entries for
+        all ports for all tests to rebaseline.
+
+        For a each test prefix, if there is entry for some port, then an entry
+        should be added
+        """
+        all_ports = {self._tool.builders.port_name_for_builder_name(b) for b in self._try_bots()}
+        for test_prefix in test_baseline_set.test_prefixes():
+            build_port_pairs = test_baseline_set.build_port_pairs(test_prefix)
+            missing_ports = all_ports - {p for _, p in build_port_pairs}
+            for port in missing_ports:
+                # TODO(qyearsley): Decide what build to use for a given port

[wkorman, 2017/04/10 20:12:27]

Just looking to make sure I understand, is this noted work basically revising to
incorporate the same/similar platform (port) fallback logic that we use for
optimizing baselines?

[qyearsley, 2017/04/10 23:52:10]

Something similar, but I believe we can't entirely rely on the fallback graph.

For example, suppose we just have results for win7. The baseline fallback path
for linux-trusty is ["linux", "win"], and "win" corresponds to win10. Similarly,
the baseline fallback path for mac10.10 is ["mac-mac10.10", "mac-mac10.11",
"mac"], which doesn't include any windows platforms.

Fallback graph:
https://drive.google.com/a/google.com/file/d/0Bzxz6fWz5JOIWlVMQkJXc1Bjd2M/vie...

So I guess one approach would be to start with the fallback path, but then if
there are no available builds for any ports there, then just use any build?

One approach would be to construct a dict of port to list of other ports
starting with similar ports just by listing all the supported versions, e.g.:


def similar_ports_dict():
  android = ['android-' + v for v in AndroidPort.SUPPORTED_VERSIONS]
  linux = ['linux-' + v for v in LinuxPort.SUPPORTED_VERSIONS]
  mac = ['mac-' + v for v in MacPort.SUPPORTED_VERSIONS]
  win = ['win-' + v for v in WinPort.SUPPORTED_VERSIONS]
  similar_ports = {}
  for v in android:
    similar_ports[v] = android + linux + mac + win
  for v in linux:
    similar_ports[v] = linux + android + mac + win
  for v in mac:
    similar_ports[v] = mac + linux + android + win
  for v in win:
    similar_ports[v] = win + linux + android + mac
  return similar_ports

My main thought here is just that, for example, if we just need to fill in one
mac port, then we should default to using another mac result first; but if there
are no other mac builds available, then probably trying to use any result is
OK...


Does that sound roughly correct? I'm hoping I can use something a little simpler
than the above.

[wkorman, 2017/04/11 20:36:49]

Yes, something like this sounds correct to me. Perhaps checkpoint with drott@ as
well for another opinion since he spends a lot of time with cross platform font
baselines. Doing this as a separate change makes sense to me.

[qyearsley, 2017/04/11 23:23:21]

Sounds good, will do :-)

+                # in a more sophisticated way, such that a build with a
+                # "similar" port will be used when available.
+                build = build_port_pairs[0][0]
+                test_baseline_set.add(test_prefix, build, port)
+        return test_baseline_set