on CompareGMs failure, show link to most recent results on this builder

master/skia_master_scripts/factory.py

 # Copyright (c) 2013 The Chromium Authors. All rights reserved.
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
 
 """Utility class to build the Skia master BuildFactory's.
 
 Based on gclient_factory.py and adds Skia-specific steps."""
 
 
@@ skipping 25 matching lines @@
 CONFIG_DEBUG = 'Debug'
 CONFIG_RELEASE = 'Release'
 CONFIGURATIONS = [CONFIG_DEBUG, CONFIG_RELEASE]
 
 
 _RUNGYP_STEP_DESCRIPTION = 'RunGYP'
 _COMPILE_STEP_PREFIX = 'Build'
 _COMPILE_RETRY_PREFIX = 'Retry_' + _COMPILE_STEP_PREFIX
 _COMPILE_NO_WERR_PREFIX = 'Retry_NoWarningsAsErrors_' + _COMPILE_STEP_PREFIX
 
+# EPOGER: how can I share this definition with compare_gms.py?

[borenet, 2014/06/05 14:36:10]

global_variables.json.

+LATEST_GM_FAILURES_PREAMBLE = 'View latest GM failures for this builder at: '
+
 
 class SkiaFactory(BuildFactory):
   """Encapsulates data and methods common to the Skia master.cfg files."""
 
   def __init__(self, other_repos=None, do_upload_render_results=False,
                do_upload_bench_results=False, do_patch_step=False,
                build_subdir='skia', target_platform=None,
                configuration=CONFIG_DEBUG, default_timeout=8*60*60,
                deps_target_os=None, environment_variables=None,
                perf_output_basedir=None, builder_name=None, flavor=None,
@@ skipping 468 matching lines @@
   def PostBench(self):
     """ Step to run after the benchmarking steps. """
     self.AddFlavoredSlaveScript(script='postbench.py',
                                 description='PostBench',
                                 exception_on_failure=True)
 
   def CompareGMs(self):
     """Compare the actually-generated GM images to the checked-in baselines."""
     self.AddSlaveScript(script='compare_gms.py',
                         description='CompareGMs',
+                        get_props_from_stdout={

[epoger, 2014/06/04 21:55:11]

Unfortunately, this doesn't work.

https://github.com/google/skia-buildbot/blob/master/master/skia_master_script...
only processes get_props_from_stdout if the build step succeeded.  In this case,
the URL is only useful when the test fails. :-(

See
http://c128.i.corp.google.com:10115/builders/Test-Ubuntu13.10-ShuttleA-NoGPU-...
: CompareGMs fails, and its stdio contains the link, but the build does not
display the property.

ALSO- I would rather have the URL be visible under the CompareGMs step on the
build page, rather than being a property of the build as a whole.  It's more
detail that developers want to look at in the context of a single step's
results...

[borenet, 2014/06/05 14:36:10]

We can probably change the logic in
master/skia_master_scripts/skia_build_step.py: SkiaBuildStep.commandComplete,
which currently doesn't bother checking for get_props_from_stdout when the
command failed.  Instead, we could check anyway but avoid raising an Exception
when we don't find a match IF the command failed.

I'm a little confused about what you're asking for with where the link is
displayed.  It should show up under CompareGMs in the list of steps (just like
got_revision shows up under Update) and it should show up in the table of build
properties.  Is that not sufficient?  I don't know if it'll actually be a
clickable link (my guess is no).

[epoger, 2014/06/05 15:08:43]

Ultimately, my vision is of something like
http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Win7/builds/16786
, where the webkit_tests step goes red to show you which TYPE of test failed,
and then you can click on a link right there to see what the specific failures
were.

How can we get there?  Or, is there some other model that's even better?

+                            'latest_gm_failures_url':
+                            '%s(\w+)' % LATEST_GM_FAILURES_PREAMBLE},
                         is_rebaseline_step=True)
 
   def CompareRenderedSKPs(self):
     """Compare the actual image results of SKP rendering to expectations."""
     self.AddSlaveScript(script='compare_rendered_skps.py',
                         description='CompareRenderedSKPs',
                         is_rebaseline_step=True)
 
   def RunBench(self):
     """ Run "bench", piping the output somewhere so we can graph
@@ skipping 263 matching lines @@
         # Perf-only builder.
         if not self._perf_output_basedir:
           raise ValueError(
               'BuildPerfOnly requires perf_output_basedir to be defined.')
         if self._configuration != CONFIG_RELEASE:
           raise ValueError('BuildPerfOnly should run in %s configuration.' %
                            CONFIG_RELEASE)
         self.PerfSteps()
     self.Validate()
     return self