telmetry: Add SkpicturePrinter benchmark.

telmetry: Add SkpicturePrinter benchmark.

Support for running measurements without a benchmark has been removed
from telemetry. This patch adds a benchmark that allows running
SkpicturePrinter measurement on arbitrary page sets, as required
for the Skia buildbots.

R=dtu@chromium.org,rmistry@chromium.org
BUG=425074
Committed: https://crrev.com/ba2fd136160971b66e8f15ced71ba91152b1f59b
Cr-Commit-Position: refs/heads/master@{#300763}

[ernstm, 2014-10-20 23:43:10]

PTAL

[rmistry, 2014-10-21 00:11:48]

rmistry@google.com changed reviewers:
+ rmistry@google.com

[rmistry, 2014-10-21 00:11:49]

Thanks Manfred. Will this also allow the use of --skp_dir (which was allowed by
the original measurement)?

[ernstm, 2014-10-21 00:53:04]

On 2014/10/21 00:11:49, rmistry wrote:
> Thanks Manfred. Will this also allow the use of --skp_dir (which was allowed
by
> the original measurement)?

Yes, --skp-outdir is still supported and required. The measurement adds that
option to the command line parser.

[rmistry, 2014-10-21 00:57:16]

LGTM

[ernstm, 2014-10-21 01:08:03]

The CQ bit was checked by ernstm@chromium.org

[commit-bot: I haz the power, 2014-10-21 01:09:10]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665193002/20001

[commit-bot: I haz the power, 2014-10-21 01:20:29]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-10-21 01:20:34]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[rmistry, 2014-10-21 11:03:48]

On 2014/10/21 01:20:34, I haz the power (commit-bot) wrote:
> Try jobs failed on following builders:
>   chromium_presubmit on tryserver.chromium.linux
>
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

Looks like presubmit failed:

** Presubmit Messages **
Pylint (172 files) (9.28s)

** Presubmit Warnings **
License must match:
.*? Copyright (\(c\)
)?(2014|2013|2012|2011|2010|2009|2008|2007|2006|2006-2008|2006-2009|2006-2010)
The Chromium Authors\. All rights reserved\.\n.*? Use of this source code is
governed by a BSD-style license that can be\n.*? found in the LICENSE file\.(?:
\*/)?\n
Found a bad license header in these files:
  tools/perf/benchmarks/skpicture_printer.py

Presubmit checks took 9.9s to calculate.

[ernstm, 2014-10-21 16:42:22]

On 2014/10/21 11:03:48, rmistry wrote:
> On 2014/10/21 01:20:34, I haz the power (commit-bot) wrote:
> > Try jobs failed on following builders:
> >   chromium_presubmit on tryserver.chromium.linux
> >
>
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
> 
> Looks like presubmit failed:
> 
> ** Presubmit Messages **
> Pylint (172 files) (9.28s)
> 
> ** Presubmit Warnings **
> License must match:
> .*? Copyright (\(c\)
> )?(2014|2013|2012|2011|2010|2009|2008|2007|2006|2006-2008|2006-2009|2006-2010)
> The Chromium Authors\. All rights reserved\.\n.*? Use of this source code is
> governed by a BSD-style license that can be\n.*? found in the LICENSE
file\.(?:
> \*/)?\n
> Found a bad license header in these files:
>   tools/perf/benchmarks/skpicture_printer.py
> 
> Presubmit checks took 9.9s to calculate.

The license header was missing.

[ernstm, 2014-10-21 16:42:26]

The CQ bit was checked by ernstm@chromium.org

[commit-bot: I haz the power, 2014-10-21 16:44:23]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665193002/40001

[commit-bot: I haz the power, 2014-10-21 18:12:49]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-10-21 18:12:50]

Try jobs failed on following builders:
  linux_chromium_rel_swarming on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)

[rmistry, 2014-10-21 20:22:42]

Looks like there was a failed test:

1 FAILED TEST

[----------] 1 test
[ RUN      ] BenchmarkSmokeTest.skpicture_printer

Traceback (most recent call last):
  run at
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py:331
    testMethod()
  wrapper at tools/telemetry/telemetry/decorators.py:55
    func(*args, **kwargs)
  BenchmarkSmokeTest at tools/perf/benchmarks/benchmark_smoke_unittest.py:56
    benchmark.ProcessCommandLineArgs(None, options)
  ProcessCommandLineArgs at tools/perf/benchmarks/skpicture_printer.py:33
    cls.PageTestClass().ProcessCommandLineArgs(parser, args)
  ProcessCommandLineArgs at tools/perf/measurements/skpicture_printer.py:23
    parser.error('Please specify --skp-outdir')
AttributeError: 'NoneType' object has no attribute 'error'

[rmistry, 2014-10-22 11:53:19]

On 2014/10/21 20:22:42, rmistry wrote:
> Looks like there was a failed test:
> 
> 1 FAILED TEST
> 
> [----------] 1 test
> [ RUN      ] BenchmarkSmokeTest.skpicture_printer
> 
> Traceback (most recent call last):
>   run at
>
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py:331
>     testMethod()
>   wrapper at tools/telemetry/telemetry/decorators.py:55
>     func(*args, **kwargs)
>   BenchmarkSmokeTest at tools/perf/benchmarks/benchmark_smoke_unittest.py:56
>     benchmark.ProcessCommandLineArgs(None, options)
>   ProcessCommandLineArgs at tools/perf/benchmarks/skpicture_printer.py:33
>     cls.PageTestClass().ProcessCommandLineArgs(parser, args)
>   ProcessCommandLineArgs at tools/perf/measurements/skpicture_printer.py:23
>     parser.error('Please specify --skp-outdir')
> AttributeError: 'NoneType' object has no attribute 'error'

Friendly ping for this CL since the Skia bot is broken

[ernstm, 2014-10-22 18:02:47]

On 2014/10/22 11:53:19, rmistry wrote:
> On 2014/10/21 20:22:42, rmistry wrote:
> > Looks like there was a failed test:
> > 
> > 1 FAILED TEST
> > 
> > [----------] 1 test
> > [ RUN      ] BenchmarkSmokeTest.skpicture_printer
> > 
> > Traceback (most recent call last):
> >   run at
> >
>
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py:331
> >     testMethod()
> >   wrapper at tools/telemetry/telemetry/decorators.py:55
> >     func(*args, **kwargs)
> >   BenchmarkSmokeTest at tools/perf/benchmarks/benchmark_smoke_unittest.py:56
> >     benchmark.ProcessCommandLineArgs(None, options)
> >   ProcessCommandLineArgs at tools/perf/benchmarks/skpicture_printer.py:33
> >     cls.PageTestClass().ProcessCommandLineArgs(parser, args)
> >   ProcessCommandLineArgs at tools/perf/measurements/skpicture_printer.py:23
> >     parser.error('Please specify --skp-outdir')
> > AttributeError: 'NoneType' object has no attribute 'error'
> 
> Friendly ping for this CL since the Skia bot is broken

This benchmark can't be tested with benchmark_smoke_unittest, because this
assumes that default values exist for all options. But SkpicturePrinter
benchmark must get the page set and the output directory from the command line.
We do have coverage for the measurement though and therefore I excluded the
benchmark from the smoke test.

[ernstm, 2014-10-22 18:02:52]

The CQ bit was checked by ernstm@chromium.org

[commit-bot: I haz the power, 2014-10-22 18:04:28]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665193002/60001

[commit-bot: I haz the power, 2014-10-22 19:34:42]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-10-22 19:34:47]

Try jobs failed on following builders:
  win_chromium_x64_rel on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)

[rmistry, 2014-10-22 20:05:57]

On 2014/10/22 19:34:47, I haz the power (commit-bot) wrote:
> Try jobs failed on following builders:
>   win_chromium_x64_rel on tryserver.chromium.win
>
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_...)

Maybe, try again with

CQ_EXCLUDE_TRYBOTS=tryserver.chromium.win:win_chromium_x64_rel

?

[ernstm, 2014-10-22 20:32:12]

The CQ bit was checked by ernstm@chromium.org

[commit-bot: I haz the power, 2014-10-22 20:33:13]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/665193002/60001

[commit-bot: I haz the power, 2014-10-22 21:25:34]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2014-10-22 21:26:08]

Patchset 4 (id:??) landed as
https://crrev.com/ba2fd136160971b66e8f15ced71ba91152b1f59b
Cr-Commit-Position: refs/heads/master@{#300763}