[perf] Add speedometer-ignition benchmark

[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq

Committed: https://crrev.com/0a791019658bc90c48afb894f3481952ac403c5f
Cr-Commit-Position: refs/heads/master@{#387921}

[Camillo Bruni, 2016-04-05 14:33:41]

Description was changed from

==========
[perf] Adding speedometer-ignition benchmark

BUG=
==========

to

==========
[perf] Adding speedometer-ignition benchmark

BUG=
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

[Camillo Bruni, 2016-04-05 14:33:58]

The CQ bit was checked by cbruni@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-04-05 14:34:18]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/1
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/1

[Camillo Bruni, 2016-04-05 14:35:49]

cbruni@chromium.org changed reviewers:
+ nednguyen@google.com, samistil@chromium.org

[Camillo Bruni, 2016-04-05 14:35:49]

PTAL
We figure that we should have the speedometer benchmark running with --ignition
on (our interpreter) to estimate the impact of our changes.

[nednguyen, 2016-04-05 14:43:16]

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/speed...
File tools/perf/benchmarks/speedometer.py (right):

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/speed...
tools/perf/benchmarks/speedometer.py:31: import v8_helper
Did you forgot to check in this file? I can't find it in code search:
https://code.google.com/p/chromium/codesearch#search/&q=v8_helper.py&sq=packa...

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/v8.py
File tools/perf/benchmarks/v8.py (right):

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/v8.py...
tools/perf/benchmarks/v8.py:26: def
CreateV8TimelineBasedMeasurementOptions(benchmark):
You don't need the benchmark param. Did the pylint warn about unused param? If
not, lemme know since it's probably a bug.

Also to remove the boiler plates in benchmarks that use this, you can create s.t
like _TimelineBasedV8Benchmark with CreateTimelineBasedMeasurementOptions(self)
being this one & have your other v8 benchmarks subclass it.

[commit-bot: I haz the power, 2016-04-05 14:43:53]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-05 14:43:54]

Dry run: Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[Camillo Bruni, 2016-04-06 08:10:25]

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/speed...
File tools/perf/benchmarks/speedometer.py (right):

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/speed...
tools/perf/benchmarks/speedometer.py:31: import v8_helper
On 2016/04/05 at 14:43:16, nednguyen wrote:
> Did you forgot to check in this file? I can't find it in code search:
https://code.google.com/p/chromium/codesearch#search/&q=v8_helper.py&sq=packa...

bah... yes :D

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/v8.py
File tools/perf/benchmarks/v8.py (right):

https://codereview.chromium.org/1860013002/diff/1/tools/perf/benchmarks/v8.py...
tools/perf/benchmarks/v8.py:26: def
CreateV8TimelineBasedMeasurementOptions(benchmark):
On 2016/04/05 at 14:43:16, nednguyen wrote:
> You don't need the benchmark param. Did the pylint warn about unused param? If
not, lemme know since it's probably a bug.
> 
> Also to remove the boiler plates in benchmarks that use this, you can create
s.t like _TimelineBasedV8Benchmark with
CreateTimelineBasedMeasurementOptions(self) being this one & have your other v8
benchmarks subclass it.

since I accidentally created v8_helper.py in the current working directory the
pylint messages didn't make sense, corrected that now.

[rmcilroy, 2016-04-06 08:35:12]

rmcilroy@chromium.org changed reviewers:
+ rmcilroy@chromium.org

[rmcilroy, 2016-04-06 08:35:12]

+1 to adding Speedometer (thanks!) but could we fix the V8 execution metrics
before adding more benchmarks that depend
on them? The numbers for compiling / parsing / execution are completely bogus
due to crbug.com/596405.

[Camillo Bruni, 2016-04-06 09:18:26]

On 2016/04/06 at 08:35:12, rmcilroy wrote:
> +1 to adding Speedometer (thanks!) but could we fix the V8 execution metrics
before adding more benchmarks that depend
> on them? The numbers for compiling / parsing / execution are completely bogus
due to crbug.com/596405.

This is only for the speedometer results, not our trace-based telemetry
measurements.
TBM v2. is in the pipeline, see
https://github.com/catapult-project/catapult/issues/2190

[rmcilroy, 2016-04-06 10:18:41]

On 2016/04/06 09:18:26, Camillo Bruni wrote:
> On 2016/04/06 at 08:35:12, rmcilroy wrote:
> > +1 to adding Speedometer (thanks!) but could we fix the V8 execution metrics
> before adding more benchmarks that depend
> > on them? The numbers for compiling / parsing / execution are completely
bogus
> due to crbug.com/596405.
> 
> This is only for the speedometer results, not our trace-based telemetry
> measurements.
> TBM v2. is in the pipeline, see
> https://github.com/catapult-project/catapult/issues/2190

Ahh right, sounds good - I mistook the changes in v8.py as being related to the
speedometer benchmark, not just refactoring related to pulling out the ignition
option. Thanks.

[Camillo Bruni, 2016-04-07 09:33:19]

PTAL again

[Camillo Bruni, 2016-04-07 09:33:24]

The CQ bit was checked by cbruni@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-04-07 09:33:36]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/20001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/20001

[commit-bot: I haz the power, 2016-04-07 09:35:41]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-07 09:35:42]

Dry run: Try jobs failed on following builders:
  mac_chromium_compile_dbg_ng on tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_comp...)
  mac_chromium_gn_rel on tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_gn_r...)
  mac_chromium_rel_ng on tryserver.chromium.mac (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)

[Camillo Bruni, 2016-04-07 13:20:37]

The CQ bit was checked by cbruni@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-04-07 13:20:52]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/40001

[nednguyen, 2016-04-07 14:52:15]

On 2016/04/07 13:20:52, commit-bot: I haz the power wrote:
> Dry run: CQ is trying da patch. Follow status at
>  https://chromium-cq-status.appspot.com/patch-status/1860013002/40001
> View timeline at
>  https://chromium-cq-status.appspot.com/patch-timeline/1860013002/40001

lgtm

Nits on the description title: "Add" instead of "Adding"

[Camillo Bruni, 2016-04-07 14:53:32]

Description was changed from

==========
[perf] Adding speedometer-ignition benchmark

BUG=
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

to

==========
[perf] Add speedometer-ignition benchmark

BUG=
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

[Camillo Bruni, 2016-04-07 14:54:07]

updated description title

[commit-bot: I haz the power, 2016-04-07 17:37:57]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-07 17:37:58]

Dry run: Try jobs failed on following builders:
  android_s5_perf_cq on tryserver.chromium.perf (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)

[Camillo Bruni, 2016-04-08 07:38:14]

The CQ bit was checked by cbruni@chromium.org

[commit-bot: I haz the power, 2016-04-08 07:38:30]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/40001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/40001

[commit-bot: I haz the power, 2016-04-08 10:00:38]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-08 10:00:40]

Try jobs failed on following builders:
  android_s5_perf_cq on tryserver.chromium.perf (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)

[Camillo Bruni, 2016-04-08 11:55:37]

On 2016/04/08 at 10:00:40, commit-bot wrote:
> Try jobs failed on following builders:
>   android_s5_perf_cq on tryserver.chromium.perf (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)


Do you have any clue why the android build would time out? 
I noticed locally that the TBM v2 run a bit slower on longer traces and the
adwords benchmark is rather big.

[Camillo Bruni, 2016-04-08 13:09:55]

Description was changed from

==========
[perf] Add speedometer-ignition benchmark

BUG=
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

to

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

[Camillo Bruni, 2016-04-15 09:52:41]

The CQ bit was checked by cbruni@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-04-15 09:52:50]

Dry run: CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/60001

[commit-bot: I haz the power, 2016-04-15 14:01:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-15 14:01:54]

Dry run: Try jobs failed on following builders:
  android_s5_perf_cq on tryserver.chromium.perf (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)

[nednguyen, 2016-04-15 15:07:03]

On 2016/04/15 14:01:54, commit-bot: I haz the power wrote:
> Dry run: Try jobs failed on following builders:
>   android_s5_perf_cq on tryserver.chromium.perf (JOB_FAILED,
>
http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_...)

If you think the failure of android_s5_perf_cq is unrelated, I would just remove
it from the CQ_EXTRA_TRYBOT list

[Camillo Bruni, 2016-04-18 08:31:00]

Description was changed from

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

to

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

[Camillo Bruni, 2016-04-18 08:31:05]

The CQ bit was checked by cbruni@chromium.org

[Camillo Bruni, 2016-04-18 08:31:06]

The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com
Link to the patchset: https://codereview.chromium.org/1860013002/#ps60001
(title: "merge master")

[commit-bot: I haz the power, 2016-04-18 08:31:12]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/60001

[commit-bot: I haz the power, 2016-04-18 11:14:46]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-04-18 11:14:47]

Try jobs failed on following builders:
  winx64_10_perf_cq on tryserver.chromium.perf (JOB_FAILED,
http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_10_perf_c...)

[nednguyen, 2016-04-18 14:48:34]

The CQ bit was checked by nednguyen@google.com

[commit-bot: I haz the power, 2016-04-18 14:48:45]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1860013002/60001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1860013002/60001

[nednguyen, 2016-04-18 14:55:34]

On 2016/04/18 14:48:45, commit-bot: I haz the power wrote:
> CQ is trying da patch. Follow status at
>  https://chromium-cq-status.appspot.com/patch-status/1860013002/60001
> View timeline at
>  https://chromium-cq-status.appspot.com/patch-timeline/1860013002/60001

If perf:winx64_10_perf_cq is causing problem again, I would also remove them.
Sorry for the annoyance!

[Camillo Bruni, 2016-04-18 16:00:54]

Description was changed from

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

to

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

[commit-bot: I haz the power, 2016-04-18 16:02:18]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2016-04-18 16:04:01]

Description was changed from

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq
==========

to

==========
[perf] Add speedometer-ignition benchmark

BUG=chromium:593793
LOG=n
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:mac_retina_perf_cq;tryserver.chromium.perf:linux_perf_cq

Committed: https://crrev.com/0a791019658bc90c48afb894f3481952ac403c5f
Cr-Commit-Position: refs/heads/master@{#387921}
==========

[commit-bot: I haz the power, 2016-04-18 16:04:04]

Patchset 4 (id:??) landed as
https://crrev.com/0a791019658bc90c48afb894f3481952ac403c5f
Cr-Commit-Position: refs/heads/master@{#387921}