Shard & set hard timeout for telemetry_perf_unittests

Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
Committed: https://crrev.com/a5bbbe202620b37a441f5de407419ea5aff33231
Cr-Commit-Position: refs/heads/master@{#401309}

[nednguyen, 2016-06-22 05:35:02]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq
==========

[nednguyen, 2016-06-22 05:35:15]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

[nednguyen, 2016-06-22 05:40:59]

nednguyen@google.com changed reviewers:
+ dpranke@chromium.org, kbr@chromium.org

[nednguyen, 2016-06-22 05:53:10]

On 2016/06/22 05:41:00, nednguyen wrote:

Verified that injected failure shown up in
https://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel...

PTAL

[nednguyen, 2016-06-22 05:55:12]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq
==========

[Dirk Pranke, 2016-06-22 16:13:45]

lgtm

[nednguyen, 2016-06-22 16:15:12]

The CQ bit was checked by nednguyen@google.com

[commit-bot: I haz the power, 2016-06-22 16:15:46]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/2089633003/60001

[nednguyen, 2016-06-22 16:18:55]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:winx64_10_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

[commit-bot: I haz the power, 2016-06-22 16:59:59]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

[commit-bot: I haz the power, 2016-06-22 17:00:00]

Committed patchset #4 (id:60001)

[commit-bot: I haz the power, 2016-06-22 17:01:46]

Description was changed from

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959
==========

to

==========
Shard & set hard timeout for telemetry_perf_unittests

BUG=620959

Committed: https://crrev.com/a5bbbe202620b37a441f5de407419ea5aff33231
Cr-Commit-Position: refs/heads/master@{#401309}
==========

[commit-bot: I haz the power, 2016-06-22 17:01:48]

Patchset 4 (id:??) landed as
https://crrev.com/a5bbbe202620b37a441f5de407419ea5aff33231
Cr-Commit-Position: refs/heads/master@{#401309}

[jbudorick, 2016-06-23 13:13:54]

jbudorick@chromium.org changed reviewers:
+ jbudorick@chromium.org, stip@chromium.org

[jbudorick, 2016-06-23 13:13:55]

+stip

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
File testing/buildbot/chromium.linux.json (right):

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
testing/buildbot/chromium.linux.json:624: "shards": 12
What was the motivation here?

This is _really_ problematic for capacity & should likely be backed out.

[nednguyen, 2016-06-23 13:18:28]

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
File testing/buildbot/chromium.linux.json (right):

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
testing/buildbot/chromium.linux.json:624: "shards": 12
On 2016/06/23 13:13:55, jbudorick (EMEA til June 30) wrote:
> What was the motivation here?
> 
> This is _really_ problematic for capacity & should likely be backed out.

Oh, this is just meant to make it consistent with other telemetry configurations
on other platform. Do we have capacity monitoring? I can adjust the number of
tests run to keep it under our budget.

[jbudorick, 2016-06-23 13:37:37]

On 2016/06/23 13:18:28, nednguyen wrote:
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> File testing/buildbot/chromium.linux.json (right):
> 
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> testing/buildbot/chromium.linux.json:624: "shards": 12
> On 2016/06/23 13:13:55, jbudorick (EMEA til June 30) wrote:
> > What was the motivation here?
> > 
> > This is _really_ problematic for capacity & should likely be backed out.
> 
> Oh, this is just meant to make it consistent with other telemetry
configurations
> on other platform. Do we have capacity monitoring? I can adjust the number of
> tests run to keep it under our budget.

I'll let stip answer the monitoring one, but the answer right now is that this
should not be sharded on android.

[jbudorick, 2016-06-23 13:39:18]

On 2016/06/23 13:37:37, jbudorick (EMEA til June 30) wrote:
> On 2016/06/23 13:18:28, nednguyen wrote:
> >
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> > File testing/buildbot/chromium.linux.json (right):
> > 
> >
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> > testing/buildbot/chromium.linux.json:624: "shards": 12
> > On 2016/06/23 13:13:55, jbudorick (EMEA til June 30) wrote:
> > > What was the motivation here?
> > > 
> > > This is _really_ problematic for capacity & should likely be backed out.
> > 
> > Oh, this is just meant to make it consistent with other telemetry
> configurations
> > on other platform. Do we have capacity monitoring? I can adjust the number
of
> > tests run to keep it under our budget.
> 
> I'll let stip answer the monitoring one, but the answer right now is that this
> should not be sharded on android.

To elaborate a bit more on why: without sharding, this suite takes ~ 5 minutes.
With sharding, each shard takes ~ 2.5 minutes. If even we could support that, it
seems like a flaw in the suite, though I'm not particularly familiar with the
suite.

[ghost stip (do not use), 2016-06-23 14:03:58]

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
File testing/buildbot/chromium.linux.json (right):

https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
testing/buildbot/chromium.linux.json:624: "shards": 12
On 2016/06/23 13:18:28, nednguyen wrote:
> On 2016/06/23 13:13:55, jbudorick (EMEA til June 30) wrote:
> > What was the motivation here?
> > 
> > This is _really_ problematic for capacity & should likely be backed out.
> 
> Oh, this is just meant to make it consistent with other telemetry
configurations
> on other platform. Do we have capacity monitoring? I can adjust the number of
> tests run to keep it under our budget.

we are currently at negative capacity. we need to crank this back to 1 until we
stabilize.

[nednguyen(REVIEW IN OTHER ACC), 2016-06-23 14:55:12]

On 2016/06/23 14:03:58, stip wrote:
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> File testing/buildbot/chromium.linux.json (right):
> 
>
https://codereview.chromium.org/2089633003/diff/60001/testing/buildbot/chromi...
> testing/buildbot/chromium.linux.json:624: "shards": 12
> On 2016/06/23 13:18:28, nednguyen wrote:
> > On 2016/06/23 13:13:55, jbudorick (EMEA til June 30) wrote:
> > > What was the motivation here?
> > > 
> > > This is _really_ problematic for capacity & should likely be backed out.
> > 
> > Oh, this is just meant to make it consistent with other telemetry
> configurations
> > on other platform. Do we have capacity monitoring? I can adjust the number
of
> > tests run to keep it under our budget.
> 
> we are currently at negative capacity. we need to crank this back to 1 until
we
> stabilize.

Will do it in another CL.