Kill endure benchmark and all that related to it

Kill endure benchmark and all that related to it.

BUG=

Committed: https://crrev.com/83dc32e0417c1c99b989f3ba0f277e28890d8799
Cr-Commit-Position: refs/heads/master@{#299008}

[nednguyen, 2014-10-01 18:41:51]

nednguyen@google.com changed reviewers:
+ chrishenry@google.com, qyearsley@chromium.org, tonyg@chromium.org

[nednguyen, 2014-10-01 18:45:07]

Patchset #1 (id:1) has been deleted

[nednguyen, 2014-10-01 18:45:19]

nednguyen@google.com changed reviewers:
+ kbr@chromium.org

[qyearsley, 2014-10-01 18:56:10]

In this CL, do you also want to remove the other endure-related code? e.g.
measurements/endure.py and all of the related page sets?

[tonyg, 2014-10-01 19:31:41]

How much is --page-repeat really in the way? Being able to run AAABBBCCC vs
ABCABCABC can trigger very different performance cases when you consider all of
the things that Chrome caches. I use the flag pretty often locally, but am not
sure how common that is. Can you explain more about the rationale for removing?

ps. +1 to remove everything related to endure.

[Ken Russell (switch to Gerrit), 2014-10-01 19:59:46]

The GPU changes LGTM but I agree with Tony that page-repeat itself seems useful.
I use it periodically for local testing though could also use pageset-repeat.

[nednguyen, 2014-10-01 20:23:17]

On 2014/10/01 19:31:41, tonyg wrote:
> How much is --page-repeat really in the way? Being able to run AAABBBCCC vs
> ABCABCABC can trigger very different performance cases when you consider all
of
> the things that Chrome caches. I use the flag pretty often locally, but am not
> sure how common that is. Can you explain more about the rationale for
removing?

If users explicitly want AAABBBCC, they can either duplicate adding the page
sets in PageRunner.__init__ or write for loop in Page.Run() method.

I find supporting --page-repeat complicate page_runner's logic. 

For example, assuming the runs are A1 A2 A3 B1 B2 B3

1) If A3 fails, should the "retry at most X times" logic be applied for
rerunning A3 only or should we repeat another 3 runs of page A? 
2) Same question as A3, if the test requirement is warm start.
3) The results system have logic for merging results of same page. If A2 fail,
do we consider the merged value for page A a failure value or average of A1 &
A3?

It's not trivial to provide one right answer for each of these questions. User
can get the behavior they want explicitly either by duplicating adding the page
sets in PageRunner.__init__ or making a for loop in Page.Run() method.

> 
> ps. +1 to remove everything related to endure.

[nednguyen, 2014-10-02 17:45:41]

On 2014/10/01 20:23:17, nednguyen wrote:
> On 2014/10/01 19:31:41, tonyg wrote:
> > How much is --page-repeat really in the way? Being able to run AAABBBCCC vs
> > ABCABCABC can trigger very different performance cases when you consider all
> of
> > the things that Chrome caches. I use the flag pretty often locally, but am
not
> > sure how common that is. Can you explain more about the rationale for
> removing?
> 
> If users explicitly want AAABBBCC, they can either duplicate adding the page
> sets in PageRunner.__init__ or write for loop in Page.Run() method.
> 
> I find supporting --page-repeat complicate page_runner's logic. 
> 
> For example, assuming the runs are A1 A2 A3 B1 B2 B3
> 
> 1) If A3 fails, should the "retry at most X times" logic be applied for
> rerunning A3 only or should we repeat another 3 runs of page A? 
> 2) Same question as A3, if the test requirement is warm start.
> 3) The results system have logic for merging results of same page. If A2 fail,
> do we consider the merged value for page A a failure value or average of A1 &
> A3?
> 
> It's not trivial to provide one right answer for each of these questions. User
> can get the behavior they want explicitly either by duplicating adding the
page
> sets in PageRunner.__init__ or making a for loop in Page.Run() method.
> 
> > 
> > ps. +1 to remove everything related to endure.

PTAL again

[qyearsley, 2014-10-02 18:24:52]

LGTM with regard to removing current endure benchmark/measurement/page_sets; I'm
not sure what the tradeoffs are for removing --page-repeat though.

[nednguyen, 2014-10-02 19:23:08]

nednguyen@google.com changed reviewers:
+ nduca@chromium.org

[tonyg, 2014-10-09 21:18:03]

I think there's a legitimate use case for customizing measurements on the
command line though without having to write python to simulate. If the thing
that is in the way is the retry logic's interaction with this flag, could we
instead remove the retry logic? Retrying is likely no longer needed and has
super dubious performance characteristics any way. What do you think about
repurposing this CL to just remove endure. Then land a patch to disable retry.
If no bots become flaky, then we're good to go there. If page_repeat is still in
the way, maybe solicit more opinion on telemetry@chromium.org to see if anyone
else finds it useful or if it's just Ken and I.

[nednguyen, 2014-10-09 21:19:36]

On 2014/10/09 21:18:03, tonyg wrote:
> I think there's a legitimate use case for customizing measurements on the
> command line though without having to write python to simulate. If the thing
> that is in the way is the retry logic's interaction with this flag, could we
> instead remove the retry logic? Retrying is likely no longer needed and has
> super dubious performance characteristics any way. What do you think about
> repurposing this CL to just remove endure. Then land a patch to disable retry.
> If no bots become flaky, then we're good to go there. If page_repeat is still
in
> the way, maybe solicit more opinion on mailto:telemetry@chromium.org to see if
anyone
> else finds it useful or if it's just Ken and I.

SGTM

[nednguyen, 2014-10-09 21:29:56]

Patchset #4 (id:130001) has been deleted

[nednguyen, 2014-10-09 21:30:41]

Patchset #3 (id:70001) has been deleted

[nednguyen, 2014-10-09 22:08:36]

On 2014/10/09 21:19:36, nednguyen wrote:
> On 2014/10/09 21:18:03, tonyg wrote:
> > I think there's a legitimate use case for customizing measurements on the
> > command line though without having to write python to simulate. If the thing
> > that is in the way is the retry logic's interaction with this flag, could we
> > instead remove the retry logic? Retrying is likely no longer needed and has
> > super dubious performance characteristics any way. What do you think about
> > repurposing this CL to just remove endure. Then land a patch to disable
retry.
> > If no bots become flaky, then we're good to go there. If page_repeat is
still
> in
> > the way, maybe solicit more opinion on mailto:telemetry@chromium.org to see
if
> anyone
> > else finds it useful or if it's just Ken and I.
> 
> SGTM

PTAL again.

[tonyg, 2014-10-09 22:28:01]

lgtm

[nednguyen, 2014-10-09 22:30:18]

The CQ bit was checked by nednguyen@google.com

[commit-bot: I haz the power, 2014-10-09 22:31:19]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/616363002/200001

[commit-bot: I haz the power, 2014-10-09 23:33:46]

Committed patchset #3 (id:200001)

[commit-bot: I haz the power, 2014-10-09 23:34:42]

Patchset 3 (id:??) landed as
https://crrev.com/83dc32e0417c1c99b989f3ba0f277e28890d8799
Cr-Commit-Position: refs/heads/master@{#299008}