tools: Add tough_fast_scrolling_cases page set.

tools: Add tough_fast_scrolling_cases page set.

This adds a page set that is useful to measure how good a browser
is at producing new content when scrolling a page fast.

MAX_SCROLL_LENGTH_PIXELS is changed into MAX_SCROLL_LENGTH_TIME_MS
to allow fast scrolling of large pages.

BUG=406404
Committed: https://crrev.com/6311db1b5f04125b7f22dc6721e1baf3fc20265f
Cr-Commit-Position: refs/heads/master@{#302355}

[reveman, 2014-10-30 22:48:47]

reveman@chromium.org changed reviewers:
+ nduca@chromium.org

[reveman, 2014-10-30 22:48:48]

These tests are really good at exposing how bad we are at scrolling pages fast
when using async uploads. mean_pixels_approximated is ~80% with async uploads
and ~0% when turning on 1-copy rasterizer (--enable-one-copy).

The problem is even more obvious when comparing the runs visually. With async
uploads you see nothing but checkerboarding, while using 1-copy checkerboarding
is non-existent. Note that results may vary depending on the device.

My greatest concern with 1-copy was that smoothness would suffer but that's not
the case afaict. Smoothness is actually measured as significantly better with
1-copy compared to async uploads.

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
File tools/perf/benchmarks/smoothness.py (right):

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
tools/perf/benchmarks/smoothness.py:133: class
SmoothnessToughFastScrollingCases(benchmark.Benchmark):
This enables this new benchmark on all devices. Not sure that's a good idea. Let
me know what you think.

[ilja, 2014-10-31 04:38:50]

ihf@chromium.org changed reviewers:
+ ihf@chromium.org

[ilja, 2014-10-31 04:38:51]

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
File tools/perf/benchmarks/smoothness.py (right):

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
tools/perf/benchmarks/smoothness.py:133: class
SmoothnessToughFastScrollingCases(benchmark.Benchmark):
I think that is fine for ChromeOS. But I also think we would need to hook it up
first to get it running. So, maybe keep a few days between landing this and
landing the change that you want to benchmark?

[reveman, 2014-10-31 16:36:52]

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
File tools/perf/benchmarks/smoothness.py (right):

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
tools/perf/benchmarks/smoothness.py:133: class
SmoothnessToughFastScrollingCases(benchmark.Benchmark):
On 2014/10/31 04:38:50, ilja wrote:
> I think that is fine for ChromeOS. But I also think we would need to hook it
up
> first to get it running. So, maybe keep a few days between landing this and
> landing the change that you want to benchmark?

I'm mostly interested in this as mechanism to prevent regressions but if we can
get this landed and enabled today then we can have it run over the weekend and I
can enable the feature that will improve this early next week. I'd rather not
enable this too close to the branch point..

[ilja, 2014-10-31 20:18:29]

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
File tools/perf/benchmarks/smoothness.py (right):

https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
tools/perf/benchmarks/smoothness.py:133: class
SmoothnessToughFastScrollingCases(benchmark.Benchmark):
On 2014/10/31 16:36:52, reveman wrote:
> On 2014/10/31 04:38:50, ilja wrote:
> > I think that is fine for ChromeOS. But I also think we would need to hook it
> up
> > first to get it running. So, maybe keep a few days between landing this and
> > landing the change that you want to benchmark?
> 
> I'm mostly interested in this as mechanism to prevent regressions but if we
can
> get this landed and enabled today then we can have it run over the weekend and
I
> can enable the feature that will improve this early next week. I'd rather not
> enable this too close to the branch point..

I hooked this test up for ChromeOS in anticipation. As soon as you land this you
should start seeing results here
https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&test...

Right now it is red as the test obviously doesn't exist yet.

[reveman, 2014-10-31 20:29:25]

On 2014/10/31 20:18:29, ilja wrote:
>
https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
> File tools/perf/benchmarks/smoothness.py (right):
> 
>
https://codereview.chromium.org/688423002/diff/1/tools/perf/benchmarks/smooth...
> tools/perf/benchmarks/smoothness.py:133: class
> SmoothnessToughFastScrollingCases(benchmark.Benchmark):
> On 2014/10/31 16:36:52, reveman wrote:
> > On 2014/10/31 04:38:50, ilja wrote:
> > > I think that is fine for ChromeOS. But I also think we would need to hook
it
> > up
> > > first to get it running. So, maybe keep a few days between landing this
and
> > > landing the change that you want to benchmark?
> > 
> > I'm mostly interested in this as mechanism to prevent regressions but if we
> can
> > get this landed and enabled today then we can have it run over the weekend
and
> I
> > can enable the feature that will improve this early next week. I'd rather
not
> > enable this too close to the branch point..
> 
> I hooked this test up for ChromeOS in anticipation. As soon as you land this
you
> should start seeing results here
>
https://wmatrix.googleplex.com/unfiltered?hide_missing=True&releases=tot&test...
> 
> Right now it is red as the test obviously doesn't exist yet.

Great, thanks!

[nduca, 2014-10-31 20:36:54]

nduca@chromium.org changed reviewers:
+ ernstm@chromium.org

[nduca, 2014-10-31 20:36:54]

ernstm is a good reviewer. this should probably be a few additional pages in the
tough_scrolling_cases pageset and then you use the pageset filtering to make a
special pageset that filters just the ones here. Manfred can show you how we did
that for the gpu raster pages in the silk pageset.

[ernstm, 2014-10-31 22:34:41]

On 2014/10/31 20:36:54, nduca wrote:
> ernstm is a good reviewer. this should probably be a few additional pages in
the
> tough_scrolling_cases pageset and then you use the pageset filtering to make a
> special pageset that filters just the ones here. Manfred can show you how we
did
> that for the gpu raster pages in the silk pageset.

LGTM

[ernstm, 2014-10-31 22:40:43]

The CQ bit was checked by ernstm@chromium.org

[commit-bot: I haz the power, 2014-10-31 22:41:44]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/688423002/20001

[commit-bot: I haz the power, 2014-10-31 23:47:16]

Committed patchset #2 (id:20001)

[commit-bot: I haz the power, 2014-10-31 23:48:35]

Patchset 2 (id:??) landed as
https://crrev.com/6311db1b5f04125b7f22dc6721e1baf3fc20265f
Cr-Commit-Position: refs/heads/master@{#302355}

[epennerAtGoogle, 2014-11-20 20:38:08]

On 2014/10/31 23:48:35, I haz the power (commit-bot) wrote:
> Patchset 2 (id:??) landed as
> https://crrev.com/6311db1b5f04125b7f22dc6721e1baf3fc20265f
> Cr-Commit-Position: refs/heads/master@{#302355}

I noticed that his is having unexpected side effects on other tests. It looks
like previously when the scroll speed was reduced, the test would run longer and
reduce noise (not only from the slower speed but also from the longer duration).

However by default this is more clear/expected so I think we can just leave it
be (someone can override time if they really want that).

[reveman, 2014-11-21 17:16:48]

On 2014/11/20 20:38:08, epennerAtGoogle wrote:
> On 2014/10/31 23:48:35, I haz the power (commit-bot) wrote:
> > Patchset 2 (id:??) landed as
> > https://crrev.com/6311db1b5f04125b7f22dc6721e1baf3fc20265f
> > Cr-Commit-Position: refs/heads/master@{#302355}
> 
> I noticed that his is having unexpected side effects on other tests. It looks
> like previously when the scroll speed was reduced, the test would run longer
and
> reduce noise (not only from the slower speed but also from the longer
duration).
> 
> However by default this is more clear/expected so I think we can just leave it
> be (someone can override time if they really want that).

Yea, we could add a minimal scroll distance that makes it behave as before when
scroll speed is reduced if necessary.