Make canvas perf tests run less than 6 sec.

Make canvas perf tests run less than 6 sec.

Perf tests in PerformanceTests/Canvas usually run 6 times; 1 for warm-up, 5 for data.
Currently, upload-canvas-2d-to-texture.html takes ~72 sec on Intel Ivy Bridge. If we have
Android perf bot, this perf test takes huge time.

upload-canvas-2d-to-texture.html calls glTexImage2d() 1000 times per one run. Ivy Bridge
without Accelerated 2D Canvas can call glTexImage2d() ~85 times/sec, so one run takes
12 sec, and 6 runs take 72 sec.

This CL limits 1 sec per one run for all canvas perf tests. To apply common logic to 6 tests,
this CL introduces canvas_runner.js, which is a wrapper of PerformanceTests/resources/runner.js.
canvas_runner.js makes sure that one run takes less than 1 sec. The algorithm is 
min(1 sec, time to call api 1000 times).

This CL effects test results slightly, because now upload-canvas-2d-to-texture.html takes
6 sec, instead of 72 sec.

BUG=242244
NOTRY=true

Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=170288

[dshwang, 2014-03-14 11:58:03]

It's follow-up CL of junj@'s http://crrev.com/19607004

[dshwang, 2014-03-14 12:02:31]

Currently, two tests set 100, and four tests set 1000.
draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:var MEASURE_DRAW_TIMES =
1000;
draw-dynamic-webgl-to-hw-accelerated-canvas-2d.html:var MEASURE_DRAW_TIMES =
100;
draw-static-canvas-2d-to-hw-accelerated-canvas-2d.html:var MEASURE_DRAW_TIMES =
1000;
draw-static-webgl-to-hw-accelerated-canvas-2d.html:var MEASURE_DRAW_TIMES =
1000;
upload-canvas-2d-to-texture.html:var MEASURE_DRAW_TIMES = 1000;
upload-webgl-to-texture.html:var MEASURE_DRAW_TIMES = 100;

If you have the best number, please recommend to me.

[Ken Russell (switch to Gerrit), 2014-03-15 00:01:48]

Reducing the number of iterations will increase the noise of the benchmark. Have
you measured the mean and stddev that the benchmark measures? How do the
before-and-after numbers compare?

[dshwang, 2014-03-15 18:47:16]

On 2014/03/15 00:01:48, Ken Russell wrote:
> Reducing the number of iterations will increase the noise of the benchmark.
Have
> you measured the mean and stddev that the benchmark measures? How do the
> before-and-after numbers compare?

Hi, I improve this CL. now I don't hardcode 100. I make a test be smart.
All test calls the API 1000 times but stop at 1 sec.

I guess it's junj@'s original intention about global |count| variable.

In my machine, I can not see any difference of results between ToT and this CL.

[Ken Russell (switch to Gerrit), 2014-03-17 06:03:24]

+tonyg as I don't know how these performance tests are run or on what machines.
Tony, will these changes affect any historical runs of these benchmarks?

I think some of these changes are not moving the code in the best direction.

https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
File
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html
(right):

https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:33:
while ((PerfTestRunner.now() - start <= MAX_MEASURE_TIME_PER_FRAME) && (count *
MEASURE_DRAW_TIMES < MAX_MEASURE_DRAW_TIMES)) {
The duplicated constants and harness code throughout these six benchmarks is
unfortunate. Can you please think of a way to share code between them (perhaps
moving the harness to a separate .js file and passing in an instance of a
constructor function with setup, run, and teardown functions on its prototype)
so that in the future any changes can be made in one place?

https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:48:
requestAnimationFrame(drawCanvas2DToCanvas2D);
Since the conditional on requestAnimationFrame was removed, how does this not go
into an infinite loop? Is this test required to run inside a test harness?
PerformanceTests/resources/runner.js seems to support both running inside
TestRunner and not.

https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
File PerformanceTests/Canvas/upload-canvas-2d-to-texture.html (right):

https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
PerformanceTests/Canvas/upload-canvas-2d-to-texture.html:46: gl.finish();
The earlier implementation using readPixels was better. There's no strong
guarantee in WebGL that calling finish() will stall the calling context because
there's no user-visible side effect.

[dshwang, 2014-03-17 16:29:45]

On 2014/03/17 06:03:24, Ken Russell wrote:
> +tonyg as I don't know how these performance tests are run or on what
machines.
> Tony, will these changes affect any historical runs of these benchmarks?
> 
> I think some of these changes are not moving the code in the best direction.
> 
>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
> File
>
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html
> (right):
> 
>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
>
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:33:
> while ((PerfTestRunner.now() - start <= MAX_MEASURE_TIME_PER_FRAME) && (count
*
> MEASURE_DRAW_TIMES < MAX_MEASURE_DRAW_TIMES)) {
> The duplicated constants and harness code throughout these six benchmarks is
> unfortunate. Can you please think of a way to share code between them (perhaps
> moving the harness to a separate .js file and passing in an instance of a
> constructor function with setup, run, and teardown functions on its prototype)
> so that in the future any changes can be made in one place?

I was looking forward to this request. I'll update.

>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
>
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:48:
> requestAnimationFrame(drawCanvas2DToCanvas2D);
> Since the conditional on requestAnimationFrame was removed, how does this not
go
> into an infinite loop? Is this test required to run inside a test harness?
> PerformanceTests/resources/runner.js seems to support both running inside
> TestRunner and not.

The fact is opposite. If we don't call
requestAnimationFrame(drawCanvas2DToCanvas2D), test stop infinitely.
runner.js hooks requestAnimationFrame() and stop the test if needed.
So currently, we have a potential hang-up bug, although 60000 is not reachable
number.

>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
> File PerformanceTests/Canvas/upload-canvas-2d-to-texture.html (right):
> 
>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
> PerformanceTests/Canvas/upload-canvas-2d-to-texture.html:46: gl.finish();
> The earlier implementation using readPixels was better. There's no strong
> guarantee in WebGL that calling finish() will stall the calling context
because
> there's no user-visible side effect.

ah, WebGL spec seems to be slightly different to OpenGL ES2 spec. Thank you for
explanation. I'll rollback to readPixels.

[Ken Russell (switch to Gerrit), 2014-03-17 16:42:17]

On 2014/03/17 16:29:45, dshwang wrote:
> On 2014/03/17 06:03:24, Ken Russell wrote:
> >
>
https://codereview.chromium.org/200253002/diff/20001/PerformanceTests/Canvas/...
> >
>
PerformanceTests/Canvas/draw-dynamic-canvas-2d-to-hw-accelerated-canvas-2d.html:48:
> > requestAnimationFrame(drawCanvas2DToCanvas2D);
> > Since the conditional on requestAnimationFrame was removed, how does this
not
> go
> > into an infinite loop? Is this test required to run inside a test harness?
> > PerformanceTests/resources/runner.js seems to support both running inside
> > TestRunner and not.
> 
> The fact is opposite. If we don't call
> requestAnimationFrame(drawCanvas2DToCanvas2D), test stop infinitely.
> runner.js hooks requestAnimationFrame() and stop the test if needed.
> So currently, we have a potential hang-up bug, although 60000 is not reachable
> number.

Where does the hook happen? I looked through
PerformanceTests/resources/runner.js and can't see where
window.requestAnimationFrame is hooked.

[dshwang, 2014-03-17 19:10:58]

On 2014/03/17 16:42:17, Ken Russell wrote:
> Where does the hook happen? I looked through
> PerformanceTests/resources/runner.js and can't see where
> window.requestAnimationFrame is hooked.

I explained a bit wrong. sorry.

PerfTestRunner.measureValueAsync() must be called repeatedly. and then runner.js
stops if needed.
requestAnimationFrame() guarantees measureValueAsync() is called.

    PerfTestRunner.measureValueAsync = function (measuredValue) {
        completedIterations++;

        try {
            ignoreWarmUpAndLog(measuredValue);
        } catch (exception) {
            logFatalError("Got an exception while logging the result with name="
+ exception.name + ", message=" + exception.message);
            return;
        }

        if (completedIterations >= iterationCount)
            finish();
    }

[dshwang, 2014-03-25 20:31:05]

Hi, I introduce canvas_runner.js to remove duplicated code.

Could you review again?

[Ken Russell (switch to Gerrit), 2014-03-27 18:02:38]

That's very nice. Thank you for doing this cleanup and such a thorough job. LGTM

[dshwang, 2014-03-27 18:25:04]

On 2014/03/27 18:02:38, Ken Russell wrote:
> That's very nice. Thank you for doing this cleanup and such a thorough job.
LGTM

My pleasure :) Thank you for lgtm!

[dshwang, 2014-03-27 18:25:09]

The CQ bit was checked by dongseong.hwang@intel.com

[commit-bot: I haz the power, 2014-03-27 18:25:30]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/dongseong.hwang@intel.com/200253002/40001

[commit-bot: I haz the power, 2014-03-27 19:37:56]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-27 19:37:57]

Try jobs failed on following builders:
  tryserver.blink on linux_blink_dbg

[dshwang, 2014-03-27 19:38:53]

The CQ bit was checked by dongseong.hwang@intel.com

[commit-bot: I haz the power, 2014-03-27 19:38:58]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/dongseong.hwang@intel.com/200253002/40001

[commit-bot: I haz the power, 2014-03-27 21:36:21]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-27 21:36:21]

Try jobs failed on following builders:
  tryserver.blink on linux_blink_dbg

[dshwang, 2014-03-27 21:52:13]

The CQ bit was checked by dongseong.hwang@intel.com

[commit-bot: I haz the power, 2014-03-27 21:52:16]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/dongseong.hwang@intel.com/200253002/40001

[commit-bot: I haz the power, 2014-03-27 23:02:38]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-03-27 23:02:38]

Try jobs failed on following builders:
  tryserver.blink on linux_blink_dbg

[dshwang, 2014-03-28 06:53:53]

The CQ bit was checked by dongseong.hwang@intel.com

[commit-bot: I haz the power, 2014-03-28 06:54:04]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/dongseong.hwang@intel.com/200253002/40001

[commit-bot: I haz the power, 2014-03-28 06:54:24]

Change committed as 170288