nanobench: lazily decode bitmaps in .skps.

nanobench: lazily decode bitmaps in .skps.

This cuts down on tool overhead when running something like recording only,
    $ out/Release/nanobench --match skp --config nonrendering
which doesn't usually ever need to decode the images.

The actual measurements for recording don't change, as the decode is not in the timed section.  It just skips irrelevant code, removing it from the profile and making the tool run faster.

This does, however, make a significant difference for playback speed.  Most skps draw faster with this patch, some slower.  I don't really have a good intuition for what's going on here.  There is a fixed clip acting as a viewport, so there are probably lots of images that don't ever need to be decoded.  Ideas?  Is this perhaps because we're now blitting from smaller, partially decoded source images?

~/skia (clean) $ compare clean.log lazy-decode-bitmaps.log
                   tabl_slashdot.skp_1	2.76ms -> 4.33ms	1.57x
               tabl_slashdot.skp_1_mpd	2.79ms -> 4.07ms	1.46x
                    tabl_sahadan.skp_1	3.41ms -> 4.87ms	1.43x
                 tabl_googleblog.skp_1	1.52ms -> 2.05ms	1.35x
               tabl_techmeme.skp_1_mpd	1.14ms -> 1.51ms	1.32x
               tabl_transformice.skp_1	2.61ms -> 3.43ms	1.31x
                tabl_sahadan.skp_1_mpd	3.54ms -> 4.48ms	1.26x
                   tabl_techmeme.skp_1	1.01ms -> 1.27ms	1.26x
                tabl_nytimes.skp_1_mpd	   1ms -> 1.23ms	1.23x
           tabl_worldjournal.skp_1_mpd	1.98ms -> 2.43ms	1.23x
                 tabl_pravda.skp_1_mpd	2.05ms -> 2.51ms	1.22x
           tabl_transformice.skp_1_mpd	2.75ms -> 3.19ms	1.16x
                    tabl_nytimes.skp_1	 874us -> 1.01ms	1.15x
                     tabl_pravda.skp_1	1.83ms -> 1.99ms	1.09x
               tabl_worldjournal.skp_1	1.76ms -> 1.91ms	1.09x
                desk_wowwiki.skp_1_mpd	 3.7ms ->  3.9ms	1.05x
                       tabl_digg.skp_1	3.99ms -> 4.16ms	1.04x
                  tabl_ukwsj.skp_1_mpd	   3ms -> 3.12ms	1.04x
                    desk_booking.skp_1	3.74ms -> 3.81ms	1.02x
    desk_googlespreadsheetdashed.skp_1	10.6ms -> 10.6ms	1x
                      tabl_ukwsj.skp_1	2.88ms -> 2.89ms	1x
desk_googlespreadsheetdashed.skp_1_mpd	11.8ms -> 11.8ms	1x
     desk_jsfiddlehumperclip.skp_1_mpd	 891us ->  888us	1x
          desk_googlespreadsheet.skp_1	4.65ms -> 4.62ms	0.99x
                  tabl_gspro.skp_1_mpd	1.97ms -> 1.94ms	0.99x
                desk_booking.skp_1_mpd	 4.1ms ->    4ms	0.98x
                     desk_carsvg.skp_1	18.2ms -> 17.7ms	0.97x
            desk_gmailthread.skp_1_mpd	2.81ms -> 2.73ms	0.97x
               desk_tigersvg.skp_1_mpd	19.5ms -> 18.9ms	0.97x
                     desk_mapsvg.skp_1	88.4ms -> 85.6ms	0.97x
                   tabl_cnet.skp_1_mpd	1.43ms -> 1.38ms	0.97x
             desk_jsfiddlebigcar.skp_1	1.26ms -> 1.22ms	0.96x
                        desk_gws.skp_1	1.87ms ->  1.8ms	0.96x
                   desk_linkedin.skp_1	2.07ms -> 1.98ms	0.96x
             tabl_deviantart.skp_1_mpd	 118ms ->  113ms	0.96x
                       tabl_cnet.skp_1	 1.2ms -> 1.14ms	0.95x
          tabl_androidpolice.skp_1_mpd	5.95ms -> 5.63ms	0.95x
                     desk_sfgate.skp_1	1.75ms -> 1.64ms	0.94x
                    desk_twitter.skp_1	  74ms -> 69.6ms	0.94x
                desk_youtube.skp_1_mpd	3.17ms -> 2.96ms	0.93x
                desk_gmailthread.skp_1	2.73ms -> 2.54ms	0.93x
            desk_silkfinance.skp_1_mpd	1.71ms -> 1.59ms	0.93x
         desk_jsfiddlebigcar.skp_1_mpd	1.45ms -> 1.35ms	0.93x
            desk_pokemonwiki.skp_1_mpd	2.72ms -> 2.51ms	0.92x
                    desk_gws.skp_1_mpd	2.14ms -> 1.98ms	0.92x
                 desk_googlehome.skp_1	 563us ->  517us	0.92x
                       desk_espn.skp_1	4.24ms -> 3.89ms	0.92x
          tabl_culturalsolutions.skp_1	12.7ms -> 11.6ms	0.91x
                 desk_sfgate.skp_1_mpd	1.91ms -> 1.74ms	0.91x
                       tabl_hsfi.skp_1	1.06ms ->  966us	0.91x
               desk_samoasvg.skp_1_mpd	10.5ms -> 9.47ms	0.91x
               desk_facebook.skp_1_mpd	 3.8ms -> 3.43ms	0.9x
                    desk_youtube.skp_1	3.52ms -> 3.14ms	0.89x
                   desk_ebay.skp_1_mpd	2.95ms -> 2.62ms	0.89x
                   desk_samoasvg.skp_1	10.9ms -> 9.66ms	0.89x
      desk_googlespreadsheet.skp_1_mpd	5.59ms -> 4.94ms	0.88x
                 desk_mapsvg.skp_1_mpd	 100ms -> 87.9ms	0.88x
                   desk_espn.skp_1_mpd	 4.7ms -> 4.12ms	0.88x
              desk_wordpress.skp_1_mpd	1.92ms -> 1.68ms	0.87x
                 tabl_deviantart.skp_1	 140ms ->  122ms	0.87x
           tabl_cuteoverload.skp_1_mpd	4.41ms -> 3.83ms	0.87x
                   desk_tigersvg.skp_1	19.6ms ->   17ms	0.87x
             tabl_googlecalendar.skp_1	4.01ms -> 3.44ms	0.86x
                    desk_blogger.skp_1	2.49ms -> 2.14ms	0.86x
             desk_chalkboard.skp_1_mpd	52.7ms ->   45ms	0.85x
                    desk_weather.skp_1	2.88ms -> 2.46ms	0.85x
                 desk_chalkboard.skp_1	  51ms -> 43.4ms	0.85x
               desk_yahooanswers.skp_1	2.74ms -> 2.32ms	0.85x
             desk_forecastio.skp_1_mpd	1.26ms -> 1.07ms	0.85x
              tabl_androidpolice.skp_1	5.18ms -> 4.34ms	0.84x
           desk_yahooanswers.skp_1_mpd	3.44ms -> 2.85ms	0.83x
                    tabl_cnn.skp_1_mpd	2.59ms -> 2.15ms	0.83x
                  desk_pinterest.skp_1	2.69ms -> 2.22ms	0.83x
                   tabl_hsfi.skp_1_mpd	 1.6ms -> 1.32ms	0.82x
      tabl_culturalsolutions.skp_1_mpd	13.8ms -> 11.3ms	0.82x
                desk_twitter.skp_1_mpd	76.6ms ->   63ms	0.82x
                       desk_ebay.skp_1	3.11ms -> 2.51ms	0.81x
                    tabl_mlb.skp_1_mpd	3.17ms -> 2.53ms	0.8x
                    tabl_mozilla.skp_1	2.42ms -> 1.91ms	0.79x
                desk_pokemonwiki.skp_1	2.84ms -> 2.22ms	0.78x
                 desk_carsvg.skp_1_mpd	23.3ms -> 17.8ms	0.77x
                    desk_wowwiki.skp_1	4.21ms -> 3.21ms	0.76x
                     desk_amazon.skp_1	 963us ->  728us	0.76x
              desk_css3gradients.skp_1	2.58ms -> 1.92ms	0.74x
               tabl_cuteoverload.skp_1	4.55ms -> 3.38ms	0.74x
                        tabl_cnn.skp_1	3.13ms -> 2.29ms	0.73x
             tabl_googleblog.skp_1_mpd	2.32ms ->  1.7ms	0.73x
                 desk_mobilenews.skp_1	3.65ms -> 2.61ms	0.71x
                 desk_googleplus.skp_1	3.76ms -> 2.66ms	0.71x
                tabl_mozilla.skp_1_mpd	2.88ms -> 2.03ms	0.71x
              desk_pinterest.skp_1_mpd	3.17ms -> 2.21ms	0.7x
          desk_css3gradients.skp_1_mpd	2.98ms -> 2.07ms	0.69x
                desk_silkfinance.skp_1	2.06ms -> 1.42ms	0.69x
                   desk_facebook.skp_1	 4.5ms -> 3.07ms	0.68x
             desk_mobilenews.skp_1_mpd	4.05ms -> 2.73ms	0.68x
                  desk_baidu.skp_1_mpd	2.73ms -> 1.81ms	0.66x
                desk_weather.skp_1_mpd	3.93ms ->  2.5ms	0.64x
                  desk_wordpress.skp_1	2.15ms -> 1.36ms	0.63x
             desk_googlehome.skp_1_mpd	1.02ms ->  605us	0.59x
                   desk_fontwipe.skp_1	 722us ->  402us	0.56x
               desk_fontwipe.skp_1_mpd	 897us ->  486us	0.54x
                      desk_baidu.skp_1	3.02ms ->  1.6ms	0.53x
                 desk_forecastio.skp_1	2.01ms ->  999us	0.5x
                 desk_amazon.skp_1_mpd	1.77ms ->  860us	0.49x

BUG=skia:

Committed: https://skia.googlesource.com/skia/+/7e225bdb1f00ae4aed524ff8d0a61df3d3abb109

Committed: https://skia.googlesource.com/skia/+/1b6b626f9bc0deebe4fe2e63f422d6b122419205

[mtklein_C, 2014-11-25 19:35:47]

mtklein@chromium.org changed reviewers:
+ reed@google.com, robertphillips@google.com, scroggo@google.com

[scroggo, 2014-11-25 20:31:13]

> This does, however, make a significant difference for
> playback speed.  Most skps draw faster with this patch,
> some slower.  I don't really have a good intuition for
> what's going on here.  There is a fixed clip acting as a
> viewport, so there are probably lots of images that don't
> ever need to be decoded.  Ideas?  Is this perhaps because
> we're now blitting from smaller, partially decoded source
> images?

I can definitely understand why playback for some SKPs would get slower: when we
just use the default, DecodeMemory, the decode is happening outside of the timed
section. If we decode lazily, we're timing the decode in addition to playback
itself.

As for the SKPs getting faster: Clipping out images does mean we skip decoding
them, but that just skips work done outside of the timed section. The decoder
knows nothing of clips, so it just decodes the whole thing. I don't know how
they got faster.

Either way, it might be (somewhat) more realistic to do decoding lazily (Chrome
uses lazy decoding, but does it differently from us), but it also means we're
timing libjpeg etc. Do we want do that? Or should we focus our testing more on
our actual drawing? (Or does it make a big enough difference for us to care?)

https://codereview.chromium.org/743613005/diff/1/bench/nanobench.cpp
File bench/nanobench.cpp (right):

https://codereview.chromium.org/743613005/diff/1/bench/nanobench.cpp#newcode491
bench/nanobench.cpp:491: pic->reset(SkPicture::CreateFromStream(stream.get(),
&sk_tools::LazyDecodeBitmap));
Alternatively, we could optionally do lazy decodes, the way bench_pictures does.

We certainly want to do lazy decoding when timing recording (as you realized
when you wrote https://codereview.chromium.org/212563009), but maybe we don't
want lazy in other cases. Or maybe we want to do both and compare.

https://codereview.chromium.org/743613005/diff/1/gyp/bench.gyp
File gyp/bench.gyp (right):

https://codereview.chromium.org/743613005/diff/1/gyp/bench.gyp#newcode17
gyp/bench.gyp:17: '../tools/LazyDecodeBitmap.cpp',
The help text in this file that says useVolatileCache is "Only meaningful if
--deferImageDecoding is set to true" is no longer true, if we defer by default.

[mtklein, 2014-11-25 21:47:31]

If Chrome does lazy decoding, let's land this.  We already try to make nanobench
match Chrome as much as possible.  I guess I'd like better intuition about why
those .skps draw faster this way---drawing slower because we move the decode
into the timing loop makes sense, I agree---but as long as we're just
establishing a new stable baseline, I think I can live with the mystery.

I don't want to make this a command line option until we find repeated need to
vary this.  We've grown a lot of complexity in these tools through options that
are never exercised.

Do I need to care about --useVolatileCache?  What's different if we use that?  I
think the note meat that unless --deferImageDecoding was set (in
bench_pictures), the code in LazyDecodeBitmap was just never called.

[scroggo, 2014-11-25 22:03:04]

On 2014/11/25 21:47:31, mtklein wrote:
> If Chrome does lazy decoding, let's land this.  We already try to make
nanobench
> match Chrome as much as possible.  I guess I'd like better intuition about why
> those .skps draw faster this way---drawing slower because we move the decode
> into the timing loop makes sense, I agree---but as long as we're just
> establishing a new stable baseline, I think I can live with the mystery.

Sgtm. Do we need to tell skia perf that there's a new baseline?

> 
> I don't want to make this a command line option until we find repeated need to
> vary this.  We've grown a lot of complexity in these tools through options
that
> are never exercised.

Sgtm

> 
> Do I need to care about --useVolatileCache?

Not sure I understand the question. In theory, it tests a feature similar to one
Chrome uses. I don't know whether we actually test using the flag vs not using
it (or whether we use it at all).

> What's different if we use that?

We use ashmem, but only on Android. (At one point I think I had a similar code
path
using Mac's version of volatile memory, but I don't think that's still there.)

> I think the note meat that unless --deferImageDecoding was set (in
> bench_pictures), the code in LazyDecodeBitmap was just never called.

That's correct. But with your patch, that note is misleading when running
nanobench. (To be fair, other apps may be including it, in which case it's
misleading there.)

lgtm

[mtklein, 2014-11-25 22:24:11]

> Sgtm. Do we need to tell skia perf that there's a new baseline?

Nope.  We just need to ack the regressions/improvements it finds.  It should peg
them to the Git hash this CL lands with.

> Not sure I understand the question. In theory, it tests a feature similar to
one
> Chrome uses. I don't know whether we actually test using the flag vs not using
> it (or whether we use it at all).
> 
> That's correct. But with your patch, that note is misleading when running
> nanobench. (To be fair, other apps may be including it, in which case it's
> misleading there.)

Gotcha.  This sounds like a follow up, to at least delete the comment, at most
delete the whole feature.

[mtklein, 2014-11-25 22:24:15]

The CQ bit was checked by mtklein@google.com

[commit-bot: I haz the power, 2014-11-25 22:25:17]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/743613005/1

[commit-bot: I haz the power, 2014-11-25 22:34:07]

Committed patchset #1 (id:1) as
https://skia.googlesource.com/skia/+/7e225bdb1f00ae4aed524ff8d0a61df3d3abb109

[mtklein, 2014-11-25 22:56:27]

A revert of this CL (patchset #1 id:1) has been created in
https://codereview.chromium.org/759753004/ by mtklein@google.com.

The reason for reverting is: Some bots crashing..

[scroggo, 2015-01-30 15:59:53]

On 2014/11/25 22:56:27, mtklein wrote:
> A revert of this CL (patchset #1 id:1) has been created in
> https://codereview.chromium.org/759753004/ by mailto:mtklein@google.com.
> 
> The reason for reverting is: Some bots crashing..

Any idea why this crashed?

[mtklein, 2015-01-30 16:05:24]

> Any idea why this crashed?

No idea.  Want to try again?

[scroggo, 2015-02-09 19:43:10]

On 2015/01/30 16:05:24, mtklein wrote:
> > Any idea why this crashed?
> 
> No idea.  Want to try again?

Sure!

[scroggo, 2015-02-09 19:43:25]

The CQ bit was checked by scroggo@google.com

[commit-bot: I haz the power, 2015-02-09 19:44:17]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/743613005/1

[commit-bot: I haz the power, 2015-02-09 19:44:29]

Committed patchset #1 (id:1) as
https://skia.googlesource.com/skia/+/1b6b626f9bc0deebe4fe2e63f422d6b122419205

[mtklein, 2015-02-09 19:58:12]

A revert of this CL (patchset #1 id:1) has been created in
https://codereview.chromium.org/902783005/ by mtklein@google.com.

The reason for reverting is: Well, it still crashes..

[scroggo, 2015-02-09 20:04:50]

On 2015/02/09 19:58:12, mtklein wrote:
> A revert of this CL (patchset #1 id:1) has been created in
> https://codereview.chromium.org/902783005/ by mailto:mtklein@google.com.
> 
> The reason for reverting is: Well, it still crashes..

Sample crash here:
http://chromegw.corp.google.com/i/client.skia/builders/Perf-Mac10.8-MacMini4....

[scroggo, 2015-02-09 20:05:14]

On 2015/02/09 20:04:50, scroggo wrote:
> On 2015/02/09 19:58:12, mtklein wrote:
> > A revert of this CL (patchset #1 id:1) has been created in
> > https://codereview.chromium.org/902783005/ by mailto:mtklein@google.com.
> > 
> > The reason for reverting is: Well, it still crashes..
> 
> Sample crash here:
>
http://chromegw.corp.google.com/i/client.skia/builders/Perf-Mac10.8-MacMini4....

(Not sure whether that's representative of all crashes.)