Issue 2618633004: Add support for Animated PNG

scroggo_chromium

Description was changed from ========== Implement Animated PNG. Implement an internal query system in PNGImageDecoder ...

3 years, 11 months ago (2017-01-06 15:23:33 UTC) #1

Description was changed from

==========
Implement Animated PNG.

Implement an internal query system in PNGImageDecoder to parse the size and
frame data, by passing a PNGImageReader::PNGParseQuery to
PNGImageDecoder::parse(). This utilizes the PNGImageReader to answer the query.
For a PNGSizeQuery, PNGImageReader processes the stream until the first IDAT
frame chunk. At this point, the size should have been decoded from the IHDR
chunk.

With a PNGFrameCountQuery, the frame data is parsed by the PNGImageReader. This
will parse the complete data stream it has so far and reports frames when
they're fully seen, so PNGImageDecoder won't report information on partial
frames. This prevents decoding half of the frame on top of the other half of
the previous frame, in case of blending and/or disposal. The expection for
this is the first frame, which is reported as soon as the starting point of
the frame is received, so it can be decoded progressively.

When the reader does not encounter an acTL chunk before the first IDAT chunk,
the
image is considered to be non-animated, and frame count parsing is stopped, even
if
frames may appear after the IDAT chunk(s). The reason for this is a) that it is
required by the specification, so most APNG files should have it, and b) that
it does not slow down decoding for non-animated PNG's, which is the something
like 99.9% of the use cases for now.

The following behavioral decisions are implemented:

- An fcTL with the wrong chunk length results in a failed decoder, since the
decoder can't make any sensible predicitions about missing data.
- An acTL with the wrong chunk length is ignored, so the resulting image will
be non animated.
- The sequence numbers in fcTL and fdAT chunks are ignored.
- The frame count in the acTL chunk is ignored. Instead, the actual number of
complete frames that are in the stream is reported.
- A frame with a frame rect that falls outside of the original image's rect is
clipped, so it fits within the image. The frame data outside of the clipped
rectangle is ignored.

Reimplement frame decoding for non-animated PNGs. When the decoder reads a
non-animated PNG, it will reuse the existing libpng pointers and continue
processing the data stream from the first IDAT chunk, where it ended after a
PNGFrameCountQuery. This is similar to the old approach and is still used to
retain performance. Progressive decoding is supported.

Frame decoding for animated images is implemented. Single frames are decoded
by mocking them as complete PNG images. This is necessary since libpng does
not support animated PNG. This works by recreating libpng pointers, and
reprocess
the PNG header data, but with the frame's width and height. Then, the frame data
is processed, by converting fdAT chunks to IDAT chunks. With this approach, the
existing infrastructure in PNGImageDecoder to write rows into an ImageFrame does
not need many changes.

Progressive decoding is supported for non-animated PNGs and for the first frame
of animated PNGs.

Failures during decoding or parsing are handled differently, based on the
image and state of the decoder:
1) When a non-animated PNG, or the first frame of an animated PNG, can't be
decoded or parsed, set the decoder to the failed state, because there are no
frames we can show to the client. Also set the state to failed if a parse error
occurs before any frames were received.
2) When a decoding failure occurs for non-first frames, we still want to show
earlier frames. This means, if frame n has a failure during decoding,
the frame count is adjusted to n - 1.
3) When a parsing failure occurs, the frame count is adjusted to the number of
successfully parsed frames, since we can still show those.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Implement Animated PNG.

Implement an internal query system in PNGImageDecoder to parse the size and
frame data, by passing a PNGImageReader::PNGParseQuery to
PNGImageDecoder::parse(). This utilizes the PNGImageReader to answer the query.
For a PNGSizeQuery, PNGImageReader processes the stream until the first IDAT
frame chunk. At this point, the size should have been decoded from the IHDR
chunk.

With a PNGFrameCountQuery, the frame data is parsed by the PNGImageReader. This
will parse the complete data stream it has so far and reports frames when
they're fully seen, so PNGImageDecoder won't report information on partial
frames. This prevents decoding half of the frame on top of the other half of
the previous frame, in case of blending and/or disposal. The expection for
this is the first frame, which is reported as soon as the starting point of
the frame is received, so it can be decoded progressively.

When the reader does not encounter an acTL chunk before the first IDAT chunk,
the
image is considered to be non-animated, and frame count parsing is stopped, even
if
frames may appear after the IDAT chunk(s). The reason for this is a) that it is
required by the specification, so most APNG files should have it, and b) that
it does not slow down decoding for non-animated PNG's, which is the something
like 99.9% of the use cases for now.

The following behavioral decisions are implemented:

- An fcTL with the wrong chunk length results in a failed decoder, since the
decoder can't make any sensible predicitions about missing data.
- An acTL with the wrong chunk length is ignored, so the resulting image will
be non animated.
- The sequence numbers in fcTL and fdAT chunks are ignored.
- The frame count in the acTL chunk is ignored. Instead, the actual number of
complete frames that are in the stream is reported.
- A frame with a frame rect that falls outside of the original image's rect is
clipped, so it fits within the image. The frame data outside of the clipped
rectangle is ignored.

Reimplement frame decoding for non-animated PNGs. When the decoder reads a
non-animated PNG, it will reuse the existing libpng pointers and continue
processing the data stream from the first IDAT chunk, where it ended after a
PNGFrameCountQuery. This is similar to the old approach and is still used to
retain performance. Progressive decoding is supported.

Frame decoding for animated images is implemented. Single frames are decoded
by mocking them as complete PNG images. This is necessary since libpng does
not support animated PNG. This works by recreating libpng pointers, and
reprocess
the PNG header data, but with the frame's width and height. Then, the frame data
is processed, by converting fdAT chunks to IDAT chunks. With this approach, the
existing infrastructure in PNGImageDecoder to write rows into an ImageFrame does
not need many changes.

Progressive decoding is supported for non-animated PNGs and for the first frame
of animated PNGs.

Failures during decoding or parsing are handled differently based on the
image and state of the decoder:
1) When a non-animated PNG or the first frame of an animated PNG can't be
decoded or parsed, set the decoder to the failed state, because there are no
frames we can show to the client. Also set the state to failed if a parse error
occurs before any frames were received.
2) When a decoding failure occurs for non-first frames, we still want to show
earlier frames. This means, if frame n has a failure during decoding,
the frame count is adjusted to n - 1.
3) When a parsing failure occurs, the frame count is adjusted to the number of
successfully parsed frames, since we can still show those.

Update the accept header to state that APNG is supported.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

scroggo_chromium

scroggo@chromium.org changed reviewers: + cblume@chromium.org, japhet@chromium.org, maxstepin@gmail.com, noel@chromium.org

3 years, 11 months ago (2017-01-06 17:13:09 UTC) #2

scroggo_chromium

This CL revives crrev.com/2386453003. It fixes a bug in that CL, updates the accept header, ...

3 years, 11 months ago (2017-01-06 17:13:10 UTC) #3

MaxStepin

On 2017/01/06 17:13:10, scroggo_chromium wrote: > This was an interesting bug caught by the fuzzer. ...

3 years, 11 months ago (2017-01-06 19:42:34 UTC) #4

Nate Chapin

https://codereview.chromium.org/2618633004/diff/60001/content/browser/loader/mime_sniffing_resource_handler.cc File content/browser/loader/mime_sniffing_resource_handler.cc (right): https://codereview.chromium.org/2618633004/diff/60001/content/browser/loader/mime_sniffing_resource_handler.cc#newcode51 content/browser/loader/mime_sniffing_resource_handler.cc:51: const char kImageAcceptHeader[] = "image/webp,image/apng,image/*,*/*;q=0.8"; On 2017/01/06 17:13:10, scroggo_chromium ...

3 years, 11 months ago (2017-01-06 19:51:30 UTC) #5

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/20001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/20001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode543 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:543: initFrameBuffer(m_currentFrame); >> This was an interesting bug caught by ...

3 years, 11 months ago (2017-01-12 22:34:52 UTC) #6

https://codereview.chromium.org/2618633004/diff/20001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/20001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:543:
initFrameBuffer(m_currentFrame);
>> This was an interesting bug caught by the fuzzer. Frame 0 has no rows. Frame
1
>> depends on frame 0, so we call takeBitmapDataIfWritable for 1 instead of
>> setSize. initFrameBuffer was never called for 0, so 1 gets an empty bitmap. 
>> When we try to write to it (or read it, depending on blending mode) later we
are
>> writing (/reading) garbage memory.
>> 
>> A couple of alternative fixes here:
>> - Make frame 1 *not* depend on frame 0, since frame 0 is just transparent.
>> - Call initFrameBuffer before rowCallback ever gets called.
>> 
>> GIF makes this same call in its frameComplete().
>
> How about bailing out immediately if height=0?
>
> Maybe PNGImageReader::parseFrameInfo() should include some sanity checks for
the
> data it reads?
>
> For reference, that's how Firefox validates fcTL data:
> png_ensure_fcTL_is_valid()
> https://dxr.mozilla.org/mozilla-central/source/media/libpng/pngset.c#1206

Good point - we could also consider this an invalid image. A downside (arguably)
is that we would fail to decode an otherwise valid image. In our GIF decoder, we
often try to correct broken images, rather than failing. I do not know whether
this is due to a significant number of broken images on the web, or observed
ability to decode particular ones in other browsers, or speculation that we
should try to support if we can. (Maybe there's a GIF encoder out there with
bugs?)

For example, we fixed a similar bug by calling initFrameBuffer in GIF after
"decoding" an empty frame[1][2]. I do not see the original image in the bug (it
was listed with a URL of [3], but that website is not loading for me).

[1] https://bugs.webkit.org/show_bug.cgi?id=22885
[2]
https://chromium.googlesource.com/chromium/src/+/2cea16890342e4984356f2e3b7c8...
[3]
http://img.waffleimages.com/9d5247a3e6a95c2966c0c5f34b47a7837309f2af/lolchrom...

https://codereview.chromium.org/2618633004/diff/60001/content/browser/loader/...
File content/browser/loader/mime_sniffing_resource_handler.cc (right):

https://codereview.chromium.org/2618633004/diff/60001/content/browser/loader/...
content/browser/loader/mime_sniffing_resource_handler.cc:51: const char
kImageAcceptHeader[] = "image/webp,image/apng,image/*,*/*;q=0.8";
On 2017/01/06 19:51:30, Nate Chapin wrote:
> On 2017/01/06 17:13:10, scroggo_chromium wrote:
> > japhet@, I do not fully understand this - it seems like image/* should cover
> > image/apng, but we also have image/webp, so I guess this is correct? (Should
I
> > switch the order so apng comes before webp?)
> 
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html is handy for this :)
> 
> Manually specifying image/apng will cause it to be preferred over other image
> types: "Media ranges can be overridden by more specific media ranges or
specific
> media types. If more than one media range applies to a given type, the most
> specific reference has precedence."

There does not appear to be a way to distinguish between animated and
non-animated images, so specifying apng means we prefer it over e.g. jpeg? (I
guess that will not be an issue, since a site will not be trying to pick between
a jpeg and an apng?)

> 
> It's unclear to me whether you want to explicitly prefer apng over other
> animated image types or leave it to servers to chooser their first-choice
image
> format.

So if we specify both image/webp and image/apng, does the ordering determine
which is our preference? Or the q value? (Thanks for the link, but I am still
confused...) How would we specify e.g. "no preference between webp and apng, but
prefer both to gif"? Something like

"image/webp;q=.9, image/apng;q=.9, image/*, */*;q=.8"

?

> I didn't see any explicit discussion of this in the Intent thread, but I
> might'be missed it.

No discussion there. Is that the correct place to discuss this?

MaxStepin

On 2017/01/12 22:34:52, scroggo_chromium wrote: > Good point - we could also consider this an ...

3 years, 11 months ago (2017-01-13 08:43:44 UTC) #7

Noel Gordon

Good points, if Firefox is stricter then we should be too, to weed out broken ...

3 years, 11 months ago (2017-01-13 10:21:46 UTC) #8

MaxStepin

Need to change append() to push_back() in PNGImageReader.cpp

3 years, 10 months ago (2017-02-09 21:00:25 UTC) #9

scroggo_chromium

Description was changed from ========== Implement Animated PNG. Implement an internal query system in PNGImageDecoder ...

3 years, 10 months ago (2017-02-17 22:56:27 UTC) #10

Description was changed from

==========
Implement Animated PNG.

Implement an internal query system in PNGImageDecoder to parse the size and
frame data, by passing a PNGImageReader::PNGParseQuery to
PNGImageDecoder::parse(). This utilizes the PNGImageReader to answer the query.
For a PNGSizeQuery, PNGImageReader processes the stream until the first IDAT
frame chunk. At this point, the size should have been decoded from the IHDR
chunk.

With a PNGFrameCountQuery, the frame data is parsed by the PNGImageReader. This
will parse the complete data stream it has so far and reports frames when
they're fully seen, so PNGImageDecoder won't report information on partial
frames. This prevents decoding half of the frame on top of the other half of
the previous frame, in case of blending and/or disposal. The expection for
this is the first frame, which is reported as soon as the starting point of
the frame is received, so it can be decoded progressively.

When the reader does not encounter an acTL chunk before the first IDAT chunk,
the
image is considered to be non-animated, and frame count parsing is stopped, even
if
frames may appear after the IDAT chunk(s). The reason for this is a) that it is
required by the specification, so most APNG files should have it, and b) that
it does not slow down decoding for non-animated PNG's, which is the something
like 99.9% of the use cases for now.

The following behavioral decisions are implemented:

- An fcTL with the wrong chunk length results in a failed decoder, since the
decoder can't make any sensible predicitions about missing data.
- An acTL with the wrong chunk length is ignored, so the resulting image will
be non animated.
- The sequence numbers in fcTL and fdAT chunks are ignored.
- The frame count in the acTL chunk is ignored. Instead, the actual number of
complete frames that are in the stream is reported.
- A frame with a frame rect that falls outside of the original image's rect is
clipped, so it fits within the image. The frame data outside of the clipped
rectangle is ignored.

Reimplement frame decoding for non-animated PNGs. When the decoder reads a
non-animated PNG, it will reuse the existing libpng pointers and continue
processing the data stream from the first IDAT chunk, where it ended after a
PNGFrameCountQuery. This is similar to the old approach and is still used to
retain performance. Progressive decoding is supported.

Frame decoding for animated images is implemented. Single frames are decoded
by mocking them as complete PNG images. This is necessary since libpng does
not support animated PNG. This works by recreating libpng pointers, and
reprocess
the PNG header data, but with the frame's width and height. Then, the frame data
is processed, by converting fdAT chunks to IDAT chunks. With this approach, the
existing infrastructure in PNGImageDecoder to write rows into an ImageFrame does
not need many changes.

Progressive decoding is supported for non-animated PNGs and for the first frame
of animated PNGs.

Failures during decoding or parsing are handled differently based on the
image and state of the decoder:
1) When a non-animated PNG or the first frame of an animated PNG can't be
decoded or parsed, set the decoder to the failed state, because there are no
frames we can show to the client. Also set the state to failed if a parse error
occurs before any frames were received.
2) When a decoding failure occurs for non-first frames, we still want to show
earlier frames. This means, if frame n has a failure during decoding,
the frame count is adjusted to n - 1.
3) When a parsing failure occurs, the frame count is adjusted to the number of
successfully parsed frames, since we can still show those.

Update the accept header to state that APNG is supported.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Add support for Animated PNG.

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR***) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.
*** FIXME: If we start decoding on an independent frame that does *not* fill
the image, we need to recreate a pngstruct and modify its IHDR.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

scroggo_chromium

PTAL The latest patch set treats errors as errors, rather than attempting to correct them ...

3 years, 10 months ago (2017-02-17 23:00:57 UTC) #11

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode42 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:42: #include <memory> Remove these two #include, PNGImageDecoder.h provides them ...

3 years, 10 months ago (2017-02-21 13:53:57 UTC) #12

Noel Gordon

Description was changed from ========== Add support for Animated PNG. Split decoding PNG images into ...

3 years, 10 months ago (2017-02-23 06:18:44 UTC) #13

Description was changed from

==========
Add support for Animated PNG.

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR***) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.
*** FIXME: If we start decoding on an independent frame that does *not* fill
the image, we need to recreate a pngstruct and modify its IHDR.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR***) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.
*** FIXME: If we start decoding on an independent frame that does *not* fill
the image, we need to recreate a pngstruct and modify its IHDR.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

scroggo_chromium

PTAL https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode42 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:42: #include <memory> On 2017/02/21 13:53:54, noel gordon wrote: ...

3 years, 10 months ago (2017-02-23 22:09:56 UTC) #14

PTAL

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:42:
#include <memory>
On 2017/02/21 13:53:54, noel gordon wrote:
> Remove these two #include, PNGImageDecoder.h provides them and I won't cry.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:53: //
It might be more logical to default to cAnimationNone, but BitmapImage
On 2017/02/21 13:53:55, noel gordon wrote:
> // It would be logical to default ... 

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:68:
parse(PNGImageReader::PNGParseQuery::PNGMetaDataQuery);
On 2017/02/21 13:53:54, noel gordon wrote:
> parse(ParseQuery::MetaData);

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:73:
parse(PNGImageReader::PNGParseQuery::PNGMetaDataQuery);
On 2017/02/21 13:53:55, noel gordon wrote:
> parse(ParseQuery::MetaData);

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:99: 
On 2017/02/21 13:53:55, noel gordon wrote:
> Move the parse() routine definition to here.

Done. For my edification, why is here better?

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:100:
void PNGImageDecoder::clearFrameBuffer(size_t frameIndex) {
On 2017/02/21 13:53:55, noel gordon wrote:
> frameIndex -> index.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:107:
if (PNG_INTERLACE_ADAM7 ==
On 2017/02/21 13:53:54, noel gordon wrote:
> Let's prefer the early return:
> 
>   png_byte interlaceType =
>       png_get_interlace_type(m_reader->pngPtr(), m_reader->infoPtr());
>   if (PNG_INTERLACE_ADAM7 != interlaceType)
>     return;
>   
>   unsigned colorChannels = m_hasAlphaChannel ? 4 : 3;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:110:
m_reader->createInterlaceBuffer(colorChannels * size().width() *
On 2017/02/21 13:53:55, noel gordon wrote:
> colorChannels * size().area() ?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:112:
return m_reader->interlaceBuffer();
On 2017/02/21 13:53:55, noel gordon wrote:
> If this failed, setFailed() will be called in ImageDecoder code
> 
>    if (!onInitFrameBuffer())
>       return setFailed();
> 
> looking at the changes to ImageDecoder.  That is troublesome (see below).

setFailed() can be called in other ways from initFrameBuffer, too. As you point
out below, this is bad when we try to access parts of PNGImageReader afterwards.

GIFImageReader is in a similar situation - it calls initFrameBuffer, which may
delete the reader itself. AFAICT, this is safe because it just returns if that
happens. But maybe it would be safer to just make the caller call setFailed?
(That is what I have done in the next patch set.)

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:117:
bool PNGImageDecoder::canReusePreviousFrameBuffer(size_t frameIndex) const {
On 2017/02/21 13:53:55, noel gordon wrote:
> frameIndex -> index.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:123:
void PNGImageDecoder::parse(PNGImageReader::PNGParseQuery query) {
On 2017/02/21 13:53:55, noel gordon wrote:
> void PNGImageDecoder::parse(ParseQuery query) {

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:144:
ImageFrame* buffer = &m_frameBufferCache[index];
On 2017/02/21 13:53:55, noel gordon wrote:
> ImageFrame& buffer = m_frameBufferCache[index];

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:147:
buffer->setOriginalFrameRect(frameInfo.frameRect);
On 2017/02/21 13:53:54, noel gordon wrote:
> "buffer->X" to "buffer.X" all through here.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:148:
buffer->setDuration(frameInfo.duration);
On 2017/02/21 13:53:54, noel gordon wrote:
> Space before this line.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:151:
buffer->setRequiredPreviousFrameIndex(
On 2017/02/21 13:53:54, noel gordon wrote:
> Space before 151.
> 
>   size_t previousFrameIndex = findRequiredPreviousFrame(index, false);
>   buffer.setRequiredPreviousFrameIndex(previousFrameIndex);
> }

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:226:
int trnsCount = 0;
On 2017/02/21 13:53:54, noel gordon wrote:
> Are |trns| and |trnsCount| used anywhere?

For completeness, this has been addressed in crrev.com/2716533002

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:239:
// Only set the size and the color space of the image once. Since non-first
On 2017/02/21 13:53:54, noel gordon wrote:
> // Only set the size and the color space of the image once since non-first
> // frames also use this method: there is no per-frame color space, and the
> // image size is determined from the header width and height.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:251:
if (!setSize(width, height)) {
On 2017/02/21 13:53:55, noel gordon wrote:
> // Set the image size now that the image header is available.
> if (!setSize(width, height)) {

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:265:
if (colorSpace) {
On 2017/02/21 13:53:54, noel gordon wrote:
> Is sk_sp<T> convertible to bool?  If so, we can use that fact, and we should
> std::move colorSpace into the callee
> 
> if (sk_sp<SkColorSpace> colorSpace = readColorSpace(png, info))
>   setEmbeddedColorSpace(std::move(colorSpace));
> 
> since colorSpace is not retained / used by any code herein after this point
> (those crazy color kids).

Addressed by crrev.com/2716533002

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: 
On 2017/02/21 13:53:55, noel gordon wrote:
> 
> Add a DCHECK(isDecodedSizeAvailable()) around here?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:277:
// TODO (msarett):
On 2017/02/21 13:53:55, noel gordon wrote:
> Comment lines 277-280: remove.  Fixed long ago.

Acknowledged.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:300:
// Update our info now.
On 2017/02/21 13:53:54, noel gordon wrote:
> // Update our info now (so we can read color channel info).
> png_read_update_info(png, info);
> 
> channels = png_get_channels(png, info);
> DCHECK(channels == 3 || channels == 4);
> m_hasAlphaChannel = (channels == 4);

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:306:
m_currentBufferSawAlpha = false;
On 2017/02/21 13:53:54, noel gordon wrote:
> Should this m_currentBufferSawAlpha = false be moved into onInitFrame?  That's
> what GIF does to init per-frame alpha tracking I think?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/02/21 13:53:54, noel gordon wrote:
> The proceeding comment is a sad story.  Is there any way round the problem it
> speaks of?  Maybe FIXME: for now.

This was caught by the following tests:
DeferredImageDecoderTest.decodeOnOtherThread
DeferredImageDecoderTest.drawIntoSkPicture
DeferredImageDecoderTest.frameOpacity

which use a PNG image, which is why we did not catch this before now. Using a
similar GIF fails frameOpacity - when we try to correct the alpha value,
originalFrameRect does not cover the full image, so we continue to assume the
frame has alpha.

Rather than working around the problem, there is a better fix - make
setMemoryAllocator (/its caller) call frameCount() rather than manually
resizing.

I've made the change in the next patch set, but it is probably worth landing as
a separate change.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324:
if (!initFrameBuffer(m_currentFrame))
On 2017/02/21 13:53:54, noel gordon wrote:
> First issue: code in diff-left short circuited real work here by calling
> 
> if (buffer.getStatus() == ImageFrame::FrameEmpty) {
>    ...
> }
> 
> so it was done once per image frame.  Now we have
> 
>   if (!initFrameBuffer(m_currentFrame))
> 
> being called to do real work on _each row_ of an image frame.  Sadness w.r.t
to
> PNG decode perf: is there some way to FIXME this?

What is the real work you're concerned about here? The function call?
(initFrameBuffer makes the same check to short circuit.) We could avoid that by
modifying this line to say

  if (buffer.getStatus() == ImageFrame::FrameEmpty
      && !initFrameBuffer(m_currentFrame))

Another possibility - call initFrameBuffer before we ever start processing
IDAT/fdAT. Then we can even

  DCHECK_EQ(FramePartial, buffer.getStatus());

With this approach, we'll need to catch the empty frame a different way.
Currently, frameComplete checks the Status - if it's FrameEmpty, we know
rowAvailable was never called. Instead, we can set a boolean for whether
rowAvailable has been called. (We can even take that further and keep track of
the rows decoded to know whether the image is truly complete, which would assist
in fixing the last three images in [1], though it deviates from the current
behavior for static PNGs.)

This also addresses the issue brought up elsewhere, that initFrameBuffer may
delete the reader.

WDYT?

[1] https://philip.html5.org/tests/apng/tests.html

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:325:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/02/21 13:53:54, noel gordon wrote:
> Second issue here: initFrameBuffer() calls onInitFrame(), and the latter can
> fail and call setFailed() as I noted earlier, right?
> 
> That means m_reader is NULL here and we crash the renderer via
> m_reader->pngPtr() == NULL->pngPtr().
> 
> You have to save the JUMBUF locally before calling initFrameBuffer(), and use
> that saved JMPBUF to longjmp() out if you want to avoid a crash.  Even so,
this
> would again have some perf impact: is there some way to FIXME this?

Some options, suggested in above responses:
- stop calling setFailed in initFrameBuffer (as is done in the next patch set)
- call initFrameBuffer outside of rowAvailable

WDYT?

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/02/21 13:53:55, noel gordon wrote:
> Third issue here: diff-left calls buffer.setHasAlpha(false);  I did not see
that
> call anywhere through here or in the functions called.   Did I miss it, or do
we
> not need to call it anymore?

Alpha is tracked differently in the multi-frame code. The code on the left calls
setHasAlpha(false) at the beginning, and then setHasAlpha(true) if a non-opaque
pixel is seen. This means that while the frame's status is FramePartial,
m_hasAlpha is false, although the SkBitmap still has an alpha type that
recognizes that there is alpha (since the image has not been completely
decoded).

In the new code, m_hasAlpha remains true (set in zeroFillFrameRect), which
matches the SkBitmap. After the frame is completely decoded, m_hasAlpha may be
set to false in correctAlphaWhenFrameBufferSawNoAlpha, which also takes into
account the frame rect and the frame that the current frame depends on. This
behavior matches GIF and allows us to share code.

Either way, we end up with the correct value once the frame is FrameComplete
(almost - see [8]). But there is a difference, as you point out - while an
ImageFrame is FramePartial, its m_hasAlpha (and therefore hasAlpha()) returns
true, whereas the old code returned false (if a transparent pixel has not yet
been seen). It is not obvious to me why the old code is desirable. The
setHasAlpha(false) after zeroFillFrameRect was introduced here [1] (EDIT: Can be
traced further back to [7]), at which point it was the same in other decoders
(e.g. JPEG). But JPEG was changed later to fix a bug [2] (it now calls
setHasAlpha(true); that is not actually necessary, since this already happens in
zeroFillFrameRect). (Without digging too much into each case, note that the rest
of the decoders make different decisions about what to do during FramePartial:

BMP - set to false, like current PNG
GIF - leave alone, so it remains true
WEBP - set to true
.)

In today's code, I don't think there's a difference. m_hasAlpha appears to only
be read (besides for the purpose of copying) when the status is FrameComplete.
e.g.

- computeAlphaType [3]
- WebGLImageConversion::ImageExtractor::extractImage (by way of hasAlpha) [4]

ImageDecoder::frameHasAlphaAtIndex [5] is an interesting case. It is only called
by DeferredImageDecoder [6], which calls it on m_actualDecoder. m_actualDecoder
is never used for decoding, so the default version of frameIsCompleteIndex
(which just checks whether the status is FrameComplete) always returns false (no
decoding means the frame will always be FrameEmpty). So for the existing
PNGImageDecoder (which does not override frameIsCompleteAtIndex), no callers
know the difference between true or false m_hasAlpha during FramePartial.
(Taking that a step further, that call to hasAlpha is also on m_actualDecoder.
Now that frameIsCompleteAtIndex may return true for an animated PNG, hasAlpha
might be called. But again, since m_actualDecoder is not used for decoding,
hasAlpha will return its initial value of true.)

[1]
https://chromium.googlesource.com/chromium/src/+/f09863db44eed54d690e52815adb...
[2]
https://chromium.googlesource.com/chromium/src/+/08f8f9ac56eb0d6bf6d876e2dffd...
[3]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/image...
[4]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
[5]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/image...
[6]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

[7]
https://chromium.googlesource.com/chromium/src/+/d963d0f9e6daae10178b3c30e008...
From that CL's description:

        Various minor cleanups to the Skia files.  Mostly non-functional, except
        for two specific changes:
        * JPEGs and PNGs were always marked as transparent; now they are only
          marked as transparent when they actually are.  I doubt this has much
          of an effect but in theory it could be used to optimize their display.

This does not lead me to think this is an important feature to preserve.

[8] The old code actually has a bug. Since we mark the ImageFrame as *not*
having alpha, and then only correct it if we find alpha, we can get into a
situation where we've left the ImageFrame thinking it's opaque when we still
have the transparent pixels from the zeroFillFrameRect call.
https://philip.html5.org/tests/apng/058.png is a good example of this. The IDAT
is incomplete, so we only decode the first 32 out of 64 rows - the last 32 are
still transparent.

And, now that I think about it some more, I realize that I have retained this
bug. libpng does not care that not all of the rows were in the IDAT (depending
on the state of the zstream, libpng may report a warning or an error, but
neither happens in this case), so we "correct" m_hasAlpha to false after we did
not see any transparent pixels.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:328:
const IntRect& frameRect = buffer.originalFrameRect();
On 2017/02/21 13:53:55, noel gordon wrote:
> There is a nice DCHECK you used elsewhere to check that frameRect is contained
> in the image size().  Would it make sense to add it here, to remind readers
that
> frameRect has been clipped to the image size()?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:342:
// Nothing to do if the row is unchanged, or the row is outside the image
On 2017/02/21 13:53:55, noel gordon wrote:
> // Nothing to do if the row is unchanged, or the row is outside the image
> // bounds. In the case that a frame presents more data than the indicated
> // frame size, ignore the extra rows and use the frame size as the source
> // of truth. libpng can send extra rows: ignore them too, this to prevent
> // memory writes outside of the image bounds (security).
> 
> 

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:405:
srcPtr = (png_bytep)dstRow;
On 2017/02/21 13:53:55, noel gordon wrote:
> nit and not your doing, mind: C-style cast not allowed per blink style.
Addressed in crrev.com/2716533002

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:475:
if (index == 0 && m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/02/21 13:53:55, noel gordon wrote:
> Comparisons to 0: use !index.
> 
> if (!index && m_reader->parseCompleted() && ...
>   return ImageDecoder::frameIsCompleteAtIndex(index);
> 
> The next sentence I read is "// For first frames of animated images," but I
> think the if above can be false and so we can fall into the code below too,
aka
> for normal PNG images. 

Good point. I see two ways this condition could return false for a normal PNG
image:
- index > 0 for a normal PNG image
  - solution: drop the check for index, which is handled by ImageDecoder anyway
- we have not yet parsed the size
  - solution: check for isDecodedSizeAvailable. If false, the frame is not
complete.
    I also had to rearrange PNGImageReader::parse to ensure that once the size
is
    available in a static PNG m_parseCompleted is set to true.

Are there others I have missed?

> Any problems with that, should I worry?

I generally worry about this method. Special-casing a static image matches the
behavior of WEBP (mostly - IIUC, WEBP will treat a single frame but animated
image the same as an animated image; I don't know whether that is possible for
WEBP, but APNG can have an acTL chunk and an fcTL chunk but only one frame), and
the existing code for PNG. But it is different from GIF, which treats a single
frame image the same as an animated one. (Note that in APNG, unlike GIF, there
is no need to look ahead to see if there are more frames, since there either was
no acTL or it indicated one frame. So in order to know where the end of the
frame is, we would have to parse ahead just to find it, whereas GIF learns this
info on the way to looking for more frames.)

If we drop the if above (or missed a case), PNG will behave no differently for
some clients (assuming I initialize m_frameInfo[0].byteLength, which I do in the
latest patch set). It will always return false, the same as when it is called by
DeferredImageDecoder [1][2], where the status will never be FrameComplete,
because m_actualDecoder was never used to decode.


[1]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
[2]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:479:
// fully received through firstFrameFullyReceived().  Non-first frames are
On 2017/02/21 13:53:55, noel gordon wrote:
> Let's break this comment into two parts.
> 
>   // For first frames ...
>   // fully received through firstFrameFullyReceived().
>   if (!index)
>      return m_reader.firstFrameFullyReceived();
> 
>   // Non-first frames are reported by |m_reader| ...
>   // ... below the frame count.
>   return index < m_reader->frameCount();
>   
>    

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:488:
return (index < m_frameBufferCache.size()
On 2017/02/21 13:53:55, noel gordon wrote:
> This ternary if statement is unclear due to 80-column line-limit foo-bar-ness:
> make it 
> 
>   if (index < m_frameBufferCache.size())
>     return m_frameBufferCache[index].duration();
>   return 0;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:493:
void PNGImageDecoder::complete() {
On 2017/02/21 13:53:54, noel gordon wrote:
> complete -> frameComplete

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:507:
correctAlphaWhenFrameBufferSawNoAlpha(m_currentFrame);
On 2017/02/21 13:53:55, noel gordon wrote:
> This code can change the image frame alpha state by the looks, and since we
have
> already called buffer.setStatus(ImageFrame::FrameComplete), the SkBitmap
backing
> will never be told, it seems to me.
> 
> Perhaps do this code first, and do the ImageFrame::FrameComplete step last.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:55: void
complete();
On 2017/02/21 13:53:55, noel gordon wrote:
> complete -> frameComplete

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:57: //
Additional methods used for APNG
On 2017/02/21 13:53:55, noel gordon wrote:
> Remove this comment.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:60:
private:
On 2017/02/21 13:53:56, noel gordon wrote:
> private:
>   using ParseQuery = PNGImageReader::ParseQuery;
> 
>   // ImageDecoder:
>   ...

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:62: void
decodeSize() override {
On 2017/02/21 13:53:56, noel gordon wrote:
> Lets move parse() up to here to keep this group together
> 
>  void decodeSize() override { parse(ParseQuery::Size); }
>  void decode(size_t) override;
>  void parse(ParseQuery);
>  size_t decodeFrameCount() override;
>  ...

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:70: bool
canReusePreviousFrameBuffer(size_t index) const override;
On 2017/02/21 13:53:56, noel gordon wrote:
> bool canReusePreviousFrameBuffer(size_t) const override;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:76: 
On 2017/02/21 13:53:55, noel gordon wrote:
> Remove this empty line.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:48:
namespace {
On 2017/02/21 13:53:56, noel gordon wrote:
> I did a quick pass over this file only for now.

Acknowledged.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:65:
void PNGAPI pngComplete(png_structp png, png_infop) {
On 2017/02/21 13:53:56, noel gordon wrote:
> Now that the PNG decoder is multi-frame, lets call this ...
> 
> pngFrameComplete

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:66:
imageDecoder(png)->complete();
On 2017/02/21 13:53:56, noel gordon wrote:
> imageDecoder(png)->frameComplete();
> 
> and adjust the PNG decoder to match this name change.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:98:
pngComplete);
On 2017/02/21 13:53:57, noel gordon wrote:
> pngFrameComplete, here and anywhere else it's used.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:145: if
(setjmp(JMPBUF(m_png))) {
On 2017/02/21 13:53:56, noel gordon wrote:
> if (setjmp(JMPBUF(m_png)))
>   return false;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:154: if
(index > 0 && m_progressiveDecodeOffset > 0) {
On 2017/02/21 13:53:56, noel gordon wrote:
> DCHECK(m_isAnimated);
> 
> if (index > 0 && m_progressiveDecodeOffset > 0)
>   clearDecodeState(0);
> 
> ...
> 
> If index and m_progressiveDecodeOffset are unsigned quantities, no need for
the
> > 0 (comparisons to 0 again).
> 

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:173: if
(index == 0 &&
On 2017/02/21 13:53:56, noel gordon wrote:
> comparisons to 0;  use !index && ...

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:184:
png_byte IEND[12] = {0, 0, 0, 0, 'I', 'E', 'N', 'D', 174, 66, 96, 130};
On 2017/02/21 13:53:56, noel gordon wrote:
> Could this be 
> 
> static png_byte IEND[12] ?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:265: //
Skip the sequence number
On 2017/02/21 13:53:57, noel gordon wrote:
> Sentences end with a period.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:295:
png_byte chunkIDAT[] = {0, 0, 0, 0, 'I', 'D', 'A', 'T'};
On 2017/02/21 13:53:57, noel gordon wrote:
> static png_byte chunkIDAT[] ?

We could, but then we'd need to copy it, since the next line modifies it.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:326: //
Compute the CRC and compare to the stored value
On 2017/02/21 13:53:56, noel gordon wrote:
> Sentences.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:350:
m_nextSequenceNumber++;
On 2017/02/21 13:53:56, noel gordon wrote:
> ++m_nextSequenceNumber;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:355:
bool PNGImageReader::parse(SegmentReader& data, PNGParseQuery query) {
On 2017/02/21 13:53:57, noel gordon wrote:
>  bool PNGImageReader::parse(SegmentReader& data, ParseQuery query)

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:364: if
(query == PNGParseQuery::PNGSizeQuery ||
On 2017/02/21 13:53:57, noel gordon wrote:
> if (query == ParseQuery::Size ...

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:378:
m_frameInfo.append(frame);
On 2017/02/21 13:53:56, noel gordon wrote:
> append -> push_back

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:383:
char readBuffer[kBufferSize];
On 2017/02/21 13:53:56, noel gordon wrote:
> DCHECK_EQ(query, ParseQuery::MetaData);
> DCHECK(m_isAnimated);
> 
> // At this point, the query is MetaData and the image is animated.
> ^^^^ remove.
> 
> 

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:385: //
At this point, the query is FrameMetaDataQuery and the image is animated.
On 2017/02/21 13:53:56, noel gordon wrote:
> 
> // Loop over the data and manually register all frames. ...
> // blah blah
> char readBuffer[kBufferSize];
> while (reader.size() >= m_readOffset + 8) {
>   ...

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:398: if
(IDAT && !m_expectIdats) {
On 2017/02/21 13:53:56, noel gordon wrote:
> { } not needed.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:403: if
(fdAT && m_expectIdats) {
On 2017/02/21 13:53:56, noel gordon wrote:
> { } not needed.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:417:
m_frameInfo.append(m_newFrame);
On 2017/02/21 13:53:56, noel gordon wrote:
> append -> push_back.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:422: if
(reader.size() < m_readOffset + 8 + 4) {
On 2017/02/21 13:53:56, noel gordon wrote:
> { } not needed.
> 
> if (reader.size() < m_readOffset + 8 + 4)
>   return true;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:438:
m_frameInfo.append(m_newFrame);
On 2017/02/21 13:53:56, noel gordon wrote:
> append -> push_back

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:479: //
@return: number of bytes processed.
On 2017/02/21 13:53:57, noel gordon wrote:
> @?

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:509: if
(setjmp(JMPBUF(m_png))) {
On 2017/02/21 13:53:56, noel gordon wrote:
> if (setjmp(JMPBUF(m_png)))
>   return false;

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:698: };
 // namespace blink
On 2017/02/21 13:53:56, noel gordon wrote:
> ; not needed.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:44: #if
!defined(PNG_LIBPNG_VER_MAJOR) || !defined(PNG_LIBPNG_VER_MINOR)
On 2017/02/21 13:53:57, noel gordon wrote:
> nit: this #if-defy section @44-53 got moved into the blink namespace.  Any
> reason?  Maybe move this section before the "namespace blink {" as in
diff-left?
> 

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:74: enum
class PNGParseQuery { PNGSizeQuery, PNGMetaDataQuery };
On 2017/02/21 13:53:57, noel gordon wrote:
> PNG prefix is somewhat redundant.  Make this:
> 
> enum class ParseQuery { Size, MetaData };

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:76: bool
parse(SegmentReader&, PNGParseQuery);
On 2017/02/21 13:53:57, noel gordon wrote:
> bool parse(SegmentReader&, ParseQuery);

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:99:
m_interlaceBuffer = wrapArrayUnique(new png_byte[size]);
On 2017/02/21 13:53:57, noel gordon wrote:
> does wrapArrayUnique need a WTF:: prefix?

No. None of the instances I see in code search use a WTF:: prefix.

https://cs.chromium.org/search/?q=wrapArrayUnique+package:%5Echromium$&type=cs

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:157: };
On 2017/02/21 13:53:57, noel gordon wrote:
> }: ?  If this ends a namespace, then make it
> 
> }  // namespace blink

Done.

scroggo_chromium

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 10 months ago (2017-02-23 22:10:06 UTC) #15

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR***) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.
*** FIXME: If we start decoding on an independent frame that does *not* fill
the image, we need to recreate a pngstruct and modify its IHDR.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

Nate Chapin

https://codereview.chromium.org/2618633004/diff/100001/content/browser/loader/mime_sniffing_resource_handler.cc File content/browser/loader/mime_sniffing_resource_handler.cc (right): https://codereview.chromium.org/2618633004/diff/100001/content/browser/loader/mime_sniffing_resource_handler.cc#newcode48 content/browser/loader/mime_sniffing_resource_handler.cc:48: "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp," I believe igrigorik recommended adding image/apng to this ...

3 years, 10 months ago (2017-02-23 22:12:52 UTC) #16

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode324 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324: if (!initFrameBuffer(m_currentFrame)) On 2017/02/23 22:09:54, scroggo_chromium wrote: > On ...

3 years, 10 months ago (2017-02-24 14:57:25 UTC) #17

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324:
if (!initFrameBuffer(m_currentFrame))
On 2017/02/23 22:09:54, scroggo_chromium wrote:
> On 2017/02/21 13:53:54, noel gordon wrote:
> > First issue: code in diff-left short circuited real work here by calling
> > 
> > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> >    ...
> > }
> > 
> > so it was done once per image frame.  Now we have
> > 
> >   if (!initFrameBuffer(m_currentFrame))
> > 
> > being called to do real work on _each row_ of an image frame.  Sadness w.r.t
> to
> > PNG decode perf: is there some way to FIXME this?
> 
> What is the real work you're concerned about here? The function call?
> (initFrameBuffer makes the same check to short circuit.) We could avoid that
by
> modifying this line to say
> 
>   if (buffer.getStatus() == ImageFrame::FrameEmpty
>       && !initFrameBuffer(m_currentFrame))
> 
> Another possibility - call initFrameBuffer before we ever start processing
> IDAT/fdAT. Then we can even
> 
>   DCHECK_EQ(FramePartial, buffer.getStatus());
> 
> With this approach, we'll need to catch the empty frame a different way.
> Currently, frameComplete checks the Status - if it's FrameEmpty, we know
> rowAvailable was never called. Instead, we can set a boolean for whether
> rowAvailable has been called. (We can even take that further and keep track of
> the rows decoded to know whether the image is truly complete, which would
assist
> in fixing the last three images in [1], though it deviates from the current
> behavior for static PNGs.)
> 
> This also addresses the issue brought up elsewhere, that initFrameBuffer may
> delete the reader.
> 
> WDYT?
> 
> [1] https://philip.html5.org/tests/apng/tests.html
> 

I moved initFrameBuffer into decode in patch set 8. This follows
WEBPImageDecoder's lead, and means rowAvailable does not need to do any work for
initializing the frame.

https://codereview.chromium.org/2618633004/diff/100001/content/browser/loader...
File content/browser/loader/mime_sniffing_resource_handler.cc (right):

https://codereview.chromium.org/2618633004/diff/100001/content/browser/loader...
content/browser/loader/mime_sniffing_resource_handler.cc:48:
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,"
On 2017/02/23 22:12:52, Nate Chapin wrote:
> I believe igrigorik recommended adding image/apng to this accept header, too
> (after image/webp I assume).

Done.

Nate Chapin

Accept header looks good, thanks! (I don't have OWNERS on those files though)

3 years, 10 months ago (2017-02-24 17:39:07 UTC) #18

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode99 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:99: On 2017/02/23 22:09:53, scroggo_chromium wrote: > On 2017/02/21 13:53:55, ...

3 years, 9 months ago (2017-02-27 07:45:31 UTC) #19

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:99: 
On 2017/02/23 22:09:53, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > Move the parse() routine definition to here.
> 
> Done. For my edification, why is here better?

It's then adjacent to the functions that use it (saving me a lot of scrolling up
and down when reviewing this code :)

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:112:
return m_reader->interlaceBuffer();
On 2017/02/23 22:09:52, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > If this failed, setFailed() will be called in ImageDecoder code
> > 
> >    if (!onInitFrameBuffer())
> >       return setFailed();
> > 
> > looking at the changes to ImageDecoder.  That is troublesome (see below).
> 
> setFailed() can be called in other ways from initFrameBuffer, too. As you
point
> out below, this is bad when we try to access parts of PNGImageReader
afterwards.

Yeah, possible security problems.

> GIFImageReader is in a similar situation - it calls initFrameBuffer, which may
> delete the reader itself. AFAICT, this is safe because it just returns if that
> happens. But maybe it would be safer to just make the caller call setFailed?
> (That is what I have done in the next patch set.)

Yes I think if callers are required to do their own failure handling, eg.,
calling setFailed() or whatever their specific decoder needs, that'd be safer
(as this PNG decoder example pointed out).

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: 
On 2017/02/23 22:09:53, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > 
> > Add a DCHECK(isDecodedSizeAvailable()) around here?
> 
> Done.

One second thoughts, this is in the middle of some the color handling code.  I
figure there is more work to do in that code, so maybe move this DCHECK to after
all the color-specific code, sorry.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:277:
// TODO (msarett):
On 2017/02/23 22:09:52, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > Comment lines 277-280: remove.  Fixed long ago.
> 
> Acknowledged.

And also addressed by crrev.com/2716533002

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/02/23 22:09:53, scroggo_chromium wrote:
> On 2017/02/21 13:53:54, noel gordon wrote:
> > The proceeding comment is a sad story.  Is there any way round the problem
it
> > speaks of?  Maybe FIXME: for now.
> 
> This was caught by the following tests:
> DeferredImageDecoderTest.decodeOnOtherThread
> DeferredImageDecoderTest.drawIntoSkPicture
> DeferredImageDecoderTest.frameOpacity

OK good, we have tests.

> which use a PNG image, which is why we did not catch this before now. Using a
> similar GIF fails frameOpacity - when we try to correct the alpha value,
> originalFrameRect does not cover the full image, so we continue to assume the
> frame has alpha.

Just in passing, it looks to me like the GIF decoder's alpha tracking is
incorrect - it set's the frame status to FrameComplete _and then updates the
frame alpha state_.  Same bug I saw in this new PNG code.

> Rather than working around the problem, there is a better fix - make
> setMemoryAllocator (/its caller) call frameCount() rather than manually
> resizing.

OK, sounds better, I will look at the fix when we get there.

> I've made the change in the next patch set, but it is probably worth landing
as
> a separate change.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324:
if (!initFrameBuffer(m_currentFrame))
On 2017/02/23 22:09:54, scroggo_chromium wrote:
> On 2017/02/21 13:53:54, noel gordon wrote:
> > First issue: code in diff-left short circuited real work here by calling
> > 
> > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> >    ...
> > }
> > 
> > so it was done once per image frame.  Now we have
> > 
> >   if (!initFrameBuffer(m_currentFrame))
> > 
> > being called to do real work on _each row_ of an image frame.  Sadness w.r.t
> to
> > PNG decode perf: is there some way to FIXME this?
> 
> What is the real work you're concerned about here? The function call?
> (initFrameBuffer makes the same check to short circuit.) We could avoid that
by
> modifying this line to say
> 
>   if (buffer.getStatus() == ImageFrame::FrameEmpty
>       && !initFrameBuffer(m_currentFrame))

Right.  So you changed initFrameBuffer to not call setFailed() anymore (that's
good), the so the caller now has to do the error handling, so we get:

if (buffer.getStatus() == ImageFrame::FrameEmpty) {
  if (!initFrameBuffer(m_currentFrame))
    longjmp(JMPBUF(m_reader->pngPtr()), 1);
}

Given that, then there is little reason for PNGImageDecoder::onInitFrameBuffer
to even exist.  You could grab it's body and put it all herein.

if (buffer.getStatus() == ImageFrame::FrameEmpty) {
  if (!initFrameBuffer(m_currentFrame))
    longjmp(JMPBUF(m_reader->pngPtr()), 1);

  + what onInitFrameBuffer was doing
}

and we could remove PNGImageDecoder::onInitFrameBuffer, right?

Also to catch the empty frame case in frameComplete(), you can check for
FrameEmpty as you were doing, no need to add a new variable to track that case.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:325:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/02/23 22:09:52, scroggo_chromium wrote:
> On 2017/02/21 13:53:54, noel gordon wrote:
> > Second issue here: initFrameBuffer() calls onInitFrame(), and the latter can
> > fail and call setFailed() as I noted earlier, right?
> > 
> > That means m_reader is NULL here and we crash the renderer via
> > m_reader->pngPtr() == NULL->pngPtr().
> > 
> > You have to save the JUMBUF locally before calling initFrameBuffer(), and
use
> > that saved JMPBUF to longjmp() out if you want to avoid a crash.  Even so,
> this
> > would again have some perf impact: is there some way to FIXME this?
> 
> Some options, suggested in above responses:
> - stop calling setFailed in initFrameBuffer (as is done in the next patch set)

good

> - call initFrameBuffer outside of rowAvailable

see above, have rowAvailable handle all the frame init stuff in a FrameEmpty
short-circuit.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/02/23 22:09:53, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > Third issue here: diff-left calls buffer.setHasAlpha(false);  I did not see
> that
> > call anywhere through here or in the functions called.   Did I miss it, or
do
> we
> > not need to call it anymore?
> 
> Alpha is tracked differently in the multi-frame code. The code on the left
calls
> setHasAlpha(false) at the beginning, and then setHasAlpha(true) if a
non-opaque
> pixel is seen. This means that while the frame's status is FramePartial,
> m_hasAlpha is false, although the SkBitmap still has an alpha type that
> recognizes that there is alpha (since the image has not been completely
> decoded).
> 
> In the new code, m_hasAlpha remains true (set in zeroFillFrameRect), which
> matches the SkBitmap. After the frame is completely decoded, m_hasAlpha may be
> set to false in correctAlphaWhenFrameBufferSawNoAlpha, which also takes into
> account the frame rect and the frame that the current frame depends on. This
> behavior matches GIF and allows us to share code.

modulo the bug in the GIF code, this paragraph describes the correct behavior.
 
> Either way, we end up with the correct value once the frame is FrameComplete
> (almost - see [8]). But there is a difference, as you point out - while an
> ImageFrame is FramePartial, its m_hasAlpha (and therefore hasAlpha()) returns
> true, whereas the old code returned false (if a transparent pixel has not yet
> been seen). It is not obvious to me why the old code is desirable.

The new code returns the correct value, right? 

 The
> setHasAlpha(false) after zeroFillFrameRect was introduced here [1] (EDIT: Can
be
> traced further back to [7]), at which point it was the same in other decoders
> (e.g. JPEG). But JPEG was changed later to fix a bug [2] (it now calls
> setHasAlpha(true); that is not actually necessary, since this already happens
in
> zeroFillFrameRect). (Without digging too much into each case, note that the
rest
> of the decoders make different decisions about what to do during FramePartial:

The fix in [2] is correct, and the setHasAlpha(true) is redundant but safe.  The
change log describes what can happen when we get the alpha wrong.  The logic of
the change in [2] applies to all image decoder types while
we are decoding their frames.
 
> BMP - set to false, like current PNG
> GIF - leave alone, so it remains true
> WEBP - set to true
> .)
> 
> In today's code, I don't think there's a difference. m_hasAlpha appears to
only
> be read (besides for the purpose of copying) when the status is FrameComplete.
> e.g.

So the result is we do report the correct alpha state, even though the decoders
you listed above seem all over the place wrt to setting initial alpha during
frame initialization.  They could be cleaned up.  Main point here is

"m_hasAlpha appears to only be read (besides for the purpose of copying) when
the status is FrameComplete."

> - computeAlphaType [3]

nod.

> - WebGLImageConversion::ImageExtractor::extractImage (by way of hasAlpha) [4]

nod.

They are both correct.
 
> ImageDecoder::frameHasAlphaAtIndex [5] is an interesting case. It is only
called
> by DeferredImageDecoder [6], which calls it on m_actualDecoder.
m_actualDecoder
> is never used for decoding, so the default version of frameIsCompleteIndex
> (which just checks whether the status is FrameComplete) always returns false
(no
> decoding means the frame will always be FrameEmpty).

m_actualDecoder is used for decoding the image header on the renderer main
thread, but when we decide to decode the image off the main thread,
m_actualDecoder gets deleted IIRC, and the image frame generator is asked for
frame state.  Is the alpha state is being incorrectly reported from a frame
generator?  That seems like a DeferredImageDecoder issue if so.

> So for the existing
> PNGImageDecoder (which does not override frameIsCompleteAtIndex), no callers
> know the difference between true or false m_hasAlpha during FramePartial.
> (Taking that a step further, that call to hasAlpha is also on m_actualDecoder.
> Now that frameIsCompleteAtIndex may return true for an animated PNG, hasAlpha
> might be called. But again, since m_actualDecoder is not used for decoding,
> hasAlpha will return its initial value of true.)
 

> This does not lead me to think this is an important feature to preserve.

alpha state detection you mean?
 
> [8] The old code actually has a bug. Since we mark the ImageFrame as *not*
> having alpha, and then only correct it if we find alpha, we can get into a
> situation where we've left the ImageFrame thinking it's opaque when we still
> have the transparent pixels from the zeroFillFrameRect call.
> https://philip.html5.org/tests/apng/058.png is a good example of this. The
IDAT
> is incomplete, so we only decode the first 32 out of 64 rows - the last 32 are
> still transparent.
> And, now that I think about it some more, I realize that I have retained this
> bug. libpng does not care that not all of the rows were in the IDAT (depending

That's correct, the too few IDAT rows problem ...

> on the state of the zstream, libpng may report a warning or an error, but
> neither happens in this case), so we "correct" m_hasAlpha to false after we
did
> not see any transparent pixels.

... could report the wrong alpha, but the too few IDAT rows case is also an
invalid APNG which we should not be decoding, right?

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:475:
if (index == 0 && m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/02/23 22:09:53, scroggo_chromium wrote:
> On 2017/02/21 13:53:55, noel gordon wrote:
> > Comparisons to 0: use !index.
> > 
> > if (!index && m_reader->parseCompleted() && ...
> >   return ImageDecoder::frameIsCompleteAtIndex(index);
> > 
> > The next sentence I read is "// For first frames of animated images," but I
> > think the if above can be false and so we can fall into the code below too,
> aka
> > for normal PNG images. 
> 
> Good point. I see two ways this condition could return false for a normal PNG
> image:
> - index > 0 for a normal PNG image
>   - solution: drop the check for index, which is handled by ImageDecoder
anyway
> - we have not yet parsed the size
>   - solution: check for isDecodedSizeAvailable. If false, the frame is not
> complete.
>     I also had to rearrange PNGImageReader::parse to ensure that once the size
> is
>     available in a static PNG m_parseCompleted is set to true.
> 
> Are there others I have missed?
> 
> > Any problems with that, should I worry?
> 
> I generally worry about this method. Special-casing a static image matches the
> behavior of WEBP (mostly - IIUC, WEBP will treat a single frame but animated
> image the same as an animated image; I don't know whether that is possible for
> WEBP, but APNG can have an acTL chunk and an fcTL chunk but only one frame),
and
> the existing code for PNG. But it is different from GIF, which treats a single
> frame image the same as an animated one. (Note that in APNG, unlike GIF, there
> is no need to look ahead to see if there are more frames, since there either
was
> no acTL or it indicated one frame. So in order to know where the end of the
> frame is, we would have to parse ahead just to find it, whereas GIF learns
this
> info on the way to looking for more frames.)
> 
> If we drop the if above (or missed a case), PNG will behave no differently for
> some clients (assuming I initialize m_frameInfo[0].byteLength, which I do in
the
> latest patch set). It will always return false, the same as when it is called
by
> DeferredImageDecoder [1][2], where the status will never be FrameComplete,
> because m_actualDecoder was never used to decode.
> 
> 
> [1]
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
> [2]
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

I expect that the following

bool PNGImageDecoder::frameIsCompleteAtIndex(size_t index) const {
  return ImageDecoder::frameIsCompleteAtIndex(index)
}

would still work.  Your image will animate, etc?

Now, I have no clue why animated image types (GIF say) require this routine to
become "frame is fully received at index", and thus all the
firstFrameFullyReceived() code that resulted here.

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode270 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: On 2017/02/27 07:45:31, noel gordon wrote: > On 2017/02/23 ...

3 years, 9 months ago (2017-02-27 20:31:05 UTC) #20

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: 
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > On 2017/02/21 13:53:55, noel gordon wrote:
> > > 
> > > Add a DCHECK(isDecodedSizeAvailable()) around here?
> > 
> > Done.
> 
> One second thoughts, this is in the middle of some the color handling code.  I
> figure there is more work to do in that code, so maybe move this DCHECK to
after
> all the color-specific code,

As I look for a better place to put it, I think this is the best, even though
it's in between color-specific code. The block above checks for the decoded
size, so it looks like this:

  if (!isDecodedSizeAvailable()) {
    // Do some work, including ensuring that the decoded size
    // is available. If not, signal an error and do not continue.
  }

  DCHECK(isDecodedSizeAvailable());

I guess either way we're splitting up something. Maybe this belongs just above
png_read_update_info?

> sorry.

No worries. Now is the time to address it!

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > On 2017/02/21 13:53:54, noel gordon wrote:
> > > The proceeding comment is a sad story.  Is there any way round the problem
> it
> > > speaks of?  Maybe FIXME: for now.
> > 
> > This was caught by the following tests:
> > DeferredImageDecoderTest.decodeOnOtherThread
> > DeferredImageDecoderTest.drawIntoSkPicture
> > DeferredImageDecoderTest.frameOpacity
> 
> OK good, we have tests.
> 
> > which use a PNG image, which is why we did not catch this before now. Using
a
> > similar GIF fails frameOpacity - when we try to correct the alpha value,
> > originalFrameRect does not cover the full image, so we continue to assume
the
> > frame has alpha.
> 
> Just in passing, it looks to me like the GIF decoder's alpha tracking is
> incorrect - it set's the frame status to FrameComplete _and then updates the
> frame alpha state_.  Same bug I saw in this new PNG code.

Agreed - GIF decoder should update its alpha state before setting its status to
FrameComplete, both logically, and because notifyPixelsChanged will not get
called after the pixels have changed.

I'm not sure that makes a difference in practice, though. notifyPixelsChanged
does a few things:

- call onNotifyPixelsChanged, a virtual method that does nothing and currently
has no overrides
- set the gen ID to 0, so the next time it is requested, a new ID is created.
The gen ID should not be requested in between the two calls, so for its purposes
it should not matter which order they happened in.
- notify listeners - this seems to tell caches to boot out the old gen ID

What would be dangerous is if we called notifyPixelsChanged, then someone else
tried to draw the bitmap (calling getGenerationID to get a newly generated ID),
and *then* we changed the alpha state, but I don't think that can happen.

I have fixed the bug in GIF (but it does not fix the bug in frameOpacity).

> 
> > Rather than working around the problem, there is a better fix - make
> > setMemoryAllocator (/its caller) call frameCount() rather than manually
> > resizing.
> 
> OK, sounds better, I will look at the fix when we get there.
> 
> > I've made the change in the next patch set, but it is probably worth landing
> as
> > a separate change.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324:
if (!initFrameBuffer(m_currentFrame))
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:54, scroggo_chromium wrote:
> > On 2017/02/21 13:53:54, noel gordon wrote:
> > > First issue: code in diff-left short circuited real work here by calling
> > > 
> > > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> > >    ...
> > > }
> > > 
> > > so it was done once per image frame.  Now we have
> > > 
> > >   if (!initFrameBuffer(m_currentFrame))
> > > 
> > > being called to do real work on _each row_ of an image frame.  Sadness
w.r.t
> > to
> > > PNG decode perf: is there some way to FIXME this?
> > 
> > What is the real work you're concerned about here? The function call?
> > (initFrameBuffer makes the same check to short circuit.) We could avoid that
> by
> > modifying this line to say
> > 
> >   if (buffer.getStatus() == ImageFrame::FrameEmpty
> >       && !initFrameBuffer(m_currentFrame))
> 
> Right.  So you changed initFrameBuffer to not call setFailed() anymore (that's
> good), the so the caller now has to do the error handling, so we get:
> 
> if (buffer.getStatus() == ImageFrame::FrameEmpty) {
>   if (!initFrameBuffer(m_currentFrame))
>     longjmp(JMPBUF(m_reader->pngPtr()), 1);
> }
> 
> Given that, then there is little reason for PNGImageDecoder::onInitFrameBuffer
> to even exist.  You could grab it's body and put it all herein.
> 
> if (buffer.getStatus() == ImageFrame::FrameEmpty) {
>   if (!initFrameBuffer(m_currentFrame))
>     longjmp(JMPBUF(m_reader->pngPtr()), 1);
> 
>   + what onInitFrameBuffer was doing
> }
> 
> and we could remove PNGImageDecoder::onInitFrameBuffer, right?

Yes.

> 
> Also to catch the empty frame case in frameComplete(), you can check for
> FrameEmpty as you were doing, no need to add a new variable to track that
case.

Agreed. The new variable is only used/necessary in patch set 8, which takes a
different approach to the initFrameBuffer problems.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:325:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:52, scroggo_chromium wrote:
> > On 2017/02/21 13:53:54, noel gordon wrote:
> > > Second issue here: initFrameBuffer() calls onInitFrame(), and the latter
can
> > > fail and call setFailed() as I noted earlier, right?
> > > 
> > > That means m_reader is NULL here and we crash the renderer via
> > > m_reader->pngPtr() == NULL->pngPtr().
> > > 
> > > You have to save the JUMBUF locally before calling initFrameBuffer(), and
> use
> > > that saved JMPBUF to longjmp() out if you want to avoid a crash.  Even so,
> > this
> > > would again have some perf impact: is there some way to FIXME this?
> > 
> > Some options, suggested in above responses:
> > - stop calling setFailed in initFrameBuffer (as is done in the next patch
set)
> 
> good
> 
> > - call initFrameBuffer outside of rowAvailable
> 
> see above, have rowAvailable handle all the frame init stuff in a FrameEmpty
> short-circuit.

Sounds fine to me. Done in the latest patch set.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > On 2017/02/21 13:53:55, noel gordon wrote:
> > > Third issue here: diff-left calls buffer.setHasAlpha(false);  I did not
see
> > that
> > > call anywhere through here or in the functions called.   Did I miss it, or
> do
> > we
> > > not need to call it anymore?
> > 
> > Alpha is tracked differently in the multi-frame code. The code on the left
> calls
> > setHasAlpha(false) at the beginning, and then setHasAlpha(true) if a
> non-opaque
> > pixel is seen. This means that while the frame's status is FramePartial,
> > m_hasAlpha is false, although the SkBitmap still has an alpha type that
> > recognizes that there is alpha (since the image has not been completely
> > decoded).
> > 
> > In the new code, m_hasAlpha remains true (set in zeroFillFrameRect), which
> > matches the SkBitmap. After the frame is completely decoded, m_hasAlpha may
be
> > set to false in correctAlphaWhenFrameBufferSawNoAlpha, which also takes into
> > account the frame rect and the frame that the current frame depends on. This
> > behavior matches GIF and allows us to share code.
> 
> modulo the bug in the GIF code, this paragraph describes the correct behavior.
>  
> > Either way, we end up with the correct value once the frame is FrameComplete
> > (almost - see [8]). But there is a difference, as you point out - while an
> > ImageFrame is FramePartial, its m_hasAlpha (and therefore hasAlpha())
returns
> > true, whereas the old code returned false (if a transparent pixel has not
yet
> > been seen). It is not obvious to me why the old code is desirable.
> 
> The new code returns the correct value, right? 

Could you be more specific? Once the frame is complete, both the new and the old
code return the correct value (again - except in the case of [8]).

While the frame is FramePartial, it's not entirely clear what "correct" is.
hasAlpha and setHasAlpha do not provide any documentation for what they mean,
but I could imagine two interpretations:

a) Report/set whether the ImageFrame has alpha. This is true if the portion of
the image that has been decoded has alpha OR if some rows have not been decoded
and therefore they are still transparent.
b) Report/set whether the portion of the image decoded so far has alpha. This
ignores any rows that have not yet been decoded.

The decoder implementations do not agree on the interpretation to use.

> 
>  The
> > setHasAlpha(false) after zeroFillFrameRect was introduced here [1] (EDIT:
Can
> be
> > traced further back to [7]), at which point it was the same in other
decoders
> > (e.g. JPEG). But JPEG was changed later to fix a bug [2] (it now calls
> > setHasAlpha(true); that is not actually necessary, since this already
happens
> in
> > zeroFillFrameRect). (Without digging too much into each case, note that the
> rest
> > of the decoders make different decisions about what to do during
FramePartial:
> 
> The fix in [2] is correct, and the setHasAlpha(true) is redundant but safe. 
The
> change log describes what can happen when we get the alpha wrong.  The logic
of
> the change in [2] applies to all image decoder types while
> we are decoding their frames.

Agreed. By the same logic, my change here is correct.

>  
> > BMP - set to false, like current PNG
> > GIF - leave alone, so it remains true
> > WEBP - set to true
> > .)
> > 
> > In today's code, I don't think there's a difference. m_hasAlpha appears to
> only
> > be read (besides for the purpose of copying) when the status is
FrameComplete.
> > e.g.
> 
> So the result is we do report the correct alpha state, even though the
decoders
> you listed above seem all over the place wrt to setting initial alpha during
> frame initialization.  They could be cleaned up.  Main point here is
> 
> "m_hasAlpha appears to only be read (besides for the purpose of copying) when
> the status is FrameComplete."
> 
> > - computeAlphaType [3]
> 
> nod.
> 
> > - WebGLImageConversion::ImageExtractor::extractImage (by way of hasAlpha)
[4]
> 
> nod.
> 
> They are both correct.

(Yes. My point here is that when choosing between (a) and (b) above, there is no
difference for most callers.)

>  
> > ImageDecoder::frameHasAlphaAtIndex [5] is an interesting case. It is only
> called
> > by DeferredImageDecoder [6], which calls it on m_actualDecoder.
> m_actualDecoder
> > is never used for decoding, so the default version of frameIsCompleteIndex
> > (which just checks whether the status is FrameComplete) always returns false
> (no
> > decoding means the frame will always be FrameEmpty).
> 
> m_actualDecoder is used for decoding the image header on the renderer main
> thread, but when we decide to decode the image off the main thread,
> m_actualDecoder gets deleted IIRC, and the image frame generator is asked for
> frame state. 

m_actualDecoder does not get deleted until all the data has been received:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

Until then, DeferredImageDecoder gets its information from m_actualDecoder.

> Is the alpha state is being incorrectly reported from a frame
> generator?  That seems like a DeferredImageDecoder issue if so.

For the purposes of this discussion, I'm concerned about the case where the data
is not all received. (As it happens, we do have issues around the opacity once
the image is fully received :( See crbug.com/593430.)

> 
> > So for the existing
> > PNGImageDecoder (which does not override frameIsCompleteAtIndex), no callers
> > know the difference between true or false m_hasAlpha during FramePartial.
> > (Taking that a step further, that call to hasAlpha is also on
m_actualDecoder.
> > Now that frameIsCompleteAtIndex may return true for an animated PNG,
hasAlpha
> > might be called. But again, since m_actualDecoder is not used for decoding,
> > hasAlpha will return its initial value of true.)
>  
> 
> > This does not lead me to think this is an important feature to preserve.
> 
> alpha state detection you mean?

Setting m_hasAlpha to false until a pixel is seen with alpha (i.e.
interpretation (b)). Your original question is essentially "what happened to
buffer.setHasAlpha(false)?" I've tried to explain that though the new code is
different, it changes the interpretation to (a), which preserves the interesting
bit - reporting the proper alpha state when the image is complete.

>  
> > [8] The old code actually has a bug. Since we mark the ImageFrame as *not*
> > having alpha, and then only correct it if we find alpha, we can get into a
> > situation where we've left the ImageFrame thinking it's opaque when we still
> > have the transparent pixels from the zeroFillFrameRect call.
> > https://philip.html5.org/tests/apng/058.png is a good example of this. The
> IDAT
> > is incomplete, so we only decode the first 32 out of 64 rows - the last 32
are
> > still transparent.
> > And, now that I think about it some more, I realize that I have retained
this
> > bug. libpng does not care that not all of the rows were in the IDAT
(depending
> 
> That's correct, the too few IDAT rows problem ...
> 
> > on the state of the zstream, libpng may report a warning or an error, but
> > neither happens in this case), so we "correct" m_hasAlpha to false after we
> did
> > not see any transparent pixels.
> 
> ... could report the wrong alpha, but the too few IDAT rows case is also an
> invalid APNG which we should not be decoding, right?

According to https://philip.html5.org/tests/apng/tests.html, yes. I do not see
anything in the specification (https://wiki.mozilla.org/APNG_Specification), but
it does seem reasonable to state that fewer IDAT rows than the number of rows
reported by the frame is an error, and per the spec we should not decode.

OTOH, this same exact case also happens for a static PNG image. libpng does not
report it as an error, and neither does today's Chromium code. I have not dug
into the history here - maybe this is an oversight, and maybe this is on
purpose.

(I drew attention to this in the CL description:
"""
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.
"""
although I was not quite correct - libpng does not always send a warning in the
case of too few rows, so we'd need to check ourselves, unless we modify libpng.)

This CL attempts to keep the behavior for decoding static PNG images unchanged.
Since this is a similar issue for static and animated images, I went the same
way for animated ones. But we could treat the two cases differently, or we could
go ahead and change the behavior for static PNGs, too. 

The third to last image in the test suite is particularly interesting - the fcTL
tells us that the default image does not fill the image. So we should either
treat it as a static PNG, or report a failure. In this case, the fcTL is
"correct" - the IDAT is truncated (it only has the number of rows reported by
the fcTL chunk), but libpng does not appear to mind. So if we revert to drawing
the default image, we run into the same problem.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:475:
if (index == 0 && m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/02/27 07:45:31, noel gordon wrote:
> On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > On 2017/02/21 13:53:55, noel gordon wrote:
> > > Comparisons to 0: use !index.
> > > 
> > > if (!index && m_reader->parseCompleted() && ...
> > >   return ImageDecoder::frameIsCompleteAtIndex(index);
> > > 
> > > The next sentence I read is "// For first frames of animated images," but
I
> > > think the if above can be false and so we can fall into the code below
too,
> > aka
> > > for normal PNG images. 
> > 
> > Good point. I see two ways this condition could return false for a normal
PNG
> > image:
> > - index > 0 for a normal PNG image
> >   - solution: drop the check for index, which is handled by ImageDecoder
> anyway
> > - we have not yet parsed the size
> >   - solution: check for isDecodedSizeAvailable. If false, the frame is not
> > complete.
> >     I also had to rearrange PNGImageReader::parse to ensure that once the
size
> > is
> >     available in a static PNG m_parseCompleted is set to true.
> > 
> > Are there others I have missed?
> > 
> > > Any problems with that, should I worry?
> > 
> > I generally worry about this method. Special-casing a static image matches
the
> > behavior of WEBP (mostly - IIUC, WEBP will treat a single frame but animated
> > image the same as an animated image; I don't know whether that is possible
for
> > WEBP, but APNG can have an acTL chunk and an fcTL chunk but only one frame),
> and
> > the existing code for PNG. But it is different from GIF, which treats a
single
> > frame image the same as an animated one. (Note that in APNG, unlike GIF,
there
> > is no need to look ahead to see if there are more frames, since there either
> was
> > no acTL or it indicated one frame. So in order to know where the end of the
> > frame is, we would have to parse ahead just to find it, whereas GIF learns
> this
> > info on the way to looking for more frames.)
> > 
> > If we drop the if above (or missed a case), PNG will behave no differently
for
> > some clients (assuming I initialize m_frameInfo[0].byteLength, which I do in
> the
> > latest patch set). It will always return false, the same as when it is
called
> by
> > DeferredImageDecoder [1][2], where the status will never be FrameComplete,
> > because m_actualDecoder was never used to decode.
> > 
> > 
> > [1]
> >
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
> > [2]
> >
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
> 
> I expect that the following
> 
> bool PNGImageDecoder::frameIsCompleteAtIndex(size_t index) const {
>   return ImageDecoder::frameIsCompleteAtIndex(index)
> }
> 
> would still work.  Your image will animate, etc?
> 
> Now, I have no clue why animated image types (GIF say) require this routine to
> become "frame is fully received at index", and thus all the
> firstFrameFullyReceived() code that resulted here.
>   
> 

This method was originally intended to mean "frame is fully received at index",
*not* "frame's status is FrameComplete at index". It was introduced in [1].
Although it was called by ImageSource::frameIsCompleteAtIndex, took the place of
code that checks for FrameComplete, and has a name that sure sounds like it does
that, the intent of [1] was to "know" that a frame was complete (/should be
complete) *without decoding*. In fact, hclam@ didn't even expect/plan for it to
be called for non-animated images [2], and the default implementation just
returned false. Pushback from pkasting@ convinced hclam@ to provide a default
implementation, though it was originally to return isAllDataReceived(), rather
than FrameComplete-ness [3].

So returning "frame is fully received at index" follows the original intent of
the method. This code even more closely follows the WEBP implementation [4],
which treats static and animated images differently (at your suggestion [5]).

Looking at these old CLs, both you[6] and pkasting@[7] seemed to have concerns
about frameIsCompleteAtIndex, but nothing has been done to clean it up. (I
encouraged aleksandar.stojiljkovic@ to split apart the notions of complete
versus received when he was modifying code that called frameIsCompleteAtIndex,
and he started [8] but ultimately dropped it.) 

[1] https://codereview.chromium.org/14317005
[2]
https://codereview.chromium.org/14317005/diff/3001/Source/core/platform/image...
[3] https://codereview.chromium.org/14317005/#ps23001
[4]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/image...
[5]
https://chromiumcodereview.appspot.com/13980003/diff/125001/Source/core/platf...
[6] https://chromiumcodereview.appspot.com/13980003#msg36
[7]
https://codereview.chromium.org/14317005/diff/23001/Source/core/platform/imag...
[8] https://chromiumcodereview.appspot.com/1962563002/

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode270 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: On 2017/02/27 20:31:05, scroggo_chromium wrote: > On 2017/02/27 07:45:31, ...

3 years, 9 months ago (2017-03-01 09:11:34 UTC) #21

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode321 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321: if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0)) On 2017/02/27 20:31:05, scroggo_chromium ...

3 years, 9 months ago (2017-03-01 09:19:35 UTC) #22

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/02/27 20:31:05, scroggo_chromium wrote:
> On 2017/02/27 07:45:31, noel gordon wrote:
> > On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > > On 2017/02/21 13:53:54, noel gordon wrote:
> > > > The proceeding comment is a sad story.  Is there any way round the
problem
> > it
> > > > speaks of?  Maybe FIXME: for now.
> > > 
> > > This was caught by the following tests:
> > > DeferredImageDecoderTest.decodeOnOtherThread
> > > DeferredImageDecoderTest.drawIntoSkPicture
> > > DeferredImageDecoderTest.frameOpacity
> > 
> > OK good, we have tests.
> > 
> > > which use a PNG image, which is why we did not catch this before now.
Using
> a
> > > similar GIF fails frameOpacity - when we try to correct the alpha value,
> > > originalFrameRect does not cover the full image, so we continue to assume
> the
> > > frame has alpha.
> > 
> > Just in passing, it looks to me like the GIF decoder's alpha tracking is
> > incorrect - it set's the frame status to FrameComplete _and then updates the
> > frame alpha state_.  Same bug I saw in this new PNG code.
> 
> Agreed - GIF decoder should update its alpha state before setting its status
to
> FrameComplete, both logically, and because notifyPixelsChanged will not get
> called after the pixels have changed.

Nod.

> I'm not sure that makes a difference in practice, though. notifyPixelsChanged
> does a few things:
> 
> - call onNotifyPixelsChanged, a virtual method that does nothing and currently
> has no overrides
> - set the gen ID to 0, so the next time it is requested, a new ID is created.
> The gen ID should not be requested in between the two calls, so for its
purposes
> it should not matter which order they happened in.
> - notify listeners - this seems to tell caches to boot out the old gen ID
> 
> What would be dangerous is if we called notifyPixelsChanged, then someone else
> tried to draw the bitmap (calling getGenerationID to get a newly generated
ID),
> and *then* we changed the alpha state, but I don't think that can happen.

Yes, I don't think it happens.  Less to worry though if we stick to the simple
rule: set the alpha you want and then set FrameComplete.

That happens in the this PNG code now, and in the GIF code too with your change,
all looks fine to me.

> I have fixed the bug in GIF (but it does not fix the bug in frameOpacity).

Odd.  I checked out this CL, it has the GIF fix, and merged it into ToT Chrome
on OSX, built and ran the tests, and frameOpacity passes for me.

I tried the same on Windows. After fixing a few compilation bugs, same result:
frameOpacity passes for me.

Not sure why it's not passing for you.  Anyho, I think it'd help here if you
synced your client up to ToT, merged in any changes, upload and send the result
to the try bots to iron out any compilation problems.

With that, I could patch the CL in with ease, and could re-test frameOpacity to
ensure I'm testing the same thing as you.  Possible?

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode324 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324: if (!initFrameBuffer(m_currentFrame)) On 2017/02/27 20:31:05, scroggo_chromium wrote: > On ...

3 years, 9 months ago (2017-03-01 09:26:50 UTC) #23

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:324:
if (!initFrameBuffer(m_currentFrame))
On 2017/02/27 20:31:05, scroggo_chromium wrote:
> On 2017/02/27 07:45:31, noel gordon wrote:
> > On 2017/02/23 22:09:54, scroggo_chromium wrote:
> > > On 2017/02/21 13:53:54, noel gordon wrote:
> > > > First issue: code in diff-left short circuited real work here by calling
> > > > 
> > > > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> > > >    ...
> > > > }
> > > > 
> > > > so it was done once per image frame.  Now we have
> > > > 
> > > >   if (!initFrameBuffer(m_currentFrame))
> > > > 
> > > > being called to do real work on _each row_ of an image frame.  Sadness
> w.r.t
> > > to
> > > > PNG decode perf: is there some way to FIXME this?
> > > 
> > > What is the real work you're concerned about here? The function call?
> > > (initFrameBuffer makes the same check to short circuit.) We could avoid
that
> > by
> > > modifying this line to say
> > > 
> > >   if (buffer.getStatus() == ImageFrame::FrameEmpty
> > >       && !initFrameBuffer(m_currentFrame))
> > 
> > Right.  So you changed initFrameBuffer to not call setFailed() anymore
(that's
> > good), the so the caller now has to do the error handling, so we get:
> > 
> > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> >   if (!initFrameBuffer(m_currentFrame))
> >     longjmp(JMPBUF(m_reader->pngPtr()), 1);
> > }
> > 
> > Given that, then there is little reason for
PNGImageDecoder::onInitFrameBuffer
> > to even exist.  You could grab it's body and put it all herein.
> > 
> > if (buffer.getStatus() == ImageFrame::FrameEmpty) {
> >   if (!initFrameBuffer(m_currentFrame))
> >     longjmp(JMPBUF(m_reader->pngPtr()), 1);
> > 
> >   + what onInitFrameBuffer was doing
> > }
> > 
> > and we could remove PNGImageDecoder::onInitFrameBuffer, right?
> 
> Yes.
> 
> > 
> > Also to catch the empty frame case in frameComplete(), you can check for
> > FrameEmpty as you were doing, no need to add a new variable to track that
> case.
> 
> Agreed. The new variable is only used/necessary in patch set 8, which takes a
> different approach to the initFrameBuffer problems.

Acknowledged.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:325:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/02/27 20:31:05, scroggo_chromium wrote:
> On 2017/02/27 07:45:31, noel gordon wrote:
> > On 2017/02/23 22:09:52, scroggo_chromium wrote:
> > > On 2017/02/21 13:53:54, noel gordon wrote:
> > > > Second issue here: initFrameBuffer() calls onInitFrame(), and the latter
> can
> > > > fail and call setFailed() as I noted earlier, right?
> > > > 
> > > > That means m_reader is NULL here and we crash the renderer via
> > > > m_reader->pngPtr() == NULL->pngPtr().
> > > > 
> > > > You have to save the JUMBUF locally before calling initFrameBuffer(),
and
> > use
> > > > that saved JMPBUF to longjmp() out if you want to avoid a crash.  Even
so,
> > > this
> > > > would again have some perf impact: is there some way to FIXME this?
> > > 
> > > Some options, suggested in above responses:
> > > - stop calling setFailed in initFrameBuffer (as is done in the next patch
> set)
> > 
> > good
> > 
> > > - call initFrameBuffer outside of rowAvailable
> > 
> > see above, have rowAvailable handle all the frame init stuff in a FrameEmpty
> > short-circuit.
> 
> Sounds fine to me. Done in the latest patch set.

Ok good.  nits: use size.area() and move the colorChannels local inside the if
statement ?

 if (PNG_INTERLACE_ADAM7 ==
     png_get_interlace_type(png, m_reader->infoPtr())) {
   unsigned colorChannels = m_reader->hasAlpha() ? 4 : 3;
   m_reader->createInterlaceBuffer(colorChannels * size().area());	
   if (!m_reader->interlaceBuffer()) {	
      longjmp(JMPBUF(png), 1);	
      return;	
    }	
  }

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode326 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: On 2017/02/27 20:31:05, scroggo_chromium wrote: > On 2017/02/27 07:45:31, ...

3 years, 9 months ago (2017-03-01 11:29:46 UTC) #24

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/02/27 20:31:05, scroggo_chromium wrote:
> On 2017/02/27 07:45:31, noel gordon wrote:
> > On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > > On 2017/02/21 13:53:55, noel gordon wrote:
> > > > Third issue here: diff-left calls buffer.setHasAlpha(false);  I did not
> see
> > > that
> > > > call anywhere through here or in the functions called.   Did I miss it,
or
> > do
> > > we
> > > > not need to call it anymore?
> > > 
> > > Alpha is tracked differently in the multi-frame code. The code on the left
> > calls
> > > setHasAlpha(false) at the beginning, and then setHasAlpha(true) if a
> > non-opaque
> > > pixel is seen. This means that while the frame's status is FramePartial,
> > > m_hasAlpha is false, although the SkBitmap still has an alpha type that
> > > recognizes that there is alpha (since the image has not been completely
> > > decoded).
> > > 
> > > In the new code, m_hasAlpha remains true (set in zeroFillFrameRect), which
> > > matches the SkBitmap. After the frame is completely decoded, m_hasAlpha
may
> be
> > > set to false in correctAlphaWhenFrameBufferSawNoAlpha, which also takes
into
> > > account the frame rect and the frame that the current frame depends on.
This
> > > behavior matches GIF and allows us to share code.
> > 
> > modulo the bug in the GIF code, this paragraph describes the correct
behavior.
> >  
> > > Either way, we end up with the correct value once the frame is
FrameComplete
> > > (almost - see [8]). But there is a difference, as you point out - while an
> > > ImageFrame is FramePartial, its m_hasAlpha (and therefore hasAlpha())
> returns
> > > true, whereas the old code returned false (if a transparent pixel has not
> yet
> > > been seen). It is not obvious to me why the old code is desirable.
> > 
> > The new code returns the correct value, right? 
> 
> Could you be more specific? Once the frame is complete, both the new and the
old
> code return the correct value (again - except in the case of [8]).

Yeap, sorry, I was replying to this part:

"But there is a difference, as you point out - while an ImageFrame is
FramePartial, its m_hasAlpha (and therefore hasAlpha()) returns true, whereas
the old code returned false (if a transparent pixel has
not yet been seen).  It is not obvious to me why the old code is desirable."

The old PNG was single frame, and to find out about frame alpha since there is
no direct public access to frame.hasAlpha() for external users, they have to
call frameHasAlphaAtIndex().  That returns true until frameIsCompleteAtIndex(),
and thereafter returns frame[0].hasAlpha().  Seems correct to me.

In the new APNG code, how does frameHasAlphaAtIndex() work?  Since
frameIsCompleteAtIndex() is been overridden and now returns "the frame fully
received" if the index > 0 (moulo any bugs in it), and that means
frame[index].hasAlpha() could be returned.  The frame's state is either
<= FramePartial or FrameComplete.  The return is true now while not
FrameComplete, and the actual frame alpha for FrameComplete.  Seems correct to
me.

To answer my own question ...

> > The new code returns the correct value, right? 

... seems like the answer is yes.

> While the frame is FramePartial, it's not entirely clear what "correct" is.
> hasAlpha and setHasAlpha do not provide any documentation for what they mean,
> but I could imagine two interpretations:
> 
> a) Report/set whether the ImageFrame has alpha. This is true if the portion of
> the image that has been decoded has alpha OR if some rows have not been
decoded
> and therefore they are still transparent.
> b) Report/set whether the portion of the image decoded so far has alpha. This
> ignores any rows that have not yet been decoded.
> 
> The decoder implementations do not agree on the interpretation to use.
> 
> > 
> >  The
> > > setHasAlpha(false) after zeroFillFrameRect was introduced here [1] (EDIT:
> Can
> > be
> > > traced further back to [7]), at which point it was the same in other
> decoders
> > > (e.g. JPEG). But JPEG was changed later to fix a bug [2] (it now calls
> > > setHasAlpha(true); that is not actually necessary, since this already
> happens
> > in
> > > zeroFillFrameRect). (Without digging too much into each case, note that
the
> > rest
> > > of the decoders make different decisions about what to do during
> FramePartial:
> > 
> > The fix in [2] is correct, and the setHasAlpha(true) is redundant but safe. 
> The
> > change log describes what can happen when we get the alpha wrong.  The logic
> of
> > the change in [2] applies to all image decoder types while
> > we are decoding their frames.
> 
> Agreed. By the same logic, my change here is correct.

Indeed correct, and by the same logic, frames with state <= framePartial have
alpha, period.  The correct answer to your question about possible
interpretations is a).  The code mentions this in a few places, ImageFrame.cpp
mentions it using JPEG as the example.


> > > BMP - set to false, like current PNG
> > > GIF - leave alone, so it remains true
> > > WEBP - set to true
> > > .)
> > > 
> > > In today's code, I don't think there's a difference. m_hasAlpha appears to
> > only
> > > be read (besides for the purpose of copying) when the status is
> FrameComplete.
> > > e.g.
> > 
> > So the result is we do report the correct alpha state, even though the
> decoders
> > you listed above seem all over the place wrt to setting initial alpha during
> > frame initialization.  They could be cleaned up.  Main point here is
> > 
> > "m_hasAlpha appears to only be read (besides for the purpose of copying)
when
> > the status is FrameComplete."

Yes, this is a HTML spec requirement.  The initial (index == 0) frame of a
static image, or animated image, can be drawn by canvas elements (canvas 2d and
WebGL).  They check FrameComplete status before they attempt to draw, and fail
the draw if FrameComplete is not true.

This means that an overridden frameIsCompleteAtIndex(index) in the new PNG
decoder needs to do the right thing for index == 0, and return the decode
complete status, not whether the frame is fully received.

I'm not sure the frameIsCompleteAtIndex() code here is doing the right thing
yet.  A layout test (draw an animated PNG to <canvas>) might help sort that out.

> > > - computeAlphaType [3]
> > 
> > nod.
> > 
> > > - WebGLImageConversion::ImageExtractor::extractImage (by way of hasAlpha)
> [4]
> > 
> > nod.
> > 
> > They are both correct.
> 
> (Yes. My point here is that when choosing between (a) and (b) above, there is
no
> difference for most callers.)
> 
> >  
> > > ImageDecoder::frameHasAlphaAtIndex [5] is an interesting case. It is only
> > called
> > > by DeferredImageDecoder [6], which calls it on m_actualDecoder.
> > m_actualDecoder
> > > is never used for decoding, so the default version of frameIsCompleteIndex
> > > (which just checks whether the status is FrameComplete) always returns
false
> > (no
> > > decoding means the frame will always be FrameEmpty).
> > 
> > m_actualDecoder is used for decoding the image header on the renderer main
> > thread, but when we decide to decode the image off the main thread,
> > m_actualDecoder gets deleted IIRC, and the image frame generator is asked
for
> > frame state. 
> 
> m_actualDecoder does not get deleted until all the data has been received:
> 
>https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graphics/DeferredImageDecoder.cpp?l=310

Ah yes, I forgotten that m_actualDecoder needs to hang around longer.

> Until then, DeferredImageDecoder gets its information from m_actualDecoder.
> 
> > Is the alpha state is being incorrectly reported from a frame
> > generator?  That seems like a DeferredImageDecoder issue if so.
> 
> For the purposes of this discussion, I'm concerned about the case where the
data
> is not all received. (As it happens, we do have issues around the opacity once
> the image is fully received :( See crbug.com/593430.)

(yes, the deferred chicken+egg bug crbug.com/593430).
 
> > > So for the existing
> > > PNGImageDecoder (which does not override frameIsCompleteAtIndex), no
callers
> > > know the difference between true or false m_hasAlpha during FramePartial.
> > > (Taking that a step further, that call to hasAlpha is also on
> m_actualDecoder.
> > > Now that frameIsCompleteAtIndex may return true for an animated PNG,
> hasAlpha
> > > might be called. But again, since m_actualDecoder is not used for
decoding,
> > > hasAlpha will return its initial value of true.)

Nothing to worry about here.  BitmapImage explains that in it public API
currentFrameKnownToBeOpaque() http://bit.ly/2lbsEBT

"frameHasAlphaAtIndex() conservatively returns false" is a roundabout way of
saying that m_actualDecoder will in the normal case, always return true to this
request.  Make sense to me since the callers of currentFrameKnownToBeOpaque()
are all on the main thread.

So no issues with m_actualDecoder being used, which was your concern above,
since that is expected behavior for a BitmapImage in the age of deferred
(non-main-thread) image decoding.

> > > This does not lead me to think this is an important feature to preserve.
> > 
> > alpha state detection you mean?
> 
> Setting m_hasAlpha to false until a pixel is seen with alpha (i.e.
> interpretation (b)).

Right, got it.

> Your original question is essentially "what happened to
> buffer.setHasAlpha(false)?" I've tried to explain that though the new code is
> different, it changes the interpretation to (a), which preserves the
interesting
> bit - reporting the proper alpha state when the image is complete.

Sounds correct to me: no need for the buffer.setHasAlpha(false) anymore.

> > > [8] The old code actually has a bug. Since we mark the ImageFrame as *not*
> > > having alpha, and then only correct it if we find alpha, we can get into a
> > > situation where we've left the ImageFrame thinking it's opaque when we
still
> > > have the transparent pixels from the zeroFillFrameRect call.
> > > https://philip.html5.org/tests/apng/058.png is a good example of this. The
> > IDAT
> > > is incomplete, so we only decode the first 32 out of 64 rows - the last 32
> are
> > > still transparent.
> > > And, now that I think about it some more, I realize that I have retained
> this
> > > bug. libpng does not care that not all of the rows were in the IDAT
> (depending
> > 
> > That's correct, the too few IDAT rows problem ...
> > 
> > > on the state of the zstream, libpng may report a warning or an error, but
> > > neither happens in this case), so we "correct" m_hasAlpha to false after
we
> > did
> > > not see any transparent pixels.
> > 
> > ... could report the wrong alpha, but the too few IDAT rows case is also an
> > invalid APNG which we should not be decoding, right?
> 
> According to https://philip.html5.org/tests/apng/tests.html, yes. I do not see
> anything in the specification (https://wiki.mozilla.org/APNG_Specification),
but
> it does seem reasonable to state that fewer IDAT rows than the number of rows
> reported by the frame is an error, and per the spec we should not decode.
>
> OTOH, this same exact case also happens for a static PNG image. libpng does
not
> report it as an error, and neither does today's Chromium code. I have not dug
> into the history here - maybe this is an oversight, and maybe this is on
> purpose.
>
> (I drew attention to this in the CL description:
> """
> - The final three images draw incorrectly. They have incorrectly sized IDATs/
> fdATs. These could be respected by checking the warning that libpng sends,
> although we currently ignore this for static PNGs anyway.
> """
> although I was not quite correct - libpng does not always send a warning in
the
> case of too few rows, so we'd need to check ourselves, unless we modify
libpng.)
>
> This CL attempts to keep the behavior for decoding static PNG images
unchanged.
> Since this is a similar issue for static and animated images, I went the same
> way for animated ones. But we could treat the two cases differently, or we
could
> go ahead and change the behavior for static PNGs, too. 
> 
> The third to last image in the test suite is particularly interesting - the
fcTL
> tells us that the default image does not fill the image. So we should either
> treat it as a static PNG, or report a failure. In this case, the fcTL is
> "correct" - the IDAT is truncated (it only has the number of rows reported by
> the fcTL chunk), but libpng does not appear to mind. So if we revert to
drawing
> the default image, we run into the same problem.

Yeah these cases seems difficult: might be a way to fix them, libpng might have
a #define to enable more reporting, but maybe not worth chasing if the result is
we change behavior for static PNG.

scroggo_chromium

PTAL. Rebased to ToT https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode270 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: On 2017/03/01 09:11:34, noel gordon ...

3 years, 9 months ago (2017-03-03 21:56:40 UTC) #25

PTAL. Rebased to ToT

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:270: 
On 2017/03/01 09:11:34, noel gordon wrote:
> On 2017/02/27 20:31:05, scroggo_chromium wrote:
> > On 2017/02/27 07:45:31, noel gordon wrote:
> > > On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > > > On 2017/02/21 13:53:55, noel gordon wrote:
> > > > > 
> > > > > Add a DCHECK(isDecodedSizeAvailable()) around here?
> > > > 
> > > > Done.
> > > 
> > > One second thoughts, this is in the middle of some the color handling
code. 
> I
> > > figure there is more work to do in that code, so maybe move this DCHECK to
> > after
> > > all the color-specific code,
> > 
> > As I look for a better place to put it, I think this is the best, even
though
> > it's in between color-specific code. The block above checks for the decoded
> > size, so it looks like this:
> > 
> >   if (!isDecodedSizeAvailable()) {
> >     // Do some work, including ensuring that the decoded size
> >     // is available. If not, signal an error and do not continue.
> >   }
> > 
> >   DCHECK(isDecodedSizeAvailable());
> > 
> > I guess either way we're splitting up something. 
> 
> Yeah, seem so.
> 
> > Maybe this belongs just above png_read_update_info?
> 
> Yeah, anywhere after the color-code. Try a few places and see how you feel
about
> it, ditch it if it's PITA.
>  
> > > sorry.
> > 
> > No worries. Now is the time to address it!
> 
> Thx.

Done.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/03/01 09:19:35, noel gordon wrote:
> On 2017/02/27 20:31:05, scroggo_chromium wrote:
> > On 2017/02/27 07:45:31, noel gordon wrote:
> > > On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > > > On 2017/02/21 13:53:54, noel gordon wrote:
> > > > > The proceeding comment is a sad story.  Is there any way round the
> problem
> > > it
> > > > > speaks of?  Maybe FIXME: for now.
> > > > 
> > > > This was caught by the following tests:
> > > > DeferredImageDecoderTest.decodeOnOtherThread
> > > > DeferredImageDecoderTest.drawIntoSkPicture
> > > > DeferredImageDecoderTest.frameOpacity
> > > 
> > > OK good, we have tests.
> > > 
> > > > which use a PNG image, which is why we did not catch this before now.
> Using
> > a
> > > > similar GIF fails frameOpacity - when we try to correct the alpha value,
> > > > originalFrameRect does not cover the full image, so we continue to
assume
> > the
> > > > frame has alpha.
> > > 
> > > Just in passing, it looks to me like the GIF decoder's alpha tracking is
> > > incorrect - it set's the frame status to FrameComplete _and then updates
the
> > > frame alpha state_.  Same bug I saw in this new PNG code.
> > 
> > Agreed - GIF decoder should update its alpha state before setting its status
> to
> > FrameComplete, both logically, and because notifyPixelsChanged will not get
> > called after the pixels have changed.
> 
> Nod.
> 
> > I'm not sure that makes a difference in practice, though.
notifyPixelsChanged
> > does a few things:
> > 
> > - call onNotifyPixelsChanged, a virtual method that does nothing and
currently
> > has no overrides
> > - set the gen ID to 0, so the next time it is requested, a new ID is
created.
> > The gen ID should not be requested in between the two calls, so for its
> purposes
> > it should not matter which order they happened in.
> > - notify listeners - this seems to tell caches to boot out the old gen ID
> > 
> > What would be dangerous is if we called notifyPixelsChanged, then someone
else
> > tried to draw the bitmap (calling getGenerationID to get a newly generated
> ID),
> > and *then* we changed the alpha state, but I don't think that can happen.
> 
> Yes, I don't think it happens.  Less to worry though if we stick to the simple
> rule: set the alpha you want and then set FrameComplete.
> 
> That happens in the this PNG code now, and in the GIF code too with your
change,
> all looks fine to me.
> 
> > I have fixed the bug in GIF (but it does not fix the bug in frameOpacity).
> 
> Odd.  I checked out this CL, it has the GIF fix, and merged it into ToT Chrome
> on OSX, built and ran the tests, and frameOpacity passes for me.
> 
> I tried the same on Windows. After fixing a few compilation bugs, same result:
> frameOpacity passes for me.
> 
> Not sure why it's not passing for you. 

A separate change fixes frameOpacity, which is also in this CL, so it does pass
for me. Without the changes to ImageFrameGenerator/ImageDecoder around
setMemoryAllocator, my updated frameOpacity test fails.

> Anyho, I think it'd help here if you
> synced your client up to ToT, merged in any changes, upload and send the
result
> to the try bots to iron out any compilation problems.
> 
> With that, I could patch the CL in with ease, and could re-test frameOpacity
to
> ensure I'm testing the same thing as you.  Possible?

I can sync up to ToT. FWIW, I like to do that as infrequently as possible during
a review, since it can create extra work (rebasing, recompiling etc) that could
otherwise be done only once. It can also make it hard to see what I changed, but
I've made the rebase its own patch set.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:325:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/03/01 09:26:49, noel gordon wrote:
> On 2017/02/27 20:31:05, scroggo_chromium wrote:
> > On 2017/02/27 07:45:31, noel gordon wrote:
> > > On 2017/02/23 22:09:52, scroggo_chromium wrote:
> > > > On 2017/02/21 13:53:54, noel gordon wrote:
> > > > > Second issue here: initFrameBuffer() calls onInitFrame(), and the
latter
> > can
> > > > > fail and call setFailed() as I noted earlier, right?
> > > > > 
> > > > > That means m_reader is NULL here and we crash the renderer via
> > > > > m_reader->pngPtr() == NULL->pngPtr().
> > > > > 
> > > > > You have to save the JUMBUF locally before calling initFrameBuffer(),
> and
> > > use
> > > > > that saved JMPBUF to longjmp() out if you want to avoid a crash.  Even
> so,
> > > > this
> > > > > would again have some perf impact: is there some way to FIXME this?
> > > > 
> > > > Some options, suggested in above responses:
> > > > - stop calling setFailed in initFrameBuffer (as is done in the next
patch
> > set)
> > > 
> > > good
> > > 
> > > > - call initFrameBuffer outside of rowAvailable
> > > 
> > > see above, have rowAvailable handle all the frame init stuff in a
FrameEmpty
> > > short-circuit.
> > 
> > Sounds fine to me. Done in the latest patch set.
> 
> Ok good.  nits: use size.area() and move the colorChannels local inside the if
> statement ?
> 
>  if (PNG_INTERLACE_ADAM7 ==
>      png_get_interlace_type(png, m_reader->infoPtr())) {
>    unsigned colorChannels = m_reader->hasAlpha() ? 4 : 3;
>    m_reader->createInterlaceBuffer(colorChannels * size().area());	
>    if (!m_reader->interlaceBuffer()) {	
>       longjmp(JMPBUF(png), 1);	
>       return;	
>     }	
>   }

I reverted back to the original code, since after removing onInitFrameBuffer I
wasn't really changing this piece.

Done in next patch set.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/03/01 11:29:46, noel gordon wrote:
> On 2017/02/27 20:31:05, scroggo_chromium wrote:
> > On 2017/02/27 07:45:31, noel gordon wrote:
> > > On 2017/02/23 22:09:53, scroggo_chromium wrote:
> > > > On 2017/02/21 13:53:55, noel gordon wrote:
> > > > > Third issue here: diff-left calls buffer.setHasAlpha(false);  I did
not
> > see
> > > > that
> > > > > call anywhere through here or in the functions called.   Did I miss
it,
> or
> > > do
> > > > we
> > > > > not need to call it anymore?
> > > > 
> > > > Alpha is tracked differently in the multi-frame code. The code on the
left
> > > calls
> > > > setHasAlpha(false) at the beginning, and then setHasAlpha(true) if a
> > > non-opaque
> > > > pixel is seen. This means that while the frame's status is FramePartial,
> > > > m_hasAlpha is false, although the SkBitmap still has an alpha type that
> > > > recognizes that there is alpha (since the image has not been completely
> > > > decoded).
> > > > 
> > > > In the new code, m_hasAlpha remains true (set in zeroFillFrameRect),
which
> > > > matches the SkBitmap. After the frame is completely decoded, m_hasAlpha
> may
> > be
> > > > set to false in correctAlphaWhenFrameBufferSawNoAlpha, which also takes
> into
> > > > account the frame rect and the frame that the current frame depends on.
> This
> > > > behavior matches GIF and allows us to share code.
> > > 
> > > modulo the bug in the GIF code, this paragraph describes the correct
> behavior.
> > >  
> > > > Either way, we end up with the correct value once the frame is
> FrameComplete
> > > > (almost - see [8]). But there is a difference, as you point out - while
an
> > > > ImageFrame is FramePartial, its m_hasAlpha (and therefore hasAlpha())
> > returns
> > > > true, whereas the old code returned false (if a transparent pixel has
not
> > yet
> > > > been seen). It is not obvious to me why the old code is desirable.
> > > 
> > > The new code returns the correct value, right? 
> > 
> > Could you be more specific? Once the frame is complete, both the new and the
> old
> > code return the correct value (again - except in the case of [8]).
> 
> Yeap, sorry, I was replying to this part:
> 
> "But there is a difference, as you point out - while an ImageFrame is
> FramePartial, its m_hasAlpha (and therefore hasAlpha()) returns true, whereas
> the old code returned false (if a transparent pixel has
> not yet been seen).  It is not obvious to me why the old code is desirable."
> 
> The old PNG was single frame, and to find out about frame alpha since there is
> no direct public access to frame.hasAlpha() for external users, they have to
> call frameHasAlphaAtIndex().  That returns true until
frameIsCompleteAtIndex(),
> and thereafter returns frame[0].hasAlpha().  Seems correct to me.
> 
> In the new APNG code, how does frameHasAlphaAtIndex() work?  Since
> frameIsCompleteAtIndex() is been overridden and now returns "the frame fully
> received" if the index > 0 (moulo any bugs in it), and that means
> frame[index].hasAlpha() could be returned.  The frame's state is either
> <= FramePartial or FrameComplete.  The return is true now while not
> FrameComplete, and the actual frame alpha for FrameComplete.  Seems correct to
> me.
> 
> To answer my own question ...
> 
> > > The new code returns the correct value, right? 
> 
> ... seems like the answer is yes.
> 
> > While the frame is FramePartial, it's not entirely clear what "correct" is.
> > hasAlpha and setHasAlpha do not provide any documentation for what they
mean,
> > but I could imagine two interpretations:
> > 
> > a) Report/set whether the ImageFrame has alpha. This is true if the portion
of
> > the image that has been decoded has alpha OR if some rows have not been
> decoded
> > and therefore they are still transparent.
> > b) Report/set whether the portion of the image decoded so far has alpha.
This
> > ignores any rows that have not yet been decoded.
> > 
> > The decoder implementations do not agree on the interpretation to use.
> > 
> > > 
> > >  The
> > > > setHasAlpha(false) after zeroFillFrameRect was introduced here [1]
(EDIT:
> > Can
> > > be
> > > > traced further back to [7]), at which point it was the same in other
> > decoders
> > > > (e.g. JPEG). But JPEG was changed later to fix a bug [2] (it now calls
> > > > setHasAlpha(true); that is not actually necessary, since this already
> > happens
> > > in
> > > > zeroFillFrameRect). (Without digging too much into each case, note that
> the
> > > rest
> > > > of the decoders make different decisions about what to do during
> > FramePartial:
> > > 
> > > The fix in [2] is correct, and the setHasAlpha(true) is redundant but
safe. 
> > The
> > > change log describes what can happen when we get the alpha wrong.  The
logic
> > of
> > > the change in [2] applies to all image decoder types while
> > > we are decoding their frames.
> > 
> > Agreed. By the same logic, my change here is correct.
> 
> Indeed correct, and by the same logic, frames with state <= framePartial have
> alpha, period.  The correct answer to your question about possible
> interpretations is a).  The code mentions this in a few places, ImageFrame.cpp
> mentions it using JPEG as the example.
> 
> 
> > > > BMP - set to false, like current PNG
> > > > GIF - leave alone, so it remains true
> > > > WEBP - set to true
> > > > .)
> > > > 
> > > > In today's code, I don't think there's a difference. m_hasAlpha appears
to
> > > only
> > > > be read (besides for the purpose of copying) when the status is
> > FrameComplete.
> > > > e.g.
> > > 
> > > So the result is we do report the correct alpha state, even though the
> > decoders
> > > you listed above seem all over the place wrt to setting initial alpha
during
> > > frame initialization.  They could be cleaned up.  Main point here is
> > > 
> > > "m_hasAlpha appears to only be read (besides for the purpose of copying)
> when
> > > the status is FrameComplete."
> 
> Yes, this is a HTML spec requirement.  The initial (index == 0) frame of a
> static image, or animated image, can be drawn by canvas elements (canvas 2d
and
> WebGL).  They check FrameComplete status before they attempt to draw, and fail
> the draw if FrameComplete is not true.
> 
> This means that an overridden frameIsCompleteAtIndex(index) in the new PNG
> decoder needs to do the right thing for index == 0, and return the decode
> complete status, not whether the frame is fully received.
> 
> I'm not sure the frameIsCompleteAtIndex() code here is doing the right thing
> yet.  A layout test (draw an animated PNG to <canvas>) might help sort that
out.

Happy to provide a layout test. Is there a particular concern you have with the
code? And what do you have in mind? Something like:
- provide enough data for part of the second frame, verify that we still draw
the first frame?

> 
> > > > - computeAlphaType [3]
> > > 
> > > nod.
> > > 
> > > > - WebGLImageConversion::ImageExtractor::extractImage (by way of
hasAlpha)
> > [4]
> > > 
> > > nod.
> > > 
> > > They are both correct.
> > 
> > (Yes. My point here is that when choosing between (a) and (b) above, there
is
> no
> > difference for most callers.)
> > 
> > >  
> > > > ImageDecoder::frameHasAlphaAtIndex [5] is an interesting case. It is
only
> > > called
> > > > by DeferredImageDecoder [6], which calls it on m_actualDecoder.
> > > m_actualDecoder
> > > > is never used for decoding, so the default version of
frameIsCompleteIndex
> > > > (which just checks whether the status is FrameComplete) always returns
> false
> > > (no
> > > > decoding means the frame will always be FrameEmpty).
> > > 
> > > m_actualDecoder is used for decoding the image header on the renderer main
> > > thread, but when we decide to decode the image off the main thread,
> > > m_actualDecoder gets deleted IIRC, and the image frame generator is asked
> for
> > > frame state. 
> > 
> > m_actualDecoder does not get deleted until all the data has been received:
> > 
>
>https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graphics/DeferredImageDecoder.cpp?l=310
> 
> Ah yes, I forgotten that m_actualDecoder needs to hang around longer.
> 
> > Until then, DeferredImageDecoder gets its information from m_actualDecoder.
> > 
> > > Is the alpha state is being incorrectly reported from a frame
> > > generator?  That seems like a DeferredImageDecoder issue if so.
> > 
> > For the purposes of this discussion, I'm concerned about the case where the
> data
> > is not all received. (As it happens, we do have issues around the opacity
once
> > the image is fully received :( See crbug.com/593430.)
> 
> (yes, the deferred chicken+egg bug crbug.com/593430).
>  
> > > > So for the existing
> > > > PNGImageDecoder (which does not override frameIsCompleteAtIndex), no
> callers
> > > > know the difference between true or false m_hasAlpha during
FramePartial.
> > > > (Taking that a step further, that call to hasAlpha is also on
> > m_actualDecoder.
> > > > Now that frameIsCompleteAtIndex may return true for an animated PNG,
> > hasAlpha
> > > > might be called. But again, since m_actualDecoder is not used for
> decoding,
> > > > hasAlpha will return its initial value of true.)
> 
> Nothing to worry about here.  BitmapImage explains that in it public API
> currentFrameKnownToBeOpaque() http://bit.ly/2lbsEBT
> 
> "frameHasAlphaAtIndex() conservatively returns false" is a roundabout way of
> saying that m_actualDecoder will in the normal case, always return true to
this
> request.  Make sense to me since the callers of currentFrameKnownToBeOpaque()
> are all on the main thread.
> 
> So no issues with m_actualDecoder being used, which was your concern above,
> since that is expected behavior for a BitmapImage in the age of deferred
> (non-main-thread) image decoding.
> 
> > > > This does not lead me to think this is an important feature to preserve.
> > > 
> > > alpha state detection you mean?
> > 
> > Setting m_hasAlpha to false until a pixel is seen with alpha (i.e.
> > interpretation (b)).
> 
> Right, got it.
> 
> > Your original question is essentially "what happened to
> > buffer.setHasAlpha(false)?" I've tried to explain that though the new code
is
> > different, it changes the interpretation to (a), which preserves the
> interesting
> > bit - reporting the proper alpha state when the image is complete.
> 
> Sounds correct to me: no need for the buffer.setHasAlpha(false) anymore.
> 
> > > > [8] The old code actually has a bug. Since we mark the ImageFrame as
*not*
> > > > having alpha, and then only correct it if we find alpha, we can get into
a
> > > > situation where we've left the ImageFrame thinking it's opaque when we
> still
> > > > have the transparent pixels from the zeroFillFrameRect call.
> > > > https://philip.html5.org/tests/apng/058.png is a good example of this.
The
> > > IDAT
> > > > is incomplete, so we only decode the first 32 out of 64 rows - the last
32
> > are
> > > > still transparent.
> > > > And, now that I think about it some more, I realize that I have retained
> > this
> > > > bug. libpng does not care that not all of the rows were in the IDAT
> > (depending
> > > 
> > > That's correct, the too few IDAT rows problem ...
> > > 
> > > > on the state of the zstream, libpng may report a warning or an error,
but
> > > > neither happens in this case), so we "correct" m_hasAlpha to false after
> we
> > > did
> > > > not see any transparent pixels.
> > > 
> > > ... could report the wrong alpha, but the too few IDAT rows case is also
an
> > > invalid APNG which we should not be decoding, right?
> > 
> > According to https://philip.html5.org/tests/apng/tests.html, yes. I do not
see
> > anything in the specification (https://wiki.mozilla.org/APNG_Specification),
> but
> > it does seem reasonable to state that fewer IDAT rows than the number of
rows
> > reported by the frame is an error, and per the spec we should not decode.
> >
> > OTOH, this same exact case also happens for a static PNG image. libpng does
> not
> > report it as an error, and neither does today's Chromium code. I have not
dug
> > into the history here - maybe this is an oversight, and maybe this is on
> > purpose.
> >
> > (I drew attention to this in the CL description:
> > """
> > - The final three images draw incorrectly. They have incorrectly sized
IDATs/
> > fdATs. These could be respected by checking the warning that libpng sends,
> > although we currently ignore this for static PNGs anyway.
> > """
> > although I was not quite correct - libpng does not always send a warning in
> the
> > case of too few rows, so we'd need to check ourselves, unless we modify
> libpng.)
> >
> > This CL attempts to keep the behavior for decoding static PNG images
> unchanged.
> > Since this is a similar issue for static and animated images, I went the
same
> > way for animated ones. But we could treat the two cases differently, or we
> could
> > go ahead and change the behavior for static PNGs, too. 
> > 
> > The third to last image in the test suite is particularly interesting - the
> fcTL
> > tells us that the default image does not fill the image. So we should either
> > treat it as a static PNG, or report a failure. In this case, the fcTL is
> > "correct" - the IDAT is truncated (it only has the number of rows reported
by
> > the fcTL chunk), but libpng does not appear to mind. So if we revert to
> drawing
> > the default image, we run into the same problem.
> 
> Yeah these cases seems difficult: might be a way to fix them, libpng might
have
> a #define to enable more reporting, but maybe not worth chasing if the result
is
> we change behavior for static PNG.

Let's take them individually:
a) Too many rows:
We just need to catch the warnings here [1][2] and consider it a failure. We
know whether we're dealing with a static PNG or an animated one, so we could
behave differently if we want to.
[1]
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
[2]
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...

b) Too few rows:
This is trickier, but not terribly hard. In some cases, libpng will report a
warning [3] or an error [4] if there is not enough IDAT data, but note that in
the latter case, it will silently move on to the next chunk if the
PNG_FLAG_ZSTREAM_ENDED flag is set. (I have not found any #defines that made a
difference here.) In order to handle this properly, we would need to keep track
of the rows received, and if we do not get enough, signal an error. Again, we
could handle it differently depending on whether it's a static PNG or not. But
the code would likely be ugly, especially for interlaced images.

[3]
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
[4]
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...

scroggo_chromium

The CQ bit was checked by scroggo@chromium.org to run a CQ dry run

3 years, 9 months ago (2017-03-03 21:56:53 UTC) #26

commit-bot: I haz the power

Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2618633004/220001

3 years, 9 months ago (2017-03-03 21:57:06 UTC) #27

commit-bot: I haz the power

The CQ bit was unchecked by commit-bot@chromium.org

3 years, 9 months ago (2017-03-03 22:30:25 UTC) #28

commit-bot: I haz the power

Dry run: Try jobs failed on following builders: win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/394182)

3 years, 9 months ago (2017-03-03 22:30:26 UTC) #29

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode321 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321: if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0)) On 2017/03/03 21:56:40, scroggo_chromium ...

3 years, 9 months ago (2017-03-06 03:06:46 UTC) #30

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:321:
if (m_frameBufferCache[0].originalFrameRect().size() == IntSize(0, 0))
On 2017/03/03 21:56:40, scroggo_chromium wrote:

> > I tried the same on Windows. After fixing a few compilation bugs, same
result:
> > frameOpacity passes for me.
> > 
> > Not sure why it's not passing for you. 
> 
> A separate change fixes frameOpacity, which is also in this CL, so it does
pass
> for me. Without the changes to ImageFrameGenerator/ImageDecoder around
> setMemoryAllocator, my updated frameOpacity test fails.

OK, the "separate change" change can be landed here since you have included it,
or as a separate change, whichever you prefer.

> > Anyho, I think it'd help here if you
> > synced your client up to ToT, merged in any changes, upload and send the
> result
> > to the try bots to iron out any compilation problems.
> > 
> > With that, I could patch the CL in with ease, and could re-test frameOpacity
> to
> > ensure I'm testing the same thing as you.  Possible?
> 
> I can sync up to ToT. FWIW, I like to do that as infrequently as possible
during
> a review, since it can create extra work (rebasing, recompiling etc) that
could
> otherwise be done only once. It can also make it hard to see what I changed,
but
> I've made the rebase its own patch set.

OK thanks, the bots do have various compilation errors for that rebase patch set
so there's still work to do there.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/03/03 21:56:40, scroggo_chromium wrote:
> On 2017/03/01 11:29:46, noel gordon wrote:

> > Yes, this is a HTML spec requirement.  The initial (index == 0) frame of a
> > static image, or animated image, can be drawn by canvas elements (canvas 2d
> and
> > WebGL).  They check FrameComplete status before they attempt to draw, and
fail
> > the draw if FrameComplete is not true.
> > 
> > This means that an overridden frameIsCompleteAtIndex(index) in the new PNG
> > decoder needs to do the right thing for index == 0, and return the decode
> > complete status, not whether the frame is fully received.
> > 
> > I'm not sure the frameIsCompleteAtIndex() code here is doing the right thing
> > yet.  A layout test (draw an animated PNG to <canvas>) might help sort that
> out.
> 
> Happy to provide a layout test. Is there a particular concern you have with
the
> code? And what do you have in mind? Something like:
> - provide enough data for part of the second frame, verify that we still draw
> the first frame?

Yes, and I filed crbug.com/698487 about it, PTAL.

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
> noel gordon wrote:
> > Yeah these cases seems difficult: might be a way to fix them, libpng might
> have
> > a #define to enable more reporting, but maybe not worth chasing if the
result
> is
> > we change behavior for static PNG.
> 

scroggo_chromium wrote:

> Let's take them individually:
> a) Too many rows:
> We just need to catch the warnings here [1][2] and consider it a failure. We
> know whether we're dealing with a static PNG or an animated one, so we could
> behave differently if we want to.
> [1]
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> [2]
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> 
> b) Too few rows:
> This is trickier, but not terribly hard. In some cases, libpng will report a
> warning [3] or an error [4] if there is not enough IDAT data, but note that in
> the latter case, it will silently move on to the next chunk if the
> PNG_FLAG_ZSTREAM_ENDED flag is set. (I have not found any #defines that made a
> difference here.) In order to handle this properly, we would need to keep
track
> of the rows received, and if we do not get enough, signal an error. Again, we
> could handle it differently depending on whether it's a static PNG or not. But
> the code would likely be ugly, especially for interlaced images.
> 
> [3]
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> [4]
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...

Suggest you push these cases off into another bug, as I expect that that cases
will take some time and fiddling to get right.

Noel Gordon

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode143 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:143: inline float pngFixedToFloat(png_fixed_point x) { Merge funk: remove this ...

3 years, 9 months ago (2017-03-06 05:21:48 UTC) #31

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode326 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: On 2017/03/06 03:06:45, noel gordon wrote: > > noel ...

3 years, 9 months ago (2017-03-07 20:25:38 UTC) #32

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/03/06 03:06:45, noel gordon wrote:
> > noel gordon wrote:
> > > Yeah these cases seems difficult: might be a way to fix them, libpng might
> > have
> > > a #define to enable more reporting, but maybe not worth chasing if the
> result
> > is
> > > we change behavior for static PNG.
> > 
> 
> scroggo_chromium wrote:
> 
> > Let's take them individually:
> > a) Too many rows:
> > We just need to catch the warnings here [1][2] and consider it a failure. We
> > know whether we're dealing with a static PNG or an animated one, so we could
> > behave differently if we want to.
> > [1]
> >
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> > [2]
> >
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> > 
> > b) Too few rows:
> > This is trickier, but not terribly hard. In some cases, libpng will report a
> > warning [3] or an error [4] if there is not enough IDAT data, but note that
in
> > the latter case, it will silently move on to the next chunk if the
> > PNG_FLAG_ZSTREAM_ENDED flag is set. (I have not found any #defines that made
a
> > difference here.) In order to handle this properly, we would need to keep
> track
> > of the rows received, and if we do not get enough, signal an error. Again,
we
> > could handle it differently depending on whether it's a static PNG or not.
But
> > the code would likely be ugly, especially for interlaced images.
> > 
> > [3]
> >
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> > [4]
> >
>
https://cs.chromium.org/chromium/src/third_party/libpng/pngpread.c?q=pngpread...
> 
> Suggest you push these cases off into another bug, as I expect that that cases
> will take some time and fiddling to get right.

Filed https://bugs.chromium.org/p/chromium/issues/detail?id=698808

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:143:
inline float pngFixedToFloat(png_fixed_point x) {
On 2017/03/06 05:21:47, noel gordon wrote:
> Merge funk: remove this function ...

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:172:
struct pngFixedToFloat {
On 2017/03/06 05:21:47, noel gordon wrote:
> ... since this is what we use now.

Acknowledged.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333:
int y = rowIndex + frameRect.y();
On 2017/03/06 05:21:47, noel gordon wrote:
> The comment says the frameRect is used as the source of truth.  But this code
> does not match the comment, viz., where is rowIndex compared to the
> frameRect.height()?
> 
> Most the code currently does is compare y to size().height(), but
> frameRect.height() can be less than that height.  So either the code or the
> comment is wrong.  Can we fix?

Code is wrong. Fixed.

Do you recall cases where libpng sends extra rows? I would expect to potentially
see them in cases like 60 in crbug.com/698808, but it does not in that case.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:334:
DCHECK_GE(y, 0);
On 2017/03/06 05:21:47, noel gordon wrote:
> If this y calculation ever wrapped around and produced a negative, a DCHECK
> won't save you from the security flaw :)
> 
> You could try using this:
> 
> if (UNLIKELY(y < 0))
>   return;
> 
> or otherwise, leave the case as it was:
> 
> if (y < 0 || y >= size().height())
>   return;

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:431:
if (alphaMask != 255 && !m_currentBufferSawAlpha)
On 2017/03/06 05:21:46, noel gordon wrote:
> Is the && !m_currentBufferSawAlpha part worth it anymore? 

I have not timed the difference. It just saves a write to memory.

> Anything wrong with
> writing as follows?
> 
> if (alphaMask != 255)
>   m_currentBufferSawAlpha = true;

That looks fine to me. Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:457: 
On 2017/03/06 05:21:46, noel gordon wrote:
> I don't see failed() being used in other image decoder impls of this function,
> not sure its needed. 

Not directly, but GIFImageDecoder checks for m_reader, which it deletes on
failure.

> But then again, with the failure strategy you are using
> for APNG, it might have a use.  Hmmm, start this routine with  
> 
> if (failed() || !isDecodedSizeAvailable())
>   return false;

That would be redundant - isDecodedSizeAvailable returns false if failed().

> 
> if that gels with your APNG failure strategy?

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:461:
if (m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/03/06 05:21:46, noel gordon wrote:
> frameIsCompleteAtIndex, frameHasAlphaAtIndex: you made some points about them
> earlier in reply #20, providing references to where Peter or myself complained
> about these functions, and the patch set that changed their meaning
> https://codereview.chromium.org/14317005 (which as an aside, lead to the
> chicken+egg bug I believe).

Do you mean crbug.com/593430?

> 
> Anyho, we could maybe simplify the code here for the first frame and other
> frames as follows:
> 
>   if (!index)
>     return ImageDecoder::frameIsCompleteAtIndex(0);
>   DCHECK(m_reader);
>   bool frameIsReceivedAtIndex = index < m_reader->frameCount();
>   return frameIsReceivedAtIndex;

That's definitely simpler, but it's different from both WEBP and GIF, which will
both return whether the frame has been received for index 0 of an animated
image.

One of the few places this makes a difference is here [1], which would be hard
to test for.

[1]
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

> 
> and with a layout test for the <canvas> use-case, I'd expect we'd be fine.

As in crrev.com/2728193004? That passes for me whether I use
firstFrameFullyReceived or ImageDecoder::frameIsCompleteAtIndex, but it does not
deal with [1].

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:481:
void PNGImageDecoder::frameComplete() {
On 2017/03/06 05:21:46, noel gordon wrote:
> Could we move this up to just after rowAvailable() please, to keep all the
> callbacks from libpng together?

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:490:
longjmp(JMPBUF(m_reader->pngPtr()), 1);
On 2017/03/06 05:21:46, noel gordon wrote:
> Add a return; after this longjmp (just like we do in rowAvailable) ?

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.h:67: 
On 2017/03/06 05:21:47, noel gordon wrote:
> nit: empty line.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:155: if
(index) {
On 2017/03/06 05:21:47, noel gordon wrote:
> Optional: would understanding be aided by having all this code in a helper
> routine?  e.g, 
> 
> bool PNGImageReader::shouldDecodeWithNewPNG(size_t index) const {
>   ...
> }

By itself, I did not think so, but I created a couple of local booleans inside
the method that I think make it clearer. Let me know what you think.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:209: if
(frameRect.location() == IntPoint() &&
On 2017/03/06 05:21:47, noel gordon wrote:
> I think you want 
> 
>   if (frameRect == IntRect(IntPoint(), m_decoder->size()))
> 
> here, right?

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:224:
memcpy(header, chunk, headerSize);
On 2017/03/06 05:21:47, noel gordon wrote:
> "more semantic insight."  This memcpy seems superfluous and so is having an
> extra png_byte header[headerSize] when we could jsut read the same data from
the
> chunk, this additional code seems to add unnecessary work, not insight.  How
> about ...
> 
>   constexpr size_t headerSize = kBufferSize;
>   char readBuffer[headerSize];
>   const png_byte* chunk =
>       readAsConstPngBytep(reader, m_initialOffset, headerSize, readBuffer);
> 
>   png_byte* header = const_cast<png_byte*>(chunk);
>   png_save_uint_32(header + 16, frameRect.width());
>   ...

Done. We still need to memcpy if the FastSharedBufferReader returned a pointer
to its own memory.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:349:
const png_byte* crcLoc =
On 2017/03/06 05:21:48, noel gordon wrote:
> abbv.  crcLoc -> crcPosition.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:351:
const png_uint_32 crc = png_get_uint_32(crcLoc);
On 2017/03/06 05:21:47, noel gordon wrote:
> s/const//

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:352:
const png_uint_32 actual = crc32(crc32(0, Z_NULL, 0), chunk, chunkLength + 4);
On 2017/03/06 05:21:47, noel gordon wrote:
> return crc == crc32(crc32(0, Z_NULL, 0), chunk, ... ?

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:357:
png_uint_32 sequenceNumber = png_get_uint_32(position);
On 2017/03/06 05:21:48, noel gordon wrote:
> sequenceNumber -> sequence and reformat the code here.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:382:
DCHECK(m_frameInfo.isEmpty());
On 2017/03/06 05:21:48, noel gordon wrote:
> Move this DCHECK to line before the m_frameInfo.push_back() ?

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:424: if
(m_newFrame.startOffset == 0) {
On 2017/03/06 05:21:47, noel gordon wrote:
> Comparison to 0.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:439:
const png_byte* sequenceNumPosition =
On 2017/03/06 05:21:47, noel gordon wrote:
> sequenceNumPosition -> sequencePosition

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:447: if
(m_newFrame.startOffset != 0) {
On 2017/03/06 05:21:48, noel gordon wrote:
> Comparsion to 0.  !m_newFrame.startOffset

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:535: //
Initialize the newFrame by setting the readOffset to 0.
On 2017/03/06 05:21:47, noel gordon wrote:
> Comment does not match the code, maybe throw it.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:549: //
If we encounter the IDAT chunk, we're done with the png header
On 2017/03/06 05:21:47, noel gordon wrote:
> Superfluous comment.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:558: //
Call headerAvailable manually. IDAT will be supplied (possibly
On 2017/03/06 05:21:48, noel gordon wrote:
> I'm not sure this comment adds value either.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:600:
m_ignoreAnimation = true;
On 2017/03/06 05:21:48, noel gordon wrote:
> The logic of how m_ignoreAnimation is used is interesting, but if you return
> false here, note that one of those invalid image tests (Default image's fcTL
> size not matching IHDR) starts to pass.

But others will fail, whereas today we successfully show the default image (for
a failure prior to IDAT in an otherwise valid static PNG).

> 
> Maybe with your failure strategy, which seems to be not to display the default
> image,

The strategy is described in the commit message, although maybe it should be
consolidated:

"If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded."

"Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed."

> you could return false anytime you set m_ignoreAnimation to false. 
> Something to think about.

Then we would call setFailed() and display nothing (unless this goes along with
more changes?), whereas currently this displays the default image, as intended.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:626:
void PNGImageReader::clearDecodeState(size_t frameIndex) {
On 2017/03/06 05:21:48, noel gordon wrote:
> frameIndex -> index.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:627: if
(frameIndex == 0) {
On 2017/03/06 05:21:47, noel gordon wrote:
> Comparison to 0.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:667: if
(!frameWidth || !frameHeight || xOffset + frameWidth > m_width ||
On 2017/03/06 05:21:48, noel gordon wrote:
> if (!checkSequenceNumber(data))
>   return false;
> if (!frameWidth || !frameHeight)
>   return false;
> if (xOffset + frameWidth > m_width || ...)
>   return false;

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:674:
m_newFrame.duration = (delayDenominator == 0)
On 2017/03/06 05:21:47, noel gordon wrote:
>   if (delayDenominator)
>     m_newFrame.duration = delayNumerator * 1000 / delayDenominator;
>   else
>     m_newFrame.duration = delayNumerator * 10;

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:677:
m_newFrame.frameRect = IntRect(xOffset, yOffset, frameWidth, frameHeight);
On 2017/03/06 05:21:48, noel gordon wrote:
> Odd that m_newFrame.frameRect is set here.  Would it make sense to set it just
> after you vetted it at line 667 above.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:678:
const uint8_t disposalMethod = data[24];
On 2017/03/06 05:21:47, noel gordon wrote:
> Space before this line, and make it
> 
> const png_byte& disposalMethod = data[24]; 

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:695:
const uint8_t alphaBlend = data[25];
On 2017/03/06 05:21:47, noel gordon wrote:
> const png_byte& alphaBlend = data[25]; 

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:707:
m_fctlNeedsDatChunk = true;
On 2017/03/06 05:21:48, noel gordon wrote:
> Space before this line.

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:83:
size_t frameCount() const;
On 2017/03/06 05:21:48, noel gordon wrote:
> Minor comment fix, we should just in-line this function.
> 
> // Number of complete frames parsed so far, includes frame 0 even if partial.
> size_t frameCount() const { return m_frameInfo.size(); }

Done.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:89: bool
firstFrameFullyReceived() const;
On 2017/03/06 05:21:48, noel gordon wrote:
> In-line this function?  To do that, you could add
> 
> static constexpr size_t kFirstFrameIndicator = 1;
> 
> to the private section of this class.

Done.

> 
> Also, if you decide to not use it in frameIsCompleteAtIndex anymore, then this
> function can be private to the ImageReader class.

Noel Gordon

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode326 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: On 2017/03/06 03:06:45, noel gordon wrote: > On 2017/03/03 ...

3 years, 9 months ago (2017-03-08 15:36:02 UTC) #33

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/80001/third_party/WebKit/Sour...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:326: 
On 2017/03/06 03:06:45, noel gordon wrote:
> On 2017/03/03 21:56:40, scroggo_chromium wrote:
> > On 2017/03/01 11:29:46, noel gordon wrote:
> 
> > > Yes, this is a HTML spec requirement.  The initial (index == 0) frame of a
> > > static image, or animated image, can be drawn by canvas elements (canvas
2d
> > and
> > > WebGL).  They check FrameComplete status before they attempt to draw, and
> fail
> > > the draw if FrameComplete is not true.
> > > 
> > > This means that an overridden frameIsCompleteAtIndex(index) in the new PNG
> > > decoder needs to do the right thing for index == 0, and return the decode
> > > complete status, not whether the frame is fully received.
> > > 
> > > I'm not sure the frameIsCompleteAtIndex() code here is doing the right
thing
> > > yet.  A layout test (draw an animated PNG to <canvas>) might help sort
that
> > out.
> > 
> > Happy to provide a layout test. Is there a particular concern you have with
> the
> > code? And what do you have in mind? Something like:
> > - provide enough data for part of the second frame, verify that we still
draw
> > the first frame?
> 
> Yes, and I filed crbug.com/698487 about it, PTAL.

Yeap looked, good description of the issue therein, thanks for filing the bug.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333:
int y = rowIndex + frameRect.y();
On 2017/03/07 20:25:36, scroggo_chromium wrote:
> On 2017/03/06 05:21:47, noel gordon wrote:
> > The comment says the frameRect is used as the source of truth.  But this
code
> > does not match the comment, viz., where is rowIndex compared to the
> > frameRect.height()?
> > 
> > Most the code currently does is compare y to size().height(), but
> > frameRect.height() can be less than that height.  So either the code or the
> > comment is wrong.  Can we fix?
> 
> Code is wrong. Fixed.

Good.

> Do you recall cases where libpng sends extra rows? I would expect to
potentially
> see them in cases like 60 in crbug.com/698808, but it does not in that case.

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1205
https://bugs.webkit.org/attachment.cgi?id=140137&action=prettypatch

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:461:
if (m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/03/07 20:25:36, scroggo_chromium wrote:
> On 2017/03/06 05:21:46, noel gordon wrote:
> > frameIsCompleteAtIndex, frameHasAlphaAtIndex: you made some points about
them
> > earlier in reply #20, providing references to where Peter or myself
complained
> > about these functions, and the patch set that changed their meaning
> > https://codereview.chromium.org/14317005 (which as an aside, lead to the
> > chicken+egg bug I believe).
> 
> Do you mean crbug.com/593430?

Yeap.

> > 
> > Anyho, we could maybe simplify the code here for the first frame and other
> > frames as follows:
> > 
> >   if (!index)
> >     return ImageDecoder::frameIsCompleteAtIndex(0);
> >   DCHECK(m_reader);
> >   bool frameIsReceivedAtIndex = index < m_reader->frameCount();
> >   return frameIsReceivedAtIndex;
> 
> That's definitely simpler, but it's different from both WEBP and GIF, which
will
> both return whether the frame has been received for index 0 of an animated
> image.
> 
> One of the few places this makes a difference is here [1], which would be hard
> to test for.

Fine with leaving it alone given hard-to-test. 

> [1]
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...

Could not quite follow here, but 535 lands in the middle of a comment

// may be in the past, meaning the next time through this function we'll

Something/one updated BitmapImage after you grabbed the URL maybe, or the
comment was the thing you wanted to point out to me?
> > 
> > and with a layout test for the <canvas> use-case, I'd expect we'd be fine.
> 
> As in crrev.com/2728193004? That passes for me whether I use
> firstFrameFullyReceived or ImageDecoder::frameIsCompleteAtIndex, but it does
not
> deal with [1].

and "hard-to-test".  Re: crrev.com/2728193004 PTAL :)

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:155: if
(index) {
On 2017/03/07 20:25:38, scroggo_chromium wrote:
> On 2017/03/06 05:21:47, noel gordon wrote:
> > Optional: would understanding be aided by having all this code in a helper
> > routine?  e.g, 
> > 
> > bool PNGImageReader::shouldDecodeWithNewPNG(size_t index) const {
> >   ...
> > }
> 
> By itself, I did not think so, but I created a couple of local booleans inside
> the method that I think make it clearer. Let me know what you think.

Yeah, the local bools are active code and help explain.  Better I think.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:209: if
(frameRect.location() == IntPoint() &&
On 2017/03/07 20:25:37, scroggo_chromium wrote:
> On 2017/03/06 05:21:47, noel gordon wrote:
> > I think you want 
> > 
> >   if (frameRect == IntRect(IntPoint(), m_decoder->size()))
> > 
> > here, right?
> 
> Done.

I should've also suggested this

if (frameRect == IntRect(0, 0, m_width, m_height))

to use here.  We need to know we have the image width / height at this point in
the code.  Use whichever one works best, your call.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:224:
memcpy(header, chunk, headerSize);
On 2017/03/07 20:25:37, scroggo_chromium wrote:
> On 2017/03/06 05:21:47, noel gordon wrote:
> > "more semantic insight."  This memcpy seems superfluous and so is having an
> > extra png_byte header[headerSize] when we could jsut read the same data from
> the
> > chunk, this additional code seems to add unnecessary work, not insight.  How
> > about ...
> > 
> >   constexpr size_t headerSize = kBufferSize;
> >   char readBuffer[headerSize];
> >   const png_byte* chunk =
> >       readAsConstPngBytep(reader, m_initialOffset, headerSize, readBuffer);
> > 
> >   png_byte* header = const_cast<png_byte*>(chunk);
> >   png_save_uint_32(header + 16, frameRect.width());
> >   ...
> 
> Done. 

>We still need to memcpy if the FastSharedBufferReader returned a pointer
> to its own memory.

Scanned the new code here, scratching my head wondering why the rest of the code
does not do similarly when we use the readAsConstPngBytep API.  I might be
missing something.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:447: if
(m_newFrame.startOffset != 0) {
On 2017/03/07 20:25:37, scroggo_chromium wrote:
> On 2017/03/06 05:21:48, noel gordon wrote:
> > Comparsion to 0.  !m_newFrame.startOffset
> 
> Done.

Bug on my part suggesting !m_newFrame.startOffset.  Wrong, please re-check your
change here, should be
 
if (m_newFrame.startOffset != 0) -> if (m_newFrame.startOffset)

please, and my apologies.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:558: //
Call headerAvailable manually. IDAT will be supplied (possibly
On 2017/03/07 20:25:37, scroggo_chromium wrote:
> On 2017/03/06 05:21:48, noel gordon wrote:
> > I'm not sure this comment adds value either.
> 
> Done.

Thanks, Function-level comment says IDAT, no need to keep mentioning IDAT a
further 3 times in comment.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:600:
m_ignoreAnimation = true;
On 2017/03/07 20:25:37, scroggo_chromium wrote:
> On 2017/03/06 05:21:48, noel gordon wrote:
> > The logic of how m_ignoreAnimation is used is interesting, but if you return
> > false here, note that one of those invalid image tests (Default image's fcTL
> > size not matching IHDR) starts to pass.
> 
> But others will fail, whereas today we successfully show the default image
(for
> a failure prior to IDAT in an otherwise valid static PNG).
> 
> > 
> > Maybe with your failure strategy, which seems to be not to display the
default
> > image,
> 
> The strategy is described in the commit message, although maybe it should be
> consolidated:

> "If they contain an error before IDAT, the image
> is treated as a static PNG, as recommended by the spec [1]. If they contain
> an error after IDAT, this is a failure, since some frames may have been
> decoded."
> 
> "Future work:
> - Revert to showing the default image for failures past IDAT. This is tricky,
> since the client may be holding on to previous frames, and we need to make
> sure they switch to using the IDAT frame, even if it is not part of the
> animation. For now, we mark the decoder as having failed."

IC, thanks for pointing at these sections of the ChangeLog, future work to do
...  think I mentioned awhile's back that I thought getting those test cases
working might be a time sink :)   Pushing them off into crbug as future work we
need to address is fine by me.
 
> > you could return false anytime you set m_ignoreAnimation to false. 
> > Something to think about.
> 
> Then we would call setFailed() and display nothing (unless this goes along
with
> more changes?), whereas currently this displays the default image, as
intended.

Displaying nothing (broken image) could be interpreted as "or show an error to
the user", which the spec might seem to allow as an option for the failing image
test cases (based on the notes I read in that Philip APNG test suite, and
assuming those notes are correct). 

Hard case is "error after IDAT", many frames already decoded / shown, and then
you hit an error, and show default image or broken image.  The visual effect of
that might be jarring to users, dunno ...

Noel Gordon

Looking good, minor nits and comment fixes ... https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp File third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp (right): https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp#newcode87 third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp:87: const ...

3 years, 9 months ago (2017-03-08 16:26:15 UTC) #34

scroggo_chromium

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-08 20:20:48 UTC) #35

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends,
although we currently ignore this for static PNGs anyway.

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed.

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode333 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333: int y = rowIndex + frameRect.y(); On 2017/03/08 15:36:01, ...

3 years, 9 months ago (2017-03-08 20:53:23 UTC) #36

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333:
int y = rowIndex + frameRect.y();
On 2017/03/08 15:36:01, noel gordon wrote:
> On 2017/03/07 20:25:36, scroggo_chromium wrote:
> > On 2017/03/06 05:21:47, noel gordon wrote:
> > > The comment says the frameRect is used as the source of truth.  But this
> code
> > > does not match the comment, viz., where is rowIndex compared to the
> > > frameRect.height()?
> > > 
> > > Most the code currently does is compare y to size().height(), but
> > > frameRect.height() can be less than that height.  So either the code or
the
> > > comment is wrong.  Can we fix?
> > 
> > Code is wrong. Fixed.
> 
> Good.
> 
> > Do you recall cases where libpng sends extra rows? I would expect to
> potentially
> > see them in cases like 60 in crbug.com/698808, but it does not in that case.
> 
> http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1205

Thanks! According to http://libpng.org/pub/png/libpng.html, this has been fixed
in versions 1.4.3 and 1.2.44, which I would expect to include our version (1.6).

> https://bugs.webkit.org/attachment.cgi?id=140137&action=prettypatch

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:461:
if (m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/03/08 15:36:01, noel gordon wrote:
> On 2017/03/07 20:25:36, scroggo_chromium wrote:
> > On 2017/03/06 05:21:46, noel gordon wrote:
> > > frameIsCompleteAtIndex, frameHasAlphaAtIndex: you made some points about
> them
> > > earlier in reply #20, providing references to where Peter or myself
> complained
> > > about these functions, and the patch set that changed their meaning
> > > https://codereview.chromium.org/14317005 (which as an aside, lead to the
> > > chicken+egg bug I believe).
> > 
> > Do you mean crbug.com/593430?
> 
> Yeap.
> 
> > > 
> > > Anyho, we could maybe simplify the code here for the first frame and other
> > > frames as follows:
> > > 
> > >   if (!index)
> > >     return ImageDecoder::frameIsCompleteAtIndex(0);
> > >   DCHECK(m_reader);
> > >   bool frameIsReceivedAtIndex = index < m_reader->frameCount();
> > >   return frameIsReceivedAtIndex;
> > 
> > That's definitely simpler, but it's different from both WEBP and GIF, which
> will
> > both return whether the frame has been received for index 0 of an animated
> > image.
> > 
> > One of the few places this makes a difference is here [1], which would be
hard
> > to test for.
> 
> Fine with leaving it alone given hard-to-test. 
> 
> > [1]
> >
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
> 
> Could not quite follow here, but 535 lands in the middle of a comment
> 
> // may be in the past, meaning the next time through this function we'll
> 
> Something/one updated BitmapImage after you grabbed the URL maybe, or the
> comment was the thing you wanted to point out to me?

Someone updated. A more permanent link, at an arbitrary commit:

https://chromium.googlesource.com/chromium/src/+/396fddbbebaa3e693fdcf2b6f6fd...

This line looks like it would break with your suggestion here. (More below.)

> > > 
> > > and with a layout test for the <canvas> use-case, I'd expect we'd be fine.
> > 
> > As in crrev.com/2728193004? That passes for me whether I use
> > firstFrameFullyReceived or ImageDecoder::frameIsCompleteAtIndex, but it does
> not
> > deal with [1].
> 
> and "hard-to-test".  Re: crrev.com/2728193004 PTAL :)
> 

Already looked, as stated in my comment above. "That passes for me [either way]"
(when run as a layout test - it fails for me either way when I just open the
page). It does not cover the case I am concerned about, though. As I understand
it, your test moves off the first frame, and then verifies that we draw the
first frame to the canvas. My concern is about when we're catching up. If
frameAfterNext is frame 0, and frameIsCompleteAtIndex(0) never returns true,
we'll break out of this loop, when we should have executed it.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:209: if
(frameRect.location() == IntPoint() &&
On 2017/03/08 15:36:01, noel gordon wrote:
> On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > On 2017/03/06 05:21:47, noel gordon wrote:
> > > I think you want 
> > > 
> > >   if (frameRect == IntRect(IntPoint(), m_decoder->size()))
> > > 
> > > here, right?
> > 
> > Done.
> 
> I should've also suggested this
> 
> if (frameRect == IntRect(0, 0, m_width, m_height))
> 
> to use here.  We need to know we have the image width / height at this point
in
> the code.  Use whichever one works best, your call.

I have a slight preference for IntRect(0, 0, m_width, m_height), but I do not
have a strong preference. They are slightly different - the former calls a
method that returns ints, and m_width and m_height are png_uint_32.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:224:
memcpy(header, chunk, headerSize);
On 2017/03/08 15:36:02, noel gordon wrote:
> On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > On 2017/03/06 05:21:47, noel gordon wrote:
> > > "more semantic insight."  This memcpy seems superfluous and so is having
an
> > > extra png_byte header[headerSize] when we could jsut read the same data
from
> > the
> > > chunk, this additional code seems to add unnecessary work, not insight. 
How
> > > about ...
> > > 
> > >   constexpr size_t headerSize = kBufferSize;
> > >   char readBuffer[headerSize];
> > >   const png_byte* chunk =
> > >       readAsConstPngBytep(reader, m_initialOffset, headerSize,
readBuffer);
> > > 
> > >   png_byte* header = const_cast<png_byte*>(chunk);
> > >   png_save_uint_32(header + 16, frameRect.width());
> > >   ...
> > 
> > Done. 
> 
> >We still need to memcpy if the FastSharedBufferReader returned a pointer
> > to its own memory.
> 
> Scanned the new code here, scratching my head wondering why the rest of the
code
> does not do similarly when we use the readAsConstPngBytep API.  I might be
> missing something.
> 

FastSharedBufferReader returns a pointer, which is either to its own memory (if
the requested block was contiguous) or to the client owned pointer (readBuffer,
in this case). If we're just reading it (as in most cases in this file), either
one is fine. But if we intend to modify it, like we do down below with
png_save_uint_32, we need to make sure we're acting on our own memory, rather
than the memory owned by the FastSharedBufferReader.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:447: if
(m_newFrame.startOffset != 0) {
On 2017/03/08 15:36:01, noel gordon wrote:
> On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > On 2017/03/06 05:21:48, noel gordon wrote:
> > > Comparsion to 0.  !m_newFrame.startOffset
> > 
> > Done.
> 
> Bug on my part suggesting !m_newFrame.startOffset.  Wrong, please re-check
your
> change here, should be
>  
> if (m_newFrame.startOffset != 0) -> if (m_newFrame.startOffset)
> 
> please, and my apologies.

Thanks. I noticed the bug and corrected :)

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:558: //
Call headerAvailable manually. IDAT will be supplied (possibly
On 2017/03/08 15:36:02, noel gordon wrote:
> On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > On 2017/03/06 05:21:48, noel gordon wrote:
> > > I'm not sure this comment adds value either.
> > 
> > Done.
> 
> Thanks, Function-level comment says IDAT, no need to keep mentioning IDAT a
> further 3 times in comment.

I only see one left after responding to the last round of comments. ("This loop
peeks at the chunk tag until the IDAT chunk is found.") Removed.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:600:
m_ignoreAnimation = true;
On 2017/03/08 15:36:01, noel gordon wrote:
> On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > On 2017/03/06 05:21:48, noel gordon wrote:
> > > The logic of how m_ignoreAnimation is used is interesting, but if you
return
> > > false here, note that one of those invalid image tests (Default image's
fcTL
> > > size not matching IHDR) starts to pass.
> > 
> > But others will fail, whereas today we successfully show the default image
> (for
> > a failure prior to IDAT in an otherwise valid static PNG).
> > 
> > > 
> > > Maybe with your failure strategy, which seems to be not to display the
> default
> > > image,
> > 
> > The strategy is described in the commit message, although maybe it should be
> > consolidated:
> 
> > "If they contain an error before IDAT, the image
> > is treated as a static PNG, as recommended by the spec [1]. If they contain
> > an error after IDAT, this is a failure, since some frames may have been
> > decoded."
> > 
> > "Future work:
> > - Revert to showing the default image for failures past IDAT. This is
tricky,
> > since the client may be holding on to previous frames, and we need to make
> > sure they switch to using the IDAT frame, even if it is not part of the
> > animation. For now, we mark the decoder as having failed."
> 
> IC, thanks for pointing at these sections of the ChangeLog, future work to do
> ...  think I mentioned awhile's back that I thought getting those test cases
> working might be a time sink :)   Pushing them off into crbug as future work
we
> need to address is fine by me.

I interpreted that to mean the three specific cases we were discussing in the
comment, which I filed as crbug.com/698808.

Other types of errors could show up after IDAT, e.g.
- more acTL chunks (I don't see that explicitly called out in the spec, but
https://philip.html5.org/tests/apng/tests.html considers it an error, at least
if there is more than one before IDAT)
- fcTL chunks with invalid data
- some internal libpng error

These we already detect as errors (unlike the ones in crbug.com/698808 which we
ignore), but we do not revert to showing the default image. (I have now filed
crbug.com/699675 to cover these cases.)

>  
> > > you could return false anytime you set m_ignoreAnimation to false. 
> > > Something to think about.
> > 
> > Then we would call setFailed() and display nothing (unless this goes along
> with
> > more changes?), whereas currently this displays the default image, as
> intended.
> 
> Displaying nothing (broken image) could be interpreted as "or show an error to
> the user", which the spec might seem to allow as an option for the failing
image
> test cases (based on the notes I read in that Philip APNG test suite, and
> assuming those notes are correct).

It more or less says that (though it seems to defer to the spec, and states
"This page is somewhat incomplete and quite possibly incorrect – use with
caution." at the top), but the spec (and the test suite) says "It is strongly
recommended that when any error is encountered decoders should discard all
subsequent frames, stop the animation, and revert to displaying the default
image. A decoder which detects an error before the animation has started should
display the default image. An error message may be displayed to the user if
appropriate."

I interpret that to prefer showing the default image. So I chose to show the
default image here, given that it's simple to handle. (We could take this a step
further and handle errors after IDAT but before we started the animation, though
I'm worried it would be ugly and it doesn't handle the larger issue - the
animation has already started.)

> 
> Hard case is "error after IDAT", many frames already decoded / shown, and then
> you hit an error, and show default image or broken image.  The visual effect
of
> that might be jarring to users, dunno ...

I think we already have everything necessary to show a broken image - call
setFailed(), which is what this patch does.


But I agree that handling these cases is harder, if we show the default image.
Perhaps the visual effect would be jarring, but that is not my primary concern.
My concern is that BitmapImage may have already decided how many frames there
are and what their durations are, some frames may have been cached etc, and I
suspect we may need to make changes upstack in order to correct that. (One hacky
solution might be to switch up decode(i) to always decode the default image, and
maybe max out the duration.)

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/graphics/DeferredImageDecoderTest.cpp:87:
const unsigned char whiteGIF[] = {
On 2017/03/08 16:26:14, noel gordon wrote:
> WhiteGIF, this is PNG no?  Comment says that: PNG.

Comment was incorrect. This is a GIF.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:486:
if (!index)
On 2017/03/08 16:26:14, noel gordon wrote:
> Seems fine.  Maybe in future we should get rid of the distinction b/w
> firstFrameFullyReceived() for the first animated frame, and index <
> m_reader->frameCount() for other animated frames, if that is possible.

Not sure I follow you. Do you mean m_reader->frameCount() should return 0 until
we have fully received the first frame? I think that would prevent us from
progressively decoding frame 0 as it becomes available.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:231:
char readBuffer[8];  // large enough to identify a chunk.
On 2017/03/08 16:26:15, noel gordon wrote:
> Move this into the while loop and no comment needed.

Done.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:320:
offset += 12 + length;
On 2017/03/08 16:26:14, noel gordon wrote:
> Space before this line.

Done.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:544: //
12 is the length, tag and crc part of the chunk, which are all 4B.
On 2017/03/08 16:26:14, noel gordon wrote:
> The comment is a fine fact, but doesn't tell me why we need to break here. 
> Because we need to ensure we have a complete chunk of data to read or some
such?
>  

It makes it simpler if we wait until the entire chunk is available. I've
replaced with the following:

// Do not attempt to parse the chunk until the entire chunk is available.

WDYT?

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:568:
m_decoder->setRepetitionCount((int)repetitionCount - 1);
On 2017/03/08 16:26:15, noel gordon wrote:
> c-style-cast.

Done.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:644:
png_uint_16 delayNumerator = png_get_uint_16(data + 20);
On 2017/03/08 16:26:15, noel gordon wrote:
> Nearly done here. Move these two lines up to 633
> 
> png_uint_32 yOffset = png_get_uint_32(data + 16);
> png_uint_16 delayNumerator = png_get_uint_16(data + 20);
> png_uint_16 delayDenominator = png_get_uint_16(data + 22);
> 
> if (!checkSequenceNumber(data))
>   ...

Done.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:138: //
This is used as a value for the byteLength of a frameInfo struct to
On 2017/03/08 16:26:15, noel gordon wrote:
> This is used as a value -> Value used

Done.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:144: //
This is used to temporarily store frame information, until it is pushed to
On 2017/03/08 16:26:15, noel gordon wrote:
> Stores information about a frame until it can be pushed to |m_frameInfo| once
> all the frame data has been read from the stream.
> 

Done.

Noel Gordon

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode333 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333: int y = rowIndex + frameRect.y(); On 2017/03/08 20:53:21, ...

3 years, 9 months ago (2017-03-13 12:16:17 UTC) #37

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:333:
int y = rowIndex + frameRect.y();
On 2017/03/08 20:53:21, scroggo_chromium wrote:
> On 2017/03/08 15:36:01, noel gordon wrote:
> > On 2017/03/07 20:25:36, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:47, noel gordon wrote:
> > > > The comment says the frameRect is used as the source of truth.  But this
> > code
> > > > does not match the comment, viz., where is rowIndex compared to the
> > > > frameRect.height()?
> > > > 
> > > > Most the code currently does is compare y to size().height(), but
> > > > frameRect.height() can be less than that height.  So either the code or
> the
> > > > comment is wrong.  Can we fix?
> > > 
> > > Code is wrong. Fixed.
> > 
> > Good.
> > 
> > > Do you recall cases where libpng sends extra rows? I would expect to
> > potentially
> > > see them in cases like 60 in crbug.com/698808, but it does not in that
case.
> > 
> > http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1205
> 
> Thanks! According to http://libpng.org/pub/png/libpng.html, this has been
fixed
> in versions 1.4.3 and 1.2.44, which I would expect to include our version
(1.6).

Yes, 1.6 has the fix.  But other packagers of Chromium code (linux distos, etc)
often configure their Chromium build to use an external (system) libpng and we
do not prevent that e.g., by checking PNG version with PNG_LIBPNG_VER_XXXXX in
the PNGImageDecoder code.  So we retain this code so we do not expose Chromium
packagers to the CVE.

If we wanted to ditch this CVE checking code, we could, but we'd maybe need to
also enforce it with PNG_LIBPNG_VER_MINOR/MAJOR etc checks to ensure the PNG
version being used has the CVE fix.

> > https://bugs.webkit.org/attachment.cgi?id=140137&action=prettypatch

^^^ notes fast/images/png-extra-row-crash.html, now called
images/png-extra-row-crash.html, presumably uses a PNG image that caused the
problem (if you wanted to see an example of such an image).

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:461:
if (m_reader->parseCompleted() && m_reader->frameCount() == 1)
On 2017/03/08 20:53:21, scroggo_chromium wrote:
> On 2017/03/08 15:36:01, noel gordon wrote:
> > On 2017/03/07 20:25:36, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:46, noel gordon wrote:
> > > > frameIsCompleteAtIndex, frameHasAlphaAtIndex: you made some points about
> > them
> > > > earlier in reply #20, providing references to where Peter or myself
> > complained
> > > > about these functions, and the patch set that changed their meaning
> > > > https://codereview.chromium.org/14317005 (which as an aside, lead to the
> > > > chicken+egg bug I believe).
> > > 
> > > Do you mean crbug.com/593430?
> > 
> > Yeap.
> > 
> > > > 
> > > > Anyho, we could maybe simplify the code here for the first frame and
other
> > > > frames as follows:
> > > > 
> > > >   if (!index)
> > > >     return ImageDecoder::frameIsCompleteAtIndex(0);
> > > >   DCHECK(m_reader);
> > > >   bool frameIsReceivedAtIndex = index < m_reader->frameCount();
> > > >   return frameIsReceivedAtIndex;
> > > 
> > > That's definitely simpler, but it's different from both WEBP and GIF,
which
> > will
> > > both return whether the frame has been received for index 0 of an animated
> > > image.
> > > 
> > > One of the few places this makes a difference is here [1], which would be
> hard
> > > to test for.
> > 
> > Fine with leaving it alone given hard-to-test. 
> > 
> > > [1]
> > >
> >
>
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/graph...
> > 
> > Could not quite follow here, but 535 lands in the middle of a comment
> > 
> > // may be in the past, meaning the next time through this function we'll
> > 
> > Something/one updated BitmapImage after you grabbed the URL maybe, or the
> > comment was the thing you wanted to point out to me?
> 
> Someone updated. A more permanent link, at an arbitrary commit:
> 
>
https://chromium.googlesource.com/chromium/src/+/396fddbbebaa3e693fdcf2b6f6fd...
> 
> This line looks like it would break with your suggestion here. (More below.)
> 
> > > > 
> > > > and with a layout test for the <canvas> use-case, I'd expect we'd be
fine.
> > > 
> > > As in crrev.com/2728193004? That passes for me whether I use
> > > firstFrameFullyReceived or ImageDecoder::frameIsCompleteAtIndex, but it
does
> > not
> > > deal with [1].
> > 
> > and "hard-to-test".  Re: crrev.com/2728193004 PTAL :)
> > 
> 
> Already looked, as stated in my comment above. "That passes for me [either
way]"
> (when run as a layout test - it fails for me either way when I just open the
> page). 

(discussed on that bug how the "open the page case" will fail unless you deal
with the browser's same origin policy).

> It does not cover the case I am concerned about, though.
> My concern is about when we're catching up. If
> frameAfterNext is frame 0, and frameIsCompleteAtIndex(0) never returns true,
> we'll break out of this loop, when we should have executed it.

When frameIsCompleteAtIndex(index) meant what is says (the frame was decoded)
the catch-up code you referred to worked.  When frameIsCompleteAtIndex(index)
was later changed to mean the frame was fully received, the the catch-up code
still "worked", but with very different visual results for users in some cases.

More comments about frameIsCompleteAtIndex() below, but agree, let's stick with
the concept of frame-is-fully-received at index so we don't change meaning for
the catch-up code you referred to (with the perma-link).

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:209: if
(frameRect.location() == IntPoint() &&
On 2017/03/08 20:53:22, scroggo_chromium wrote:
> On 2017/03/08 15:36:01, noel gordon wrote:
> > On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:47, noel gordon wrote:
> > > > I think you want 
> > > > 
> > > >   if (frameRect == IntRect(IntPoint(), m_decoder->size()))
> > > > 
> > > > here, right?
> > > 
> > > Done.
> > 
> > I should've also suggested this
> > 
> > if (frameRect == IntRect(0, 0, m_width, m_height))
> > 
> > to use here.  We need to know we have the image width / height at this point
> in
> > the code.  Use whichever one works best, your call.
> 
> I have a slight preference for IntRect(0, 0, m_width, m_height), but I do not
> have a strong preference. They are slightly different - the former calls a
> method that returns ints, and m_width and m_height are png_uint_32.

No problems with int/uint conversion here: headerAvailable() has been called,
and it restricts the maximum image width and height, right?  IntRect(0, 0,
m_width, m_height) would better match the rest of the code.  Suggest you go with
IntRect(0, 0, m_width, m_height) here [1].

[1] This brings up another point about how m_width / m_height are set, which we
can discuss in the next round.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:224:
memcpy(header, chunk, headerSize);
On 2017/03/08 20:53:21, scroggo_chromium wrote:
> On 2017/03/08 15:36:02, noel gordon wrote:
> > On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:47, noel gordon wrote:
> > > > "more semantic insight."  This memcpy seems superfluous and so is having
> an
> > > > extra png_byte header[headerSize] when we could jsut read the same data
> from
> > > the
> > > > chunk, this additional code seems to add unnecessary work, not insight. 
> How
> > > > about ...
> > > > 
> > > >   constexpr size_t headerSize = kBufferSize;
> > > >   char readBuffer[headerSize];
> > > >   const png_byte* chunk =
> > > >       readAsConstPngBytep(reader, m_initialOffset, headerSize,
> readBuffer);
> > > > 
> > > >   png_byte* header = const_cast<png_byte*>(chunk);
> > > >   png_save_uint_32(header + 16, frameRect.width());
> > > >   ...
> > > 
> > > Done. 
> > 
> > >We still need to memcpy if the FastSharedBufferReader returned a pointer
> > > to its own memory.
> > 
> > Scanned the new code here, scratching my head wondering why the rest of the
> code
> > does not do similarly when we use the readAsConstPngBytep API.  I might be
> > missing something.
> > 
> 
> FastSharedBufferReader returns a pointer, which is either to its own memory
(if
> the requested block was contiguous) or to the client owned pointer
(readBuffer,
> in this case). If we're just reading it (as in most cases in this file),
either
> one is fine. But if we intend to modify it, like we do down below with
> png_save_uint_32, we need to make sure we're acting on our own memory, rather
> than the memory owned by the FastSharedBufferReader.

Ah we modify it, gotcha.  Agree you need to memcpy, makes sense to me now.

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:558: //
Call headerAvailable manually. IDAT will be supplied (possibly
On 2017/03/08 20:53:22, scroggo_chromium wrote:
> On 2017/03/08 15:36:02, noel gordon wrote:
> > On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:48, noel gordon wrote:
> > > > I'm not sure this comment adds value either.
> > > 
> > > Done.
> > 
> > Thanks, Function-level comment says IDAT, no need to keep mentioning IDAT a
> > further 3 times in comment.
> 
> I only see one left after responding to the last round of comments. ("This
loop
> peeks at the chunk tag until the IDAT chunk is found.") Removed.

"removed" -> nod, that's the spirit :)

https://codereview.chromium.org/2618633004/diff/220001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:600:
m_ignoreAnimation = true;
On 2017/03/08 20:53:22, scroggo_chromium wrote:
> On 2017/03/08 15:36:01, noel gordon wrote:
> > On 2017/03/07 20:25:37, scroggo_chromium wrote:
> > > On 2017/03/06 05:21:48, noel gordon wrote:
> > > > The logic of how m_ignoreAnimation is used is interesting, but if you
> return
> > > > false here, note that one of those invalid image tests (Default image's
> fcTL
> > > > size not matching IHDR) starts to pass.
> > > 
> > > But others will fail, whereas today we successfully show the default image
> > (for
> > > a failure prior to IDAT in an otherwise valid static PNG).
> > > 
> > > > 
> > > > Maybe with your failure strategy, which seems to be not to display the
> > default
> > > > image,
> > > 
> > > The strategy is described in the commit message, although maybe it should
be
> > > consolidated:
> > 
> > > "If they contain an error before IDAT, the image
> > > is treated as a static PNG, as recommended by the spec [1]. If they
contain
> > > an error after IDAT, this is a failure, since some frames may have been
> > > decoded."
> > > 
> > > "Future work:
> > > - Revert to showing the default image for failures past IDAT. This is
> tricky,
> > > since the client may be holding on to previous frames, and we need to make
> > > sure they switch to using the IDAT frame, even if it is not part of the
> > > animation. For now, we mark the decoder as having failed."
> > 
> > IC, thanks for pointing at these sections of the ChangeLog, future work to
do
> > ...  think I mentioned awhile's back that I thought getting those test cases
> > working might be a time sink :)   Pushing them off into crbug as future work
> we
> > need to address is fine by me.
> 
> I interpreted that to mean the three specific cases we were discussing in the
> comment, which I filed as crbug.com/698808.
> 
> Other types of errors could show up after IDAT, e.g.
> - more acTL chunks (I don't see that explicitly called out in the spec, but
> https://philip.html5.org/tests/apng/tests.html considers it an error, at least
> if there is more than one before IDAT)
> - fcTL chunks with invalid data
> - some internal libpng error
> 
> These we already detect as errors (unlike the ones in crbug.com/698808 which
we
> ignore), but we do not revert to showing the default image. (I have now filed
> crbug.com/699675 to cover these cases.)

(Thanks for filing the bug).
  
> > > > you could return false anytime you set m_ignoreAnimation to false. 
> > > > Something to think about.
> > > 
> > > Then we would call setFailed() and display nothing (unless this goes along
> > with
> > > more changes?), whereas currently this displays the default image, as
> > intended.
> > 
> > Displaying nothing (broken image) could be interpreted as "or show an error
to
> > the user", which the spec might seem to allow as an option for the failing
> image
> > test cases (based on the notes I read in that Philip APNG test suite, and
> > assuming those notes are correct).
> 
> It more or less says that (though it seems to defer to the spec, and states
> "This page is somewhat incomplete and quite possibly incorrect – use with
> caution." at the top), but the spec (and the test suite) says "It is strongly
> recommended that when any error is encountered decoders should discard all
> subsequent frames, stop the animation, and revert to displaying the default
> image. A decoder which detects an error before the animation has started
should
> display the default image. An error message may be displayed to the user if
> appropriate."
> 
> I interpret that to prefer showing the default image. So I chose to show the
> default image here, given that it's simple to handle. (We could take this a
step
> further and handle errors after IDAT but before we started the animation,
though
> I'm worried it would be ugly and it doesn't handle the larger issue - the
> animation has already started.)

(Don't like the sound of ugly :)  One possible interpretation is prefer to
display the default image, as you note.  But that last sentence "An error
message may be displayed to the user if appropriate" could also be interpreted
to mean do that in all cases.  The spec prose is not clear as to which, making
it harder for implementations to conform: I see no compat b/w implementing
browsers in the failure cases at this time ¯\(ツ)/¯.

> > Hard case is "error after IDAT", many frames already decoded / shown, and
then
> > you hit an error, and show default image or broken image.  The visual effect
> of
> > that might be jarring to users, dunno ...
> 
> I think we already have everything necessary to show a broken image - call
> setFailed(), which is what this patch does.

Nod.
 
> But I agree that handling these cases is harder, if we show the default image.
> Perhaps the visual effect would be jarring, but that is not my primary
concern.
> My concern is that BitmapImage may have already decided how many frames there
> are and what their durations are, some frames may have been cached etc, and I
> suspect we may need to make changes upstack in order to correct that. (One
hacky
> solution might be to switch up decode(i) to always decode the default image,
and
> maybe max out the duration.)

You have bug filed a bug re: the "harder, if we show the default image" route.
Addressing that bug might turn up the BitmapImage issues you mention.  We'll
see.

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:486:
if (!index)
On 2017/03/08 20:53:22, scroggo_chromium wrote:
> On 2017/03/08 16:26:14, noel gordon wrote:
> > Seems fine.  Maybe in future we should get rid of the distinction b/w
> > firstFrameFullyReceived() for the first animated frame, and index <
> > m_reader->frameCount() for other animated frames, if that is possible.
> 
> Not sure I follow you. Do you mean m_reader->frameCount() should return 0
until
> we have fully received the first frame? 

Not quite.

> I think that would prevent us from
> progressively decoding frame 0 as it becomes available.

It would, but my question is about something else ...

firstFrameFullyReceived() seems to be a reflection of m_reader's internal magic
whereby it puts the first frame into FrameInfo[] _before it is fully received_.

If frames were only put into FrameInfo[] _when they were fully received_, for
all index >= 0, then index < m_reader->frameCount() would be enough to implement
frameIsCompleteAtIndex() for an APNG.  (This would require some re-org in the
way the reader's part of decoding works.  Not sure how easy it would be, but I'm
not asking you to do it now either, just a quibble from me about that aspect of
the design).

Anyho, consider adding a reader function frameIsReceivedAtIndex(index) for the
animated image case, and put the reader's special cases therein:

bool PNGImageReader::frameIsReceivedAtIndex(index) const {
  if (!index)
    return firstFrameFullyReceived();
  return index < m_reader->frameCount();
}

Next, since you had concerns about progressive decoding, could we add a test for
that in the animated image case please?  A bug about it would be enough.  You
would add a HTTP layout test using the load-and-stall script, see for example:

LayoutTests/http/tests/images/png-partial-load.html
LayoutTests/http/tests/images/jpeg-partial-load.html

Possible?

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:544: //
12 is the length, tag and crc part of the chunk, which are all 4B.
On 2017/03/08 20:53:22, scroggo_chromium wrote:
> On 2017/03/08 16:26:14, noel gordon wrote:
> > The comment is a fine fact, but doesn't tell me why we need to break here. 
> > Because we need to ensure we have a complete chunk of data to read or some
> such?
> >  
> 
> It makes it simpler if we wait until the entire chunk is available. I've
> replaced with the following:
> 
> // Do not attempt to parse the chunk until the entire chunk is available.
> 
> WDYT?

Tells us "why?", rather than "what?", better imho.

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp (right): https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp#newcode486 third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:486: if (!index) On 2017/03/13 12:16:17, noel gordon wrote: > ...

3 years, 9 months ago (2017-03-13 13:50:08 UTC) #38

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageDecoder.cpp:486:
if (!index)
On 2017/03/13 12:16:17, noel gordon wrote:
> On 2017/03/08 20:53:22, scroggo_chromium wrote:
> > On 2017/03/08 16:26:14, noel gordon wrote:
> > > Seems fine.  Maybe in future we should get rid of the distinction b/w
> > > firstFrameFullyReceived() for the first animated frame, and index <
> > > m_reader->frameCount() for other animated frames, if that is possible.
> > 
> > Not sure I follow you. Do you mean m_reader->frameCount() should return 0
> until
> > we have fully received the first frame? 
> 
> Not quite.
> 
> > I think that would prevent us from
> > progressively decoding frame 0 as it becomes available.
> 
> It would, but my question is about something else ...
> 
> firstFrameFullyReceived() seems to be a reflection of m_reader's internal
magic
> whereby it puts the first frame into FrameInfo[] _before it is fully
received_.
> 
> If frames were only put into FrameInfo[] _when they were fully received_, for
> all index >= 0, then index < m_reader->frameCount() would be enough to
implement
> frameIsCompleteAtIndex() for an APNG.  (This would require some re-org in the
> way the reader's part of decoding works.  Not sure how easy it would be, but
I'm
> not asking you to do it now either, just a quibble from me about that aspect
of
> the design).
> 
> Anyho, consider adding a reader function frameIsReceivedAtIndex(index) for the
> animated image case, and put the reader's special cases therein:
> 
> bool PNGImageReader::frameIsReceivedAtIndex(index) const {
>   if (!index)
>     return firstFrameFullyReceived();
>   return index < m_reader->frameCount();
> }

Done.

> 
> Next, since you had concerns about progressive decoding, could we add a test
for
> that in the animated image case please?  A bug about it would be enough.  You
> would add a HTTP layout test using the load-and-stall script, see for example:
> 
> LayoutTests/http/tests/images/png-partial-load.html
> LayoutTests/http/tests/images/jpeg-partial-load.html
> 
> Possible?

Filed crbug.com/700892

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/240001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:544: //
12 is the length, tag and crc part of the chunk, which are all 4B.
On 2017/03/13 12:16:17, noel gordon wrote:
> On 2017/03/08 20:53:22, scroggo_chromium wrote:
> > On 2017/03/08 16:26:14, noel gordon wrote:
> > > The comment is a fine fact, but doesn't tell me why we need to break here.

> > > Because we need to ensure we have a complete chunk of data to read or some
> > such?
> > >  
> > 
> > It makes it simpler if we wait until the entire chunk is available. I've
> > replaced with the following:
> > 
> > // Do not attempt to parse the chunk until the entire chunk is available.
> > 
> > WDYT?
> 
> Tells us "why?", rather than "what?", better imho.

Done.

Noel Gordon

Code is looking solid, mostly nits ... https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp (right): https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp#newcode135 third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:135: const bool ...

3 years, 9 months ago (2017-03-13 16:00:45 UTC) #39

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp (right): https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp#newcode135 third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:135: const bool matchesIHDR = On 2017/03/13 16:00:45, noel gordon ...

3 years, 9 months ago (2017-03-13 16:37:10 UTC) #40

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp
(right):

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:135:
const bool matchesIHDR =
On 2017/03/13 16:00:45, noel gordon wrote:
> nit: frameSizeMatchesIHDR ?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:142: //
Return false on a fatal error
On 2017/03/13 16:00:44, noel gordon wrote:
> nit: sentences end with a "."

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:160:
const bool decodeAsNewPNG = shouldDecodeWithNewPNG(index);
On 2017/03/13 16:00:45, noel gordon wrote:
> nit: const bool decodeWithNewPNG = ?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:176:
const bool decodedFrameCompletely = progressivelyDecodeFirstFrame(reader);
On 2017/03/13 16:00:45, noel gordon wrote:
> nit: maybe bool decodedEntireFrame = ?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:179: 
On 2017/03/13 16:00:45, noel gordon wrote:
> This empty line could be removed ?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:206:
char readBuffer[kBufferSize];
On 2017/03/13 16:00:45, noel gordon wrote:
> nit: kBufferSize -> headerSize.

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:252: //
At this point, three scenarios are possible:
On 2017/03/13 16:00:45, noel gordon wrote:
> // Three scenarios are possible here:

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:256: //
3) This is any other chunk, most likely an IDAT chunk.
On 2017/03/13 16:00:44, noel gordon wrote:
> How about
> 
> // 3) This is any other chunk. Send it to libpng to decode.
> 
> or
>  
> // 3) This is any other chunk. Pass it to libpng for processing.
> 
> here?  /me confused about what "most likely an IDAT chunk" was trying to tell
> us, or if it was important (if it was, then something else is needed here to
> tell us why IDAT likelihood matters).

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:329:
const size_t kSizeNeededForfcTL = 26 + 4;
On 2017/03/13 16:00:44, noel gordon wrote:
> nit: would "constexpr size_t ..." work here ?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:442: //
IEND
On 2017/03/13 16:00:45, noel gordon wrote:
> Perhaps
> 
>   } else { // IEND
>     ...
> 
> here.

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:472:
m_readOffset += 12 + length;
On 2017/03/13 16:00:45, noel gordon wrote:
> loop condition: space before this line.

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:523: //
Process APNG chunks manually, and pass other chunks to libpng.
On 2017/03/13 16:00:45, noel gordon wrote:
> nit ", and pass" -> ", pass"

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:619:
#define kAPNG_BLEND_OP_SOURCE 0
On 2017/03/13 16:00:45, noel gordon wrote:
> We could drop the comment re: the spec here and maybe not use #define's when
we
> could use a typed enum these days, e.g., we could define
> 
> enum BlendOperations : png_byte {
>   APNG_BLEND_OP_SOURCE,
>   APNG_BLEND_OP_OVER
> };
> 
> or similar, right before it is used inside parseFrameInfo(), and the same for
> DisposeOperations.  Thoughts?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:622: //
Extract the frame control info and store it in m_newFrame. The length check
On 2017/03/13 16:00:45, noel gordon wrote:
> "the frame control ..." -> "the fcTL frame control ..."

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.cpp:623: //
on the data chunk has been done by the calling code.
On 2017/03/13 16:00:45, noel gordon wrote:
> "data chunk" -> "fcTL data"

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h
(right):

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:86: bool
frameIsFullyReceivedAtIndex(size_t index) const {
On 2017/03/13 16:00:45, noel gordon wrote:
> Think I said frameIsReceivedAtIndex, but yeah, I mentioned "fully" as a word
in
> my replies.  /me confused you, sorry.  Let's go with
> 
> nit: s/Fully//

No, I was not confused. "Fully" goes along with the method it calls
(firstFrameFullyReceived), and provides more information.

> and s/size_t index/size_t/

You would rather that than inline the method?

> 
> and we'll also make a change to the WEBP decoder to align the received at
index
> concept (see below).

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:139: //
since the byteLength field of a frame is at least 12, in the case of an
On 2017/03/13 16:00:45, noel gordon wrote:
> That the byteLength is a least 12 is maybe enough? -> ", in the case of an
empty
> fdAT or IDAT chunk." is irrelevant, or confusing?

Done.

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
File third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp
(right):

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Sou...
third_party/WebKit/Source/platform/image-decoders/webp/WEBPImageDecoder.cpp:167:
bool frameIsLoadedAtIndex = index < m_frameBufferCache.size();
On 2017/03/13 16:00:45, noel gordon wrote:
> nit: frameIsLoadedAtIndex -> frameIsReceivedAtIndex ?

Done.

Noel Gordon

https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h File third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h (right): https://codereview.chromium.org/2618633004/diff/300001/third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h#newcode86 third_party/WebKit/Source/platform/image-decoders/png/PNGImageReader.h:86: bool frameIsFullyReceivedAtIndex(size_t index) const { On 2017/03/13 16:37:10, scroggo_chromium ...

3 years, 9 months ago (2017-03-13 16:56:06 UTC) #41

Noel Gordon

LGTM Nice work. With the changes we've made, bugs filed for follow-up work and tests, ...

3 years, 9 months ago (2017-03-13 17:03:29 UTC) #42

scroggo_chromium

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-13 17:42:59 UTC) #43

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001)
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer rather than
recreating, for faster runtimes.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it returns
false, the client is expected to call setFailed (also update the clients).
This is safer, because setFailed may delete an object that called
initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

scroggo_chromium

On 2017/03/13 17:03:29, noel gordon wrote: > LGTM > > Nice work. With the changes ...

3 years, 9 months ago (2017-03-13 17:49:50 UTC) #44

scroggo_chromium

scroggo@chromium.org changed reviewers: + kinuko@chromium.org

3 years, 9 months ago (2017-03-13 17:51:00 UTC) #45

scroggo_chromium

kinuko@, will you please look at the files in content/browser/loader/ ? I still need OWNERS ...

3 years, 9 months ago (2017-03-13 17:51:01 UTC) #46

scroggo_chromium

The CQ bit was checked by scroggo@chromium.org to run a CQ dry run

3 years, 9 months ago (2017-03-13 17:51:17 UTC) #47

commit-bot: I haz the power

Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2618633004/340001

3 years, 9 months ago (2017-03-13 17:52:02 UTC) #48

commit-bot: I haz the power

The CQ bit was unchecked by commit-bot@chromium.org

3 years, 9 months ago (2017-03-13 18:44:30 UTC) #49

commit-bot: I haz the power

Dry run: Try jobs failed on following builders: mac_chromium_rel_ng on master.tryserver.chromium.mac (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_ng/builds/405617)

3 years, 9 months ago (2017-03-13 18:44:32 UTC) #50

scroggo

The CQ bit was checked by scroggo@google.com to run a CQ dry run

3 years, 9 months ago (2017-03-13 19:11:03 UTC) #51

commit-bot: I haz the power

Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2618633004/360001

3 years, 9 months ago (2017-03-13 19:11:37 UTC) #52

commit-bot: I haz the power

The CQ bit was unchecked by commit-bot@chromium.org

3 years, 9 months ago (2017-03-13 19:49:17 UTC) #53

commit-bot: I haz the power

Dry run: Try jobs failed on following builders: win_clang on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/186079)

3 years, 9 months ago (2017-03-13 19:49:18 UTC) #54

Noel Gordon

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-14 00:14:06 UTC) #55

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the pngstruct used during parsing is reused. Otherwise
a new pngstruct is created for the duration of the decode.

Follow the APNG spec as closely as possible. Use
https://philip.html5.org/tests/apng/tests.html
as a guide for intended behavior. All of the valid APNGs on that page work as
expected. For the invalid ones:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's
LayoutTests/fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer rather than
recreating, for faster runtimes.

Update the accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it returns
false, the client is expected to call setFailed (also update the clients).
This is safer, because setFailed may delete an object that called
initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer rather than
recreating, for faster runtimes.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it
returns false, the client is now expected to call setFailed (update the WEBP
and GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

Noel Gordon

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-14 00:16:37 UTC) #56

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer rather than
recreating, for faster runtimes.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it
returns false, the client is now expected to call setFailed (update the WEBP
and GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, for faster runtimes.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it
returns false, the client is now expected to call setFailed (update the WEBP
and GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

Noel Gordon

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-14 00:36:00 UTC) #57

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, for faster runtimes.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed from within ImageDecoder::initFrameBuffer. If it
returns false, the client is now expected to call setFailed (update the WEBP
and GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

Noel Gordon

Description was changed from ========== Add support for Animated PNG Split decoding PNG images into ...

3 years, 9 months ago (2017-03-14 00:41:44 UTC) #58

Description was changed from

==========
Add support for Animated PNG

Split decoding PNG images into two stages: parsing and decoding. During
parse, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Update the browser accept header to state that APNG is supported.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Update the browser accept header to state that APNG is supported. Split
the decoding PNG images into two stages: parsing and decoding. During
parsing, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

Noel Gordon

Description was changed from ========== Add support for Animated PNG Update the browser accept header ...

3 years, 9 months ago (2017-03-14 00:43:05 UTC) #59

Description was changed from

==========
Add support for Animated PNG

Update the browser accept header to state that APNG is supported. Split
the decoding PNG images into two stages: parsing and decoding. During
parsing, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Update the browser accept header to state that APNG is supported. Split
the decoding of PNG images into two stages: parsing and decoding. During
parsing, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

kinuko

On 2017/03/13 17:51:01, scroggo_chromium wrote: > kinuko@, will you please look at the files in ...

3 years, 9 months ago (2017-03-14 13:35:24 UTC) #60

scroggo

The CQ bit was checked by scroggo@google.com to run a CQ dry run

3 years, 9 months ago (2017-03-14 15:38:26 UTC) #61

commit-bot: I haz the power

Dry run: CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2618633004/380001

3 years, 9 months ago (2017-03-14 15:38:51 UTC) #62

commit-bot: I haz the power

The CQ bit was unchecked by commit-bot@chromium.org

3 years, 9 months ago (2017-03-14 17:27:24 UTC) #63

commit-bot: I haz the power

Dry run: Try jobs failed on following builders: win_chromium_rel_ng on master.tryserver.chromium.win (JOB_FAILED, http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/400353)

3 years, 9 months ago (2017-03-14 17:27:26 UTC) #64

scroggo_chromium

https://codereview.chromium.org/2618633004/diff/380001/content/browser/loader/mime_sniffing_resource_handler.cc File content/browser/loader/mime_sniffing_resource_handler.cc (right): https://codereview.chromium.org/2618633004/diff/380001/content/browser/loader/mime_sniffing_resource_handler.cc#newcode50 content/browser/loader/mime_sniffing_resource_handler.cc:50: "image/apng,*/*;q=0.8"; japhet@ or kinuko@, could this change be causing ...

3 years, 9 months ago (2017-03-14 18:18:53 UTC) #65

scroggo

On 2017/03/14 18:18:53, scroggo_chromium wrote: > https://codereview.chromium.org/2618633004/diff/380001/content/browser/loader/mime_sniffing_resource_handler.cc > File content/browser/loader/mime_sniffing_resource_handler.cc (right): > > https://codereview.chromium.org/2618633004/diff/380001/content/browser/loader/mime_sniffing_resource_handler.cc#newcode50 > ...

3 years, 9 months ago (2017-03-14 19:32:49 UTC) #66

scroggo

The patchset sent to the CQ was uploaded after l-g-t-m from noel@chromium.org, kinuko@chromium.org Link to ...

3 years, 9 months ago (2017-03-14 19:36:09 UTC) #68

commit-bot: I haz the power

CQ is trying da patch. Follow status at https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.org/2618633004/400001

3 years, 9 months ago (2017-03-14 19:36:37 UTC) #69

MaxStepin

On 2017/03/13 17:49:50, scroggo_chromium wrote: > On 2017/03/13 17:03:29, noel gordon wrote: > > LGTM ...

3 years, 9 months ago (2017-03-14 21:35:15 UTC) #70

commit-bot: I haz the power

CQ is committing da patch. Bot data: {"patchset_id": 400001, "attempt_start_ts": 1489520169348710, "parent_rev": "7f1e72d80a08c9ea488709193334ccbc0b4188f1", "commit_rev": "7d2b8c45afc9c0230410011293cc2e1dbb8943a7"}

3 years, 9 months ago (2017-03-14 21:36:54 UTC) #71

commit-bot: I haz the power

Description was changed from ========== Add support for Animated PNG Update the browser accept header ...

3 years, 9 months ago (2017-03-14 21:37:37 UTC) #72

Message was sent while issue was closed.

Description was changed from

==========
Add support for Animated PNG

Update the browser accept header to state that APNG is supported. Split
the decoding of PNG images into two stages: parsing and decoding. During
parsing, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.
==========

to

==========
Add support for Animated PNG

Update the browser accept header to state that APNG is supported. Split
the decoding of PNG images into two stages: parsing and decoding. During
parsing, chunks are handled one of three ways:
- acTL and fcTL chunks, which specify properties of an APNG are read and
the properties are stored. If they contain an error before IDAT, the image
is treated as a static PNG, as recommended by the spec [1]. If they contain
an error after IDAT, this is a failure, since some frames may have been
decoded. CRCs are calculated and compared to their expected values.
Mismatches are considered errors.
- fdAT and IDAT chunks have their locations stored for decoding later. Any
ordering violations result in either a static image or failed image,
depending on whether IDAT has been seen yet.
- Other chunks before IDAT are passed to libpng for processing.

Each frame is decoded as if it were a complete PNG file. fdATs are converted
to IDATs (and their CRCs are ignored**) and the IHDR is modified for subset
frames. The rowAvailable callback positions subset frames properly within the
full image buffer. For a static PNG or the first frame decoded (assuming its
size matches IHDR) the png struct used during parsing is reused. Otherwise,
a new png struct is created for the duration of the decode.

Follow the APNG spec as closely as possible and use the APNG test page [2] as
a guide for intended behavior. All of the valid APNG on the test page work
as expected. For the invalid APNG:
- Errors that occur before IDAT show the default image, as intended
- Errors afterwards are typically treated as failures (see Future work, below)
- The final three images draw incorrectly. They have incorrectly sized IDATs/
fdATs. These could be respected by checking the warning that libpng sends
(in the case of extra data) or keeping track of how many rows have been seen
(in the case of too little data), although we currently ignore this for static
PNGs anyway. (crbug.com/698808)

The first frame can be decoded progressively. Other frames are not reported
until the following fcTL chunk has been reached during parse.

Add a reference test, modified from WebKit's fast/images/animated-png.html
with the following changes:
- use window.internals.advanceImageAnimation instead of waiting for a
timeout, for more reliable testing
- disable two of the images, which look the same to my eyes, but are not
identical (I suspect due to blending differences as compared to how the
reference tests were created).

Add gtests. Update progressive tests to reuse a SharedBuffer, rather than
recreating it, to reduce test run-time.

Fix a bug in ImageFrameGenerator where it called setMemoryAllocator before
setting the data, and add related tests.

Stop calling setFailed inside ImageDecoder::initFrameBuffer, return false
instead. Clients are now expected to call setFailed (update the WEBP and
GIF clients). This is safer, because setFailed may delete an object that
called initFrameBuffer.

In WEBPImageDecoder: rename frameIsLoadedAtIndex to frameIsReceivedAtIndex,
for consistency with PNGImageDecoder.

Always call ImageFrame::setStatus last (e.g. after calling onInitFrameBuffer,
correctAlphaWhenFrameBufferSawNoAlpha).

Future work:
- Revert to showing the default image for failures past IDAT. This is tricky,
since the client may be holding on to previous frames, and we need to make
sure they switch to using the IDAT frame, even if it is not part of the
animation. For now, we mark the decoder as having failed. (crbug.com/699675)

** We cannot allow libpng to check the CRC since we modified the chunk. We
could check the CRC directly as a separate step.

[1] https://wiki.mozilla.org/APNG_Specification
[2] https://philip.html5.org/tests/apng/tests.html

BUG=1171 BUG=437662

Initial patch is a re-upload of issue 2386453003 at patchset 39
(http://crrev.com/2386453003#ps1260001) by joostouwerling@google.com, which
also used https://codereview.chromium.org/1567053002/ by maxstepin@gmail.com
as a reference.

Review-Url: https://codereview.chromium.org/2618633004
Cr-Commit-Position: refs/heads/master@{#456840}
Committed:
https://chromium.googlesource.com/chromium/src/+/7d2b8c45afc9c0230410011293cc...
==========

commit-bot: I haz the power

Committed patchset #21 (id:400001) as https://chromium.googlesource.com/chromium/src/+/7d2b8c45afc9c0230410011293cc2e1dbb8943a7

3 years, 9 months ago (2017-03-14 21:37:39 UTC) #73

scroggo_chromium

3 years, 9 months ago (2017-03-15 15:18:25 UTC) #74

Message was sent while issue was closed.

On 2017/03/14 21:35:15, MaxStepin wrote:
> On 2017/03/13 17:49:50, scroggo_chromium wrote:
> > On 2017/03/13 17:03:29, noel gordon wrote:
> > > LGTM
> > > 
> > > Nice work.  With the changes we've made, bugs filed for follow-up work and
> > > tests, etc, you might want to look over the change description to check
it's
> > > up-to-date.  Perhaps add a line attributing the contributions of Joost and
> Max
> > > (since their work helped develop this patch, IIUC).
> > 
> > Done
> 
> BTW, what about createInterlaceBuffer() issue we discussed in
> http://crrev.com/2386453003
> Maybe you can file it as a separate bug for follow-up work? Otherwise we might
> forget about it.

Filed crbug.com/701779

Issue 2618633004: Add support for Animated PNG (Closed)

Description

Patch Set 1 : Reupload of 2386453003 #

Patch Set 2 : Fix bug caught by fuzzer #

Patch Set 3 : Update accept header and its test #

Patch Set 4 : Small cleanups #

Patch Set 5 : Reject bad data. Cleanups #

Patch Set 6 : Respond to comments. Fix a bug. #

Patch Set 7 : Update frame accept header #

Patch Set 8 : Call initFrameBuffer before decoding #

Patch Set 9 : Fix GIF alpha issue #

Patch Set 10 : Inline onInitFrameBuffer in PNG #

Patch Set 11 : Respond to comments #

Patch Set 12 : Rebase #

Patch Set 13 : Respond to comments #

Patch Set 14 : Respond to comments #

Patch Set 15 : Small cleanups #

Patch Set 16 : Respond to comments #

Patch Set 17 : Respond to comments #

Patch Set 18 : Rebase #

Patch Set 19 : Fix heap-buffer-overflow in test (caught by ASAN) #

Patch Set 20 : Use a custom allocator in GIFImageDecoderTest #

Patch Set 21 : Fix LayoutTest due to accept header change #

Messages