cc: custom bilinear filtering for YUV quad.

cc: custom bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (2, 2) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (2, 2) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133, 660368
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel

[dshwang, 2016-10-07 17:52:33]

Description was changed from

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (3, 3) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (3, 3) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4    U 2x2    U sampling by GL_LINEAR
+--+--+  +--+--+  +--+--+
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  | +--++
+-----+  +-----+  +-----|
|  |· |  |  |· |  | ||·||
|  |  |  |  |  |  | +--+|
+--+--+  +--+--+  +-----+

BUG=445071, 622133
TEST=existing cc_unittests.
==========

to

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (3, 3) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (3, 3) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4    U 2x2    U sampling by GL_LINEAR
+--+--+  +--+--+  +--+--+
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  | +--++
+-----+  +-----+  +-----|
|  |· |  |  |· |  | ||·||
|  |  |  |  |  |  | +--+|
+--+--+  +--+--+  +-----+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

[inactive_dshwang_plz_cc_intel, 2016-10-07 17:52:39]

The CQ bit was checked by dongseong.hwang@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-07 17:53:13]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[dshwang, 2016-10-07 17:53:54]

dongseong.hwang@intel.com changed reviewers:
+ hubbe@chromium.org

[dshwang, 2016-10-07 17:53:55]

I found bluring bug of YUV quad drawing. It's why we see unnecessary bluring in
https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36

hubbe@, could you review?

[hubbe, 2016-10-07 18:12:00]

On 2016/10/07 17:53:55, dshwang wrote:
> I found bluring bug of YUV quad drawing. It's why we see unnecessary bluring
in
> https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> 
> hubbe@, could you review?

This is wrong. U and V planes *should* have a larger sampling area. Not doing so
will lead to over-sharpening and ugly jaggies.

[commit-bot: I haz the power, 2016-10-07 19:35:04]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-07 19:35:05]

Dry run: Try jobs failed on following builders:
  linux_precise_blink_rel on master.tryserver.blink (JOB_FAILED,
http://build.chromium.org/p/tryserver.blink/builders/linux_precise_blink_rel/...)

[dshwang, 2016-10-10 10:44:02]

Description was changed from

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (3, 3) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (3, 3) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4    U 2x2    U sampling by GL_LINEAR
+--+--+  +--+--+  +--+--+
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  | +--++
+-----+  +-----+  +-----|
|  |· |  |  |· |  | ||·||
|  |  |  |  |  |  | +--+|
+--+--+  +--+--+  +-----+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

to

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (3, 3) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (3, 3) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

[dshwang, 2016-10-10 11:08:58]

On 2016/10/07 18:12:00, hubbe wrote:
> On 2016/10/07 17:53:55, dshwang wrote:
> > I found bluring bug of YUV quad drawing. It's why we see unnecessary bluring
> in
> > https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> > 
> > hubbe@, could you review?
> 
> This is wrong. U and V planes *should* have a larger sampling area. Not doing
so
> will lead to over-sharpening and ugly jaggies.

Nope, U and V planes *should* have a same sampling area to Y plane, because they
contribute on the same pixel.

The clear counter example is
https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
In theory, the pixel test *must* produce same image, no matter GL_LINEAR or
GL_NEAREST, because the size of source video frame (200x200) is same to target
size on the screen.
However, yuv_stripes.png is much blurrer than yuv_stripes_nearest.png, while
yuv_stripes_rgba.png is as sharp as yuv_stripes_nearest.png 
gl_renderer obviously has bug.

It's because the size of Y plane is different to the size of U plane and
built-in GPU bilinear filter cannot handle it.
U plane has twice sampling area to Y plane, and it causes blurring effect.

Look at the example in the description.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

'·' represents sampling point of the current fragment shader. '·' on Y texture
locates the middle of (3,3) texel. so it looks up almost only (3,3) texel.
However, '·' on U texture locates the upper left conner of (2,2) texel. built-in
bilinear filter looks up 1/16 * ((0,0) texel) + 3/16 * ((1,0) texel + (0,1)
texel) + 9/16 * (1,1) texel.
It's why yuv_stripes.png has lots of bleeding edge.

Solution: look up U texture as same as Y texture.

[dshwang, 2016-10-10 11:16:26]

On 2016/10/10 11:08:58, dshwang wrote:
> On 2016/10/07 18:12:00, hubbe wrote:
> > On 2016/10/07 17:53:55, dshwang wrote:
> > > I found bluring bug of YUV quad drawing. It's why we see unnecessary
bluring
> > in
> > > https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> > > 
> > > hubbe@, could you review?
> > 
> > This is wrong. U and V planes *should* have a larger sampling area. Not
doing
> so
> > will lead to over-sharpening and ugly jaggies.
> 
> Nope, U and V planes *should* have a same sampling area to Y plane, because
they
> contribute on the same pixel.
> 
> The clear counter example is
> https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> In theory, the pixel test *must* produce same image, no matter GL_LINEAR or
> GL_NEAREST, because the size of source video frame (200x200) is same to target
> size on the screen.
> However, yuv_stripes.png is much blurrer than yuv_stripes_nearest.png, while
> yuv_stripes_rgba.png is as sharp as yuv_stripes_nearest.png 

For yuv_stripes_rgba.png, LIBYUV_I420_TO_ARGB convert YUV to RGBA, and
gl_renderer draws the RGBA texture.

> gl_renderer obviously has bug.
> 
> It's because the size of Y plane is different to the size of U plane and
> built-in GPU bilinear filter cannot handle it.
> U plane has twice sampling area to Y plane, and it causes blurring effect.
> 
> Look at the example in the description.
> 
> Y 4x4      U 2x2    U sampling by GL_LINEAR
> +-+-+-+-+  +--+--+  +--+--+
> | | | | |  |  |  |  |  |  |
> +-------+  |  |  |  | +--++
> | | | | |  +-----+  +-----|
> +-------+  |  |· |  | ||·||
> | | |·| |  |  |  |  | +--+|
> +-------+  +--+--+  +-----+
> | | | | |
> +-+-+-+-+
> 
> '·' represents sampling point of the current fragment shader. '·' on Y texture
> locates the middle of (3,3) texel. so it looks up almost only (3,3) texel.
> However, '·' on U texture locates the upper left conner of (2,2) texel.
built-in
> bilinear filter looks up 1/16 * ((0,0) texel) + 3/16 * ((1,0) texel + (0,1)
> texel) + 9/16 * (1,1) texel.
> It's why yuv_stripes.png has lots of bleeding edge.

I mixed start index with 0 and 1 together. sorry. let me correct them as 0 start
index. upper left texel is (0,0).

'·' represents sampling point of the current fragment shader. '·' on Y texture
locates the middle of (2,2) texel. so it looks up almost only (2,2) texel.
However, '·' on U texture locates the upper left conner of (1,1) texel. built-in
bilinear filter looks up 1/16 * ((0,0) texel) + 3/16 * ((1,0) texel + (0,1)
texel) + 9/16 * (1,1) texel.
It's why yuv_stripes.png has lots of bleeding edge.

> 
> Solution: look up U texture as same as Y texture.

[dshwang, 2016-10-10 11:17:06]

Description was changed from

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (3, 3) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (3, 3) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

to

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (2, 2) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (2, 2) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

[inactive_dshwang_plz_cc_intel, 2016-10-10 13:00:55]

The CQ bit was checked by dongseong.hwang@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-10 13:01:06]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-10 14:42:01]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-10 14:42:02]

Dry run: This issue passed the CQ dry run.

[hubbe, 2016-10-10 17:36:47]

On 2016/10/10 11:16:26, dshwang wrote:
> On 2016/10/10 11:08:58, dshwang wrote:
> > On 2016/10/07 18:12:00, hubbe wrote:
> > > On 2016/10/07 17:53:55, dshwang wrote:
> > > > I found bluring bug of YUV quad drawing. It's why we see unnecessary
> bluring
> > > in
> > > > https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> > > > 
> > > > hubbe@, could you review?
> > > 
> > > This is wrong. U and V planes *should* have a larger sampling area. Not
> doing
> > so
> > > will lead to over-sharpening and ugly jaggies.
> > 
> > Nope, U and V planes *should* have a same sampling area to Y plane, because
> they
> > contribute on the same pixel.
> > 
> > The clear counter example is
> > https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c36
> > In theory, the pixel test *must* produce same image, no matter GL_LINEAR or
> > GL_NEAREST, because the size of source video frame (200x200) is same to
target
> > size on the screen.
> > However, yuv_stripes.png is much blurrer than yuv_stripes_nearest.png, while
> > yuv_stripes_rgba.png is as sharp as yuv_stripes_nearest.png 
> 
> For yuv_stripes_rgba.png, LIBYUV_I420_TO_ARGB convert YUV to RGBA, and
> gl_renderer draws the RGBA texture.
> 
> > gl_renderer obviously has bug.
> > 
> > It's because the size of Y plane is different to the size of U plane and
> > built-in GPU bilinear filter cannot handle it.
> > U plane has twice sampling area to Y plane, and it causes blurring effect.
> > 
> > Look at the example in the description.
> > 
> > Y 4x4      U 2x2    U sampling by GL_LINEAR
> > +-+-+-+-+  +--+--+  +--+--+
> > | | | | |  |  |  |  |  |  |
> > +-------+  |  |  |  | +--++
> > | | | | |  +-----+  +-----|
> > +-------+  |  |· |  | ||·||
> > | | |·| |  |  |  |  | +--+|
> > +-------+  +--+--+  +-----+
> > | | | | |
> > +-+-+-+-+
> > 
> > '·' represents sampling point of the current fragment shader. '·' on Y
texture
> > locates the middle of (3,3) texel. so it looks up almost only (3,3) texel.
> > However, '·' on U texture locates the upper left conner of (2,2) texel.
> built-in
> > bilinear filter looks up 1/16 * ((0,0) texel) + 3/16 * ((1,0) texel + (0,1)
> > texel) + 9/16 * (1,1) texel.
> > It's why yuv_stripes.png has lots of bleeding edge.
> 
> I mixed start index with 0 and 1 together. sorry. let me correct them as 0
start
> index. upper left texel is (0,0).
> 
> '·' represents sampling point of the current fragment shader. '·' on Y texture
> locates the middle of (2,2) texel. so it looks up almost only (2,2) texel.
> However, '·' on U texture locates the upper left conner of (1,1) texel.
built-in
> bilinear filter looks up 1/16 * ((0,0) texel) + 3/16 * ((1,0) texel + (0,1)
> texel) + 9/16 * (1,1) texel.
> It's why yuv_stripes.png has lots of bleeding edge.
> 
> > 
> > Solution: look up U texture as same as Y texture.

Imagine that U/V were the only textures. Would you use this sampler to show it
on the screen?
Your shader is over-sharpening the texture. The software implementation does it
too, because that's easier
than doing a full linear interpolation. However, either way you do it, it's not
going to make a large difference
because it only affects the color domain, where most people don't notice it.

The software and the hardware renderer do in fact differ in how they interpret
the YUV data, but the hardware renderer is more correct in my opinion.

[dshwang, 2016-10-10 18:03:41]

On 2016/10/10 17:36:47, hubbe wrote:
> Imagine that U/V were the only textures. Would you use this sampler to show it
> on the screen?

Yes, I'll use this sampler. in I420 case, one U pixel represent 4 real pixels.
If sampling U plane by built-in biliner filter, one real pixel is sampled by 16
virtual pixels (i.e. 4 pixels in U plane)

> Your shader is over-sharpening the texture. The software implementation does
it
> too, because that's easier
> than doing a full linear interpolation. 

Software implementation is not overshapend. It's just correct.
Software implementation convert YUV to RGBA. Both contain correct image
information.
Sampling is applied when RGBA texture is drawn by gl_renderer. gl_renderer use
GL_LINEAR so correctly samples them for screen.
There is not any point in which oversharpness happens.

My sampling is same to what software implementationg is doing, while it merges
conversion and drawing together.

> However, either way you do it, it's not
> going to make a large difference
> because it only affects the color domain, where most people don't notice it.

It is noticible enough. Compare
https://codereview.chromium.org/2400033004/diff/20001/cc/test/data/yuv_stripe...
NOTE: the new one is not oversharpened. It shows what is original video frame.
Keep in mind the pixel test draws 200x200 video frame on 200x200 target rect.
If the pixel test draws 200x200 video frame on 300x300 target rect, my sampler
and software implementation make sample very correctly. It is what
yuv_stripes_clipped.png shows
https://codereview.chromium.org/2400033004/diff/20001/cc/test/data/yuv_stripe...

> The software and the hardware renderer do in fact differ in how they interpret
> the YUV data, but the hardware renderer is more correct in my opinion.

Current hardware renderer produce very blurring frame. Software shows more
correct image.

[hubbe, 2016-10-10 18:21:40]

On 2016/10/10 18:03:41, dshwang wrote:
> On 2016/10/10 17:36:47, hubbe wrote:
> > Imagine that U/V were the only textures. Would you use this sampler to show
it
> > on the screen?
> 
> Yes, I'll use this sampler. in I420 case, one U pixel represent 4 real pixels.
> If sampling U plane by built-in biliner filter, one real pixel is sampled by
16
> virtual pixels (i.e. 4 pixels in U plane)
> 

It is irrelevant how many pixels something represents. Regardless how many
pixels something
represents, when you upscale it you use the same method. Bilinear is not a great
method for
upscaling, but it's better than just duplicating the values.

> > Your shader is over-sharpening the texture. The software implementation does
> it
> > too, because that's easier
> > than doing a full linear interpolation. 
> 
> Software implementation is not overshapend. It's just correct.

By what definition is it "correct"?
In fact, in the world of image scaling, there is no such thing as a "correct"
algorithm.
The only algorithm that has any mathematical basis is the "sinc" resampler,
which doesn't look very good in practice.

> Software implementation convert YUV to RGBA. Both contain correct image
> information.
> Sampling is applied when RGBA texture is drawn by gl_renderer. gl_renderer use
> GL_LINEAR so correctly samples them for screen.
> There is not any point in which oversharpness happens.
> 

The software algorithm duplicates the U/V values four times, this over-sharpens
the image.

> My sampling is same to what software implementationg is doing, while it merges
> conversion and drawing together.
> 

Agreed.

> > However, either way you do it, it's not
> > going to make a large difference
> > because it only affects the color domain, where most people don't notice it.
> 
> It is noticible enough. Compare
>
https://codereview.chromium.org/2400033004/diff/20001/cc/test/data/yuv_stripe...

This is not a real image and is not appropriate for quality assessment.

> NOTE: the new one is not oversharpened. It shows what is original video frame.

The original RGB data doesn't exist, so how can you possibly know that the new
RGB
data matches the original?

> Keep in mind the pixel test draws 200x200 video frame on 200x200 target rect.
> If the pixel test draws 200x200 video frame on 300x300 target rect, my sampler
> and software implementation make sample very correctly. It is what
> yuv_stripes_clipped.png shows
>
https://codereview.chromium.org/2400033004/diff/20001/cc/test/data/yuv_stripe...
> 
> > The software and the hardware renderer do in fact differ in how they
interpret
> > the YUV data, but the hardware renderer is more correct in my opinion.
> 
> Current hardware renderer produce very blurring frame. Software shows more
> correct image.

IF your algorithm is correct (which it is not in my opinion) then we would still
have to
figure out what the cost is in terms of power and performance and then decide if
it would
be worth it or not.

[dshwang, 2016-10-10 18:52:58]

On 2016/10/10 18:21:40, hubbe wrote:
> On 2016/10/10 18:03:41, dshwang wrote:
> > On 2016/10/10 17:36:47, hubbe wrote:
> > > Your shader is over-sharpening the texture. The software implementation
does
> > it
> > > too, because that's easier
> > > than doing a full linear interpolation. 
> > 
> > Software implementation is not overshapend. It's just correct.
> 
> By what definition is it "correct"?
> In fact, in the world of image scaling, there is no such thing as a "correct"
> algorithm.
> The only algorithm that has any mathematical basis is the "sinc" resampler,
> which doesn't look very good in practice.

I'm not taking about image scaling. I'm talking about how I420 should be
rendered.
each Y pixel maps to each pixel in display, while each U pixel maps to 4 pixels
in display.
Imagine the implementation draws the pixel corresponding 'a' in following
figure.
The pixel should be calculated using 'a' and 'e'. But gpu bilinear mixes U
values with 'e' pixel, upper pixel to 'e', lefter pixel to 'e', upper-left pixel
to 'e'.

Y plane   U plane
+---+     +---+
|a|b|     |   |
+---+     | e |
|c|d|     |   |
+---+     +---+


> > Software implementation convert YUV to RGBA. Both contain correct image
> > information.
> > Sampling is applied when RGBA texture is drawn by gl_renderer. gl_renderer
use
> > GL_LINEAR so correctly samples them for screen.
> > There is not any point in which oversharpness happens.
> > 
> 
> The software algorithm duplicates the U/V values four times, this
over-sharpens
> the image.

That's what I420 intends. Most implementation renders it in the way.
I think it's not good for chromium to render it by its own (ad-hoc) way. rather
over-sharpen image is better than inconsistency. 
Web developers should use YUV444 or their own scaling algorithm by webgl if they
don't like over-sharpen behavior.

> > My sampling is same to what software implementationg is doing, while it
merges
> > conversion and drawing together.
> > 
> 
> Agreed.
> 
> > Current hardware renderer produce very blurring frame. Software shows more
> > correct image.
> 
> IF your algorithm is correct (which it is not in my opinion) then we would
still
> have to
> figure out what the cost is in terms of power and performance and then decide
if
> it would
> be worth it or not.

I think there is not nondeterministic code with regards to power consumption. It
will not add more overhead than |use_color_lut_| path.
I'll test before and after using chromebook and report it.

[hubbe, 2016-10-10 19:26:03]

On 2016/10/10 18:52:58, dshwang wrote:
> On 2016/10/10 18:21:40, hubbe wrote:
> > On 2016/10/10 18:03:41, dshwang wrote:
> > > On 2016/10/10 17:36:47, hubbe wrote:
> > > > Your shader is over-sharpening the texture. The software implementation
> does
> > > it
> > > > too, because that's easier
> > > > than doing a full linear interpolation. 
> > > 
> > > Software implementation is not overshapend. It's just correct.
> > 
> > By what definition is it "correct"?
> > In fact, in the world of image scaling, there is no such thing as a
"correct"
> > algorithm.
> > The only algorithm that has any mathematical basis is the "sinc" resampler,
> > which doesn't look very good in practice.
> 
> I'm not taking about image scaling. I'm talking about how I420 should be
> rendered.
> each Y pixel maps to each pixel in display, while each U pixel maps to 4
pixels
> in display.
> Imagine the implementation draws the pixel corresponding 'a' in following
> figure.
> The pixel should be calculated using 'a' and 'e'. But gpu bilinear mixes U
> values with 'e' pixel, upper pixel to 'e', lefter pixel to 'e', upper-left
pixel
> to 'e'.
> 
> Y plane   U plane
> +---+     +---+
> |a|b|     |   |
> +---+     | e |
> |c|d|     |   |
> +---+     +---+
> 
> 
> > > Software implementation convert YUV to RGBA. Both contain correct image
> > > information.
> > > Sampling is applied when RGBA texture is drawn by gl_renderer. gl_renderer
> use
> > > GL_LINEAR so correctly samples them for screen.
> > > There is not any point in which oversharpness happens.
> > > 
> > 
> > The software algorithm duplicates the U/V values four times, this
> over-sharpens
> > the image.
> 
> That's what I420 intends. Most implementation renders it in the way.
> I think it's not good for chromium to render it by its own (ad-hoc) way.
rather
> over-sharpen image is better than inconsistency. 
> Web developers should use YUV444 or their own scaling algorithm by webgl if
they
> don't like over-sharpen behavior.
> 
> > > My sampling is same to what software implementationg is doing, while it
> merges
> > > conversion and drawing together.
> > > 
> > 
> > Agreed.
> > 
> > > Current hardware renderer produce very blurring frame. Software shows more
> > > correct image.
> > 
> > IF your algorithm is correct (which it is not in my opinion) then we would
> still
> > have to
> > figure out what the cost is in terms of power and performance and then
decide
> if
> > it would
> > be worth it or not.
> 
> I think there is not nondeterministic code with regards to power consumption.
It
> will not add more overhead than |use_color_lut_| path.
> I'll test before and after using chromebook and report it.

If you want me to approve this change, you would need to do three things:

1) find a standard that specifies how yuv data is supposed to be interpreted,
saying "this is how other people does it" is not good enough.
2) Show that your way of rendering actually looks better on real-world videos.
(This can be difficult, as it is very subjective, and moving and still images
are perceived differently. I suspect that over-sharpening may look pretty when
paused, but cause annoyances when you press play.)
3) Show that this doesn't affect performance or power too much. (The right way
to do this is to check it in behind a flag, and run it through our test lab with
and without the flag and compare the numbers. However, you would need to do 1
and 2 above first.)

   /Hubbe

PS: I've done enough work on images scaling to know what I think works well and
what doesn't work well. I might be wrong of course, but it will take a fair
amount of work to convince me.

[fbarchard1, 2016-10-28 23:27:29]

Some general comments:
Subsampling is codec specific: mpeg2 is different than h264 etc.
Subsampling and scaling may want different methods.  May want to upsample the
chroma to a temporary buffer using a codec/subsampling specific method, then
render with a high quality resampler.
libyuv YUV420 to RGB is point sampled on chroma.

[dshwang, 2016-11-02 15:02:22]

Description was changed from

==========
cc: custome bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (2, 2) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (2, 2) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========

to

==========
cc: custom bilinear filtering for YUV quad.

YUV quad needs a custom bilinear filter because U plane can be smaller than Y
plane.
For example, look at the following figures. When the current fragment samples
the middle point of (2, 2) texel of Y texture, it points out upper left of (1,
1) texel of U texture. The default GL_LINEAR samples 4 texels of U plane, while
it samples only (2, 2) texel of Y plane.  TextureLookupBilinear() samples U
plane with Y plane fraction to sample like Y plane.

Y 4x4      U 2x2    U sampling by GL_LINEAR
+-+-+-+-+  +--+--+  +--+--+
| | | | |  |  |  |  |  |  |
+-------+  |  |  |  | +--++
| | | | |  +-----+  +-----|
+-------+  |  |· |  | ||·||
| | |·| |  |  |  |  | +--+|
+-------+  +--+--+  +-----+
| | | | |
+-+-+-+-+

BUG=445071, 622133, 660368
TEST=existing cc_unittests.
CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel
==========