Make the video layer contents visible in OverlayFullscreenVideo mode

Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521
Committed: https://crrev.com/e779e5a0f7e8bab98dab472cdbb29a850294a34b
Cr-Commit-Position: refs/heads/master@{#391728}

[watk, 2016-04-21 00:38:30]

watk@chromium.org changed reviewers:
+ qinmin@chromium.org, trchen@chromium.org

[watk, 2016-04-21 00:38:30]

PTAL. I still have to get the layout tests passing. Three of them now fail
because you can see the video whereas before you couldn't.

https://codereview.chromium.org/1907753002/diff/1/third_party/WebKit/Source/c...
File
third_party/WebKit/Source/core/layout/compositing/CompositedLayerMapping.cpp
(right):

https://codereview.chromium.org/1907753002/diff/1/third_party/WebKit/Source/c...
third_party/WebKit/Source/core/layout/compositing/CompositedLayerMapping.cpp:45:
#include "core/layout/LayoutVideo.h"
TODO remove this and HTMLME if possible

[qinmin, 2016-04-21 16:33:11]

https://codereview.chromium.org/1907753002/diff/1/content/renderer/media/andr...
File content/renderer/media/android/webmediaplayer_android.h (right):

https://codereview.chromium.org/1907753002/diff/1/content/renderer/media/andr...
content/renderer/media/android/webmediaplayer_android.h:351:
scoped_refptr<media::VideoFrame> fullscreen_frame_;
why cannot we just use current_frame_?

[watk, 2016-04-21 18:56:28]

https://codereview.chromium.org/1907753002/diff/1/content/renderer/media/andr...
File content/renderer/media/android/webmediaplayer_android.h (right):

https://codereview.chromium.org/1907753002/diff/1/content/renderer/media/andr...
content/renderer/media/android/webmediaplayer_android.h:351:
scoped_refptr<media::VideoFrame> fullscreen_frame_;
On 2016/04/21 16:33:11, qinmin wrote:
> why cannot we just use current_frame_?

Well, it's a strange case. VideoResourceUpdater, which is used by the
compositor, asserts that each new VideoFrame it see have unique timestamps in
debug mode.. So that was DCHECKing when I created a new fullscreen frame each
time because they all had timestamp 0. So we could either:

a) change VideoResourceUpdater to not require that. Larger change and not clear
how to keep the useful behavior of catching bugs for normal playbacks where the
timestamps should be unique.
b) give fullscreen frames a dummy timestamp. Ugly and weird.
c) only ever create one fullscreen frame. Seemed like a reasonable thing to do
anyway as a micro optimization and it should use very 
little memory because it's a 1x1 image.

[watk, 2016-04-25 20:00:28]

Friendly ping. trchen@ is there any particular blink owner you would suggest for
this?

[watk, 2016-04-25 20:01:07]

Patchset #2 (id:20001) has been deleted

[watk, 2016-04-26 22:51:51]

ping

[watk, 2016-04-26 22:57:59]

watk@chromium.org changed reviewers:
+ chrishtr@chromium.org
- trchen@chromium.org

[watk, 2016-04-26 22:58:00]

+chrishtr: PTAL at CompositedLayerMapping

[qinmin, 2016-04-26 23:38:22]

webmediaplayer_android change lgtm

[watk, 2016-04-28 17:55:37]

Friendly ping chrishtr

[watk, 2016-04-28 17:55:39]

Friendly ping chrishtr

[watk, 2016-04-29 23:39:58]

ping again

[chrishtr, 2016-05-03 16:31:04]

Why do you only need to change WebMediaPlayerAndroid? The CL description seems
to imply
the opposite. i.e. the comment "for WebMediaPlayerAndroid there is no
difference", but
seems to indicate some change for WebMediaPlayerImpl (which is I guess for all
other platforms?

I also don't quite get the point of the CL description parts that talk about why
WebMediaPlayerImpl needs composited frames. Can you explain more what you are
getting at?

https://codereview.chromium.org/1907753002/diff/60001/content/renderer/media/...
File content/renderer/media/android/webmediaplayer_android.cc (right):

https://codereview.chromium.org/1907753002/diff/60001/content/renderer/media/...
content/renderer/media/android/webmediaplayer_android.cc:1646: // as well, the
whole view will be transparent and we will see through to the
By "whole view" you mean the LayerTreeView, right? i.e. this code?

https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit...

[watk, 2016-05-03 19:27:20]

message: On 2016/05/03 16:31:04, chrishtr wrote:
> Why do you only need to change WebMediaPlayerAndroid? The CL description seems
> to imply
> the opposite. i.e. the comment "for WebMediaPlayerAndroid there is no
> difference", but
> seems to indicate some change for WebMediaPlayerImpl (which is I guess for all
> other platforms?
> 
> I also don't quite get the point of the CL description parts that talk about
why
> WebMediaPlayerImpl needs composited frames. Can you explain more what you are
> getting at?

I guess my description is taking a lot of context for granted :)

I'll explain here and try to reword the description a bit.

We're moving to using WebMediaPlayerImpl on Android. Instead of using this
overlayfullscreenvideo seeing with WMPI we are using overlay planes. So for
fullscreen video, everything works the same was as it does on desktop, except
that we mark video frames as "allow_overlay". The ui compositor schedules them
as overlay planes and we render them to the same SurfaceView underlay that we
use on Android today. The important difference between WMPI and WMPA here is
that with WMPI, video frames need to be scheduled as overlays by the compositor
in order to be displayed. WMPA was rendering to the underlay without touching
the cc::VideoLayer.

The unforeseen problem with this scheme is that now we are rendering the 
black background, whereas previously the background was the default background
of the underlay. We don't have any synchronization between orientation changes
and redrawing everything at the new orientation/size, so orientation changes 
have a black rectangle covering the video while the previous frame is
displayed at the new orientation.

So to fix this the plan is to start using OverlayFullscreenVideo with WMPI for
Android only. We don't have this orientation change jank on other platforms.

But before I can make that change we need the VideoLayer contents to be visible
so that video frames can be scheduled as overlay planes. Or if we're doing
software decoding, we need to render the frames with opengl the normal way
instead
of rendering them to the underlay. Software decoding didn't exist on Android
before the switch to WMPI.

The current CL lets us make the change to WMPI while maintaining the existing
behavior for WMPA.

>
https://codereview.chromium.org/1907753002/diff/60001/content/renderer/media/...
> File content/renderer/media/android/webmediaplayer_android.cc (right):
> 
>
https://codereview.chromium.org/1907753002/diff/60001/content/renderer/media/...
> content/renderer/media/android/webmediaplayer_android.cc:1646: // as well, the
> whole view will be transparent and we will see through to the
> By "whole view" you mean the LayerTreeView, right? i.e. this code?
> 
>
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit...

Yep, I'll write LayerTreeView explicitly.

[watk, 2016-05-03 19:50:43]

Description was changed from

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

This CL has no visible effect, but paves the way for a future change.

Previously, in OverlayFullscreenVideo mode the video GraphicsLayer
contents was set to invisible so that so that the underlay was visible
through it. With this change we leave the video layer contents visible,
but make the current frame a transparent VideoFrame in fullscreen. This
looks exactly the same as before, but WebMediaPlayer controls the
visibility of the video frame. For WebMediaPlayerAndroid there is no
difference; it always wants a transparent video frame in fullscreen. On
the other hand, WebMediaPlayerImpl needs its video frames to be
composited because,
  a) they may be software frames, in which case they won't be rendered
     to a hardware underlay and need to be composited normally.
  b) if they're hardware frames, they need to be scheduled as overlay
     planes by the ui compositor, which means they need to be composited
     by the renderer compositor.

BUG=603521
==========

to

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only 
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521
==========

[chrishtr, 2016-05-04 23:13:56]

lgtm

[watk, 2016-05-04 23:37:45]

The CQ bit was checked by watk@chromium.org

[watk, 2016-05-04 23:37:45]

The patchset sent to the CQ was uploaded after l-g-t-m from qinmin@chromium.org
Link to the patchset: https://codereview.chromium.org/1907753002/#ps100001
(title: "Clarified comment")

[commit-bot: I haz the power, 2016-05-04 23:38:04]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patch-status/1907753002/100001
View timeline at
 https://chromium-cq-status.appspot.com/patch-timeline/1907753002/100001

[commit-bot: I haz the power, 2016-05-05 02:02:37]

Description was changed from

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only 
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521
==========

to

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only 
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521
==========

[commit-bot: I haz the power, 2016-05-05 02:02:40]

Committed patchset #5 (id:100001)

[commit-bot: I haz the power, 2016-05-05 02:03:46]

Description was changed from

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only 
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521
==========

to

==========
Make the video layer contents visible in OverlayFullscreenVideo mode

Previously, in OverlayFullscreenVideo mode, the video GraphicsLayer contents
was set to invisible because the video frames were rendered directly to an
underlay without involving the cc::VideoLayer.

With this change the video layer contents is changed to visible to give
WebMediaPlayer implementations more control. Previously only
WebMediaPlayerAndroid needed to use OverlayFullscreenVideo mode, but a
future CL will enable it for WebMediaPlayerImpl too. WMPA and WMPI have
different requirements for OverlayFullscreenVideo, but they both use the
fact that the LayerTreeView background is made transparent.

WMPA will continue to make the whole LayerTreeView transparent, but it now
does so by setting the layer's current VideoFrame to a transparent frame.

WMPI will not make the video frame transparent, but will let it be
composited as usual. With WMPI we need to composite the frames because:
  a) In hardware underlay mode the ui compositor is responsible for timing
     the release of video frames to the underlay surface by scheduling them
     as overlay planes. So we need the ui compositor to see every new video
     frame.
  b) With WMPI we can have software decoded frames for which we currently
     have no way of rendering to the underlay surface. As a result,
     they will be rendered by the compositor to the main surface.

BUG=603521

Committed: https://crrev.com/e779e5a0f7e8bab98dab472cdbb29a850294a34b
Cr-Commit-Position: refs/heads/master@{#391728}
==========

[commit-bot: I haz the power, 2016-05-05 02:03:47]

Patchset 5 (id:??) landed as
https://crrev.com/e779e5a0f7e8bab98dab472cdbb29a850294a34b
Cr-Commit-Position: refs/heads/master@{#391728}