Mac Overlays: Plumb through GL solid color overlay support

Mac Overlays: Plumb through GL solid color overlay support


BUG=533677

[ccameron, 2015-10-08 23:04:29]

ccameron@chromium.org changed reviewers:
+ alexst@chromium.org, reveman@chromium.org

[ccameron, 2015-10-08 23:04:30]

This is the extra function that we discussed in the meeting earlier. Lmk if this
seems reasonable to you.

https://codereview.chromium.org/1394403002/diff/1/gpu/command_buffer/build_gl...
File gpu/command_buffer/build_gles2_cmd_buffer.py (right):

https://codereview.chromium.org/1394403002/diff/1/gpu/command_buffer/build_gl...
gpu/command_buffer/build_gles2_cmd_buffer.py:4136: 'client_test': False,
This is still using the CHROMIUM_schedule_overlay_plane extension. Should we
consider a different extension? I'd be inclined to say "not yet".

[reveman, 2015-10-09 14:17:56]

Seems reasonable as short term solution if it's critical to making progress but
I think we need a long term plan before we can land this. I don't think we
should implement a compositor API using CHROMIUM_schedule_overlay_plane or
through the command buffer in general.

Is it possible to handle solid color ca layers in the browser but io surface
(glimage) ones on the gpu process side? or does everything need to be done on
the same side?

If solid color quads are preventing us from having all content be ca layers,
maybe we can instead disable solid color tile detection as a short term
solution?

[ccameron, 2015-10-09 16:44:02]

On 2015/10/09 14:17:56, reveman wrote:
> Seems reasonable as short term solution if it's critical to making progress
but
> I think we need a long term plan before we can land this. I don't think we
> should implement a compositor API using CHROMIUM_schedule_overlay_plane or
> through the command buffer in general.
> 
> Is it possible to handle solid color ca layers in the browser but io surface
> (glimage) ones on the gpu process side? or does everything need to be done on
> the same side?

Not really possible -- it does need to be done all-in-the-same-side.

> If solid color quads are preventing us from having all content be ca layers,
> maybe we can instead disable solid color tile detection as a short term
> solution?

That will probably lose us lots of the power that we gain from this approach
(I've noticed that Safari is missing such detection, and appears to suffer
because of it).

The alternative approach is to plumb the GMB handle through to the browser
process already. It will also require that, for hardware video resources, where
the IOSurface is created in the bowels of the decoder code, we register that
IOSurface as a GMB. Would you prefer that I attempt this before going forward
with the "through the command buffer" approach?

[reveman, 2015-10-09 17:46:46]

On 2015/10/09 at 16:44:02, ccameron wrote:
> On 2015/10/09 14:17:56, reveman wrote:
> > Seems reasonable as short term solution if it's critical to making progress
but
> > I think we need a long term plan before we can land this. I don't think we
> > should implement a compositor API using CHROMIUM_schedule_overlay_plane or
> > through the command buffer in general.
> > 
> > Is it possible to handle solid color ca layers in the browser but io surface
> > (glimage) ones on the gpu process side? or does everything need to be done
on
> > the same side?
> 
> Not really possible -- it does need to be done all-in-the-same-side.
> 
> > If solid color quads are preventing us from having all content be ca layers,
> > maybe we can instead disable solid color tile detection as a short term
> > solution?
> 
> That will probably lose us lots of the power that we gain from this approach
(I've noticed that Safari is missing such detection, and appears to suffer
because of it).
> 
> The alternative approach is to plumb the GMB handle through to the browser
process already. It will also require that, for hardware video resources, where
the IOSurface is created in the bowels of the decoder code, we register that
IOSurface as a GMB. Would you prefer that I attempt this before going forward
with the "through the command buffer" approach?

To me, what we're trying to do on Mac seems more like having the browser
compositor be a "delegating" compositor than a compositor with a gl renderer.
The parent compositor in this case is the ca layer system. So in that regard,
producing all ca layers on the browser side makes more sense to me. Adding
ScheduleXYZOverlayPlane functions for everything that we might want the ca layer
system to handle doesn't seem appropriate. 

I just like to know what the plan is so I know if this patch is a temporary
solution or something we're going to use long term and add more of.

FYI, the CreateGpuMemoryBufferFromHandle API
(https://docs.google.com/a/google.com/document/d/1AKDYwF6b9eOhnQszd4ipCaBebqi8...)
I'm adding is exactly what we need for importing IOSurfaces created by a video
decoder into chromium's GMB system. We probably want to use that for quads too
if we're going down the non-ScheduleXYZOverlayPlane path as that allows us to
explicit create IOSurface backed resources in the child compositor.

[ccameron, 2015-10-09 19:12:40]

So, we have two issues going on.

First issue: When we created the "surfaceless" output surfaces (the ones that
draw to a backbuffer that is displayed using ScheduleOverlayPlane, where
"SwapBuffers" just means "draw the previously scheduled back buffer plane"), we
could have gone further, and moved the display of those planes to the browser
process.

Second issue: The mac CALayer path is much more flexible than the hardware
overlay path (basically amounting to a "sometimes" compositor), and running both
of them through the "schedule overlay planes" API is something of an abuse of
the concept of overlay planes. I am curious about the flexibility of the "real"
hardware overlays API, and if perhaps implementing a compositor using that API
would be something other OSes would want to do.

On 2015/10/09 17:46:46, reveman wrote:
> To me, what we're trying to do on Mac seems more like having the browser
> compositor be a "delegating" compositor than a compositor with a gl renderer.

For quite some time to come (6-9 months, at least), we'll still need to be able
to fall back to using the GLRenderer, because there are lots of edge cases that
we don't yet handle. When we can't handle something, the GLRenderer will draw to
its backbuffer, and we'll have to display that backbuffer.

> I just like to know what the plan is so I know if this patch is a temporary
> solution or something we're going to use long term and add more of.
>
> FYI, the CreateGpuMemoryBufferFromHandle API
>
(https://docs.google.com/a/google.com/document/d/1AKDYwF6b9eOhnQszd4ipCaBebqi8...)
> I'm adding is exactly what we need for importing IOSurfaces created by a video
> decoder into chromium's GMB system. We probably want to use that for quads too
> if we're going down the non-ScheduleXYZOverlayPlane path as that allows us to
> explicit create IOSurface backed resources in the child compositor.

I feel that we're not ready to take on the "first issue". For the moment, I find
it much easier to reason about everything when it goes through the command
buffer. I'd very strongly prefer to keep everything going through the command
buffer for now. If we want to take on the "first issue", we should do it in a
way that makes sense on CrOS, Android, and possibly Windows as well. I don't
want that to be holding up the large-and-immediately-available power gains on
Mac.

WRT the second issue ... what are your thoughts on having us just fork the
ScheduleOverlayPlane path, and create a "ScheduleNativeLayer" path, which does
basically the same thing, but without pretending to actually map to hardware
overlays? This is actually fairly important, because we're running into issues
with sharing the code to create overlays -- the existing pathway wants to, for
example, reject things with premultiplied alpha (while we don't). Also we're
going to want to enable promotion of textures and tiles, but only for this
"CALayer path" -- we don't want CrOS to start promoting the first tile that it
sees just because it can (we should keep it focused on video, etc).

So, my vote would be to ditch this patch, and change it to something that does a
ScheduleNativeLayer( ... ), which specifies things that aren't allowed by
overlay planes, such as
- opacity
- background color
- floating-point bounds
- etc
And, at the same time, create a "NativeLayer" fork of the OverlayPlane stuff,
which will, in effect, be the "all or nothing" strategy that we've been doing.

[reveman, 2015-10-13 00:20:22]

On 2015/10/09 at 19:12:40, ccameron wrote:
> So, we have two issues going on.
> 
> First issue: When we created the "surfaceless" output surfaces (the ones that
draw to a backbuffer that is displayed using ScheduleOverlayPlane, where
"SwapBuffers" just means "draw the previously scheduled back buffer plane"), we
could have gone further, and moved the display of those planes to the browser
process.
> 
> Second issue: The mac CALayer path is much more flexible than the hardware
overlay path (basically amounting to a "sometimes" compositor), and running both
of them through the "schedule overlay planes" API is something of an abuse of
the concept of overlay planes. I am curious about the flexibility of the "real"
hardware overlays API, and if perhaps implementing a compositor using that API
would be something other OSes would want to do.
> 
> On 2015/10/09 17:46:46, reveman wrote:
> > To me, what we're trying to do on Mac seems more like having the browser
> > compositor be a "delegating" compositor than a compositor with a gl
renderer.
> 
> For quite some time to come (6-9 months, at least), we'll still need to be
able to fall back to using the GLRenderer, because there are lots of edge cases
that we don't yet handle. When we can't handle something, the GLRenderer will
draw to its backbuffer, and we'll have to display that backbuffer.
> 
> > I just like to know what the plan is so I know if this patch is a temporary
> > solution or something we're going to use long term and add more of.
> >
> > FYI, the CreateGpuMemoryBufferFromHandle API
> >
(https://docs.google.com/a/google.com/document/d/1AKDYwF6b9eOhnQszd4ipCaBebqi8...)
> > I'm adding is exactly what we need for importing IOSurfaces created by a
video
> > decoder into chromium's GMB system. We probably want to use that for quads
too
> > if we're going down the non-ScheduleXYZOverlayPlane path as that allows us
to
> > explicit create IOSurface backed resources in the child compositor.
> 
> I feel that we're not ready to take on the "first issue". For the moment, I
find it much easier to reason about everything when it goes through the command
buffer. I'd very strongly prefer to keep everything going through the command
buffer for now. If we want to take on the "first issue", we should do it in a
way that makes sense on CrOS, Android, and possibly Windows as well. I don't
want that to be holding up the large-and-immediately-available power gains on
Mac.

Makes sense. FYI, Wayland output on Linux would be a good example of where we
would benefit from the same type of mechanism on a different platform.

> 
> WRT the second issue ... what are your thoughts on having us just fork the
ScheduleOverlayPlane path, and create a "ScheduleNativeLayer" path, which does
basically the same thing, but without pretending to actually map to hardware
overlays? This is actually fairly important, because we're running into issues
with sharing the code to create overlays -- the existing pathway wants to, for
example, reject things with premultiplied alpha (while we don't). Also we're
going to want to enable promotion of textures and tiles, but only for this
"CALayer path" -- we don't want CrOS to start promoting the first tile that it
sees just because it can (we should keep it focused on video, etc).
> 
> So, my vote would be to ditch this patch, and change it to something that does
a ScheduleNativeLayer( ... ), which specifies things that aren't allowed by
overlay planes, such as
> - opacity
> - background color
> - floating-point bounds
> - etc
> And, at the same time, create a "NativeLayer" fork of the OverlayPlane stuff,
which will, in effect, be the "all or nothing" strategy that we've been doing.

Sgtm. Maybe even make it MacOSX specific ScheduleNativeCALayer.