WebGL & 16-bit video stream: upload to FLOAT texture.

content/renderer/media/webmediaplayer_ms.cc

Index: content/renderer/media/webmediaplayer_ms.cc
diff --git a/content/renderer/media/webmediaplayer_ms.cc b/content/renderer/media/webmediaplayer_ms.cc
index 8d833ee750ef40b9b4dcdf54daab43e1138d00d9..f08de8f35b59dbd4cfda188ef6e58cce4f175caf 100644
--- a/content/renderer/media/webmediaplayer_ms.cc
+++ b/content/renderer/media/webmediaplayer_ms.cc
@@ -600,6 +600,41 @@ bool WebMediaPlayerMS::copyVideoTextureToPlatformTexture(
       premultiply_alpha, flip_y);
 }
 
+bool WebMediaPlayerMS::texImageImpl(TexImageFunctionID functionID,
+                                    unsigned target,
+                                    gpu::gles2::GLES2Interface* gl,
+                                    int level,
+                                    int internalformat,
+                                    unsigned format,
+                                    unsigned type,
+                                    int xoffset,
+                                    int yoffset,
+                                    int zoffset,
+                                    bool flip_y,
+                                    bool premultiply_alpha) {
+  TRACE_EVENT0("media", "WebMediaPlayerMS:texImageImpl");
+  DCHECK(thread_checker_.CalledOnValidThread());
+
+  const scoped_refptr<media::VideoFrame> video_frame =
+      compositor_->GetCurrentFrameWithoutUpdatingStatistics();
+  if (!video_frame || !video_frame->IsMappable() ||
+      video_frame->HasTextures() ||
+      media::VideoFrame::NumPlanes(video_frame->format()) != 1) {

[emircan, 2016/12/01 23:23:37]

Should you just check for PIXEL_FORMAT_Y16 format here? There are 9 formats that
have single plane.
https://cs.chromium.org/chromium/src/media/base/video_frame.cc?rcl=0&l=503

[aleksandar.stojiljkovic, 2016/12/02 20:23:41]

Done.
The idea is to use this to optimize for Y8 too, and maybe other. In follow up
patches, plan to add more code to SkCanvasVideoRenderer::Tex(Sub)Image2D/3D
methods. This code is already prototyped at crrev.com/2121043002/. So, the
format check is deferred to SkCanvasVideoRenderer to keep the check on one place
and avoid the need to change the code here. Anyway, your point is valid so using
Y16 check now as you suggested and will change it when adding more formats.  
 
Related to other single plane formats,... when no flip_y is supplied, this
approach saves at least one memcpy and a swizzle compared to fallback path at
the bottom of void WebGLRenderingContextBase::texImageHelperHTMLVideoElement.
So, it is potentially applicable for RGB based formats too.

[emircan, 2016/12/02 20:33:34]

Thanks for the explanation. For media streams, we usually receive texture-HW
decode- or I420 frames.

+    return false;
+  }
+
+  if (functionID == TexImage2D) {
+    return media::SkCanvasVideoRenderer::TexImage2D(
+        target, gl, video_frame.get(), level, internalformat, format, type,
+        flip_y, premultiply_alpha);
+  } else if (functionID == TexSubImage2D) {
+    return media::SkCanvasVideoRenderer::TexSubImage2D(
+        target, gl, video_frame.get(), level, format, type, xoffset, yoffset,
+        flip_y, premultiply_alpha);
+  }

[emircan, 2016/12/01 23:23:37]

nit: If you don't expect this function to be called with any other |functionID|,
you can add NOTREACHED() here to make that clear.

[aleksandar.stojiljkovic, 2016/12/02 20:23:41]

Other parameter values are expected too, we just don't handle teximage3D in this
patch yet) - adding it when the tests gets added. Anyway, as defined returning
false when call is not handled and no need for NOTREACHED.

+  return false;
+}
+
 void WebMediaPlayerMS::OnFirstFrameReceived(media::VideoRotation video_rotation,
                                             bool is_opaque) {
   DVLOG(1) << __func__;