Prototype HTMLVideoElement properties for WebGL texImage2D

third_party/WebKit/Source/modules/webgl/WebGLRenderingContextBase.cpp

 /*
  * Copyright (C) 2009 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
  * are met:
  * 1. Redistributions of source code must retain the above copyright
  *    notice, this list of conditions and the following disclaimer.
  * 2. Redistributions in binary form must reproduce the above copyright
  *    notice, this list of conditions and the following disclaimer in the
@@ skipping 5267 matching lines @@
     // to system memory if possible.  Otherwise, it will fall back to the normal
     // SW path.
 
     // Note that neither
     // HTMLVideoElement::copyVideoTextureToPlatformTexture nor
     // ImageBuffer::copyToPlatformTexture allocate the destination texture
     // any more.
     texImage2DBase(target, level, internalformat, video->videoWidth(),
                    video->videoHeight(), 0, format, type, nullptr);
 
     if (video->copyVideoTextureToPlatformTexture(contextGL(), texture->object(),

[DaleCurtis, 2017/03/17 21:47:52]

Sorry, can you instead add the parameters you want to get to this call? I.e.
copyVideoTextureToPlatformTexture() should now take pointers for width, height,
and timestamp. We want to avoid any method that is not this call from providing
parameters.

[Kai Ninomiya, 2017/03/18 01:08:58]

Hm, adding it to copyVideoTextureToPlatformTexture and texImageImpl wouldn't be
hard (each only has one caller). But I may be missing something here: I think it
would have to also be added to paintCurrentFrame for the other paths, which has
6 callers and would be a little more intrusive than I want for a prototype.

Perhaps this could be implemented only for those 2 upload paths, but that might
not be reliable enough for YouTube's testing. I'm not sure what formats take
those paths on what platforms.

[DaleCurtis, 2017/03/20 17:58:13]

Is there any other way to upload a frame? Also 6 callers isn't that much; plus
if you really want to Blink allows default values, so you can just set the new
parameters as nullptr for optional extraction.

[Ken Russell (switch to Gerrit), 2017/03/20 19:14:14]

Dale, there are three different entry points on HTMLVideoElement used to upload
the current frame to WebGL (as well as for other purposes) -- paintCurrentFrame,
copyVideoTextureToPlatformTexture, and texImageImpl.

Since this is known to be a prototype I think that we should minimize the impact
to the code base, and agree with Kai's decision in this area.

If we are going to plumb this query out to all of the call sites then I think a
struct containing this info should be defined somewhere (presumably
src/third_party/WebKit/public/platform/WebMediaPlayer.h) and we should let the
pointer to that be null, rather than passing three pointers to all of these
APIs.

[DaleCurtis, 2017/03/20 21:59:54]

Yes, this all sounds reasonable to me for the experiment. If we decide to launch
these properties out of experimental the API should change in the way you
suggest (details passed along in struct form at time of upload).

                                                  m_unpackPremultiplyAlpha,
                                                  m_unpackFlipY)) {
+      texture->videoUpdateLastUploaded(video->webMediaPlayer());
       return;
     }
 
     // Try using an accelerated image buffer, this allows YUV conversion to be
     // done on the GPU.
     std::unique_ptr<ImageBufferSurface> surface =
         WTF::wrapUnique(new AcceleratedImageBufferSurface(
             IntSize(video->videoWidth(), video->videoHeight())));
     if (surface->isValid()) {
       std::unique_ptr<ImageBuffer> imageBuffer(
           ImageBuffer::create(std::move(surface)));
       if (imageBuffer) {
         // The video element paints an RGBA frame into our surface here. By
         // using an AcceleratedImageBufferSurface, we enable the WebMediaPlayer
         // implementation to do any necessary color space conversion on the GPU
         // (though it may still do a CPU conversion and upload the results).
         video->paintCurrentFrame(
             imageBuffer->canvas(),
             IntRect(0, 0, video->videoWidth(), video->videoHeight()), nullptr);
 
         // This is a straight GPU-GPU copy, any necessary color space conversion
         // was handled in the paintCurrentFrameInContext() call.
 
         if (imageBuffer->copyToPlatformTexture(
                 functionIDToSnapshotReason(functionID), contextGL(),
                 texture->object(), internalformat, type, level,
                 m_unpackPremultiplyAlpha, m_unpackFlipY, IntPoint(0, 0),
                 IntRect(0, 0, video->videoWidth(), video->videoHeight()))) {
+          texture->videoUpdateLastUploaded(video->webMediaPlayer());
           return;
         }
       }
     }
   }
 
   if (sourceImageRectIsDefault) {
     // Try using optimized CPU-GPU path for some formats: e.g. Y16 and Y8. It
     // leaves early for other formats or if frame is stored on GPU.
     ScopedUnpackParametersResetRestore(
         this, m_unpackFlipY || m_unpackPremultiplyAlpha);
     if (video->texImageImpl(
             static_cast<WebMediaPlayer::TexImageFunctionID>(functionID), target,
             contextGL(), level, convertTexInternalFormat(internalformat, type),
             format, type, xoffset, yoffset, zoffset, m_unpackFlipY,
             m_unpackPremultiplyAlpha &&
-                m_unpackColorspaceConversion == GL_NONE))
+                m_unpackColorspaceConversion == GL_NONE)) {
+      texture->videoUpdateLastUploaded(video->webMediaPlayer());
       return;
+    }
   }
 
   RefPtr<Image> image = videoFrameToImage(video);
   if (!image)
     return;
   texImageImpl(functionID, target, level, internalformat, xoffset, yoffset,
                zoffset, format, type, image.get(),
                WebGLImageConversion::HtmlDomVideo, m_unpackFlipY,
                m_unpackPremultiplyAlpha, sourceImageRect, depth,
                unpackImageHeight);
+  texture->videoUpdateLastUploaded(video->webMediaPlayer());
 }
 
 void WebGLRenderingContextBase::texImageBitmapByGPU(ImageBitmap* bitmap,
                                                     GLuint targetTexture,
                                                     GLenum targetInternalformat,
                                                     GLenum targetType,
                                                     GLint targetLevel,
                                                     bool flipY) {
   bitmap->bitmapImage()->copyToTexture(drawingBuffer()->contextProvider(),
                                        targetTexture, targetInternalformat,
@@ skipping 2483 matching lines @@
 
 void WebGLRenderingContextBase::getHTMLOrOffscreenCanvas(
     HTMLCanvasElementOrOffscreenCanvas& result) const {
   if (canvas())
     result.setHTMLCanvasElement(canvas());
   else
     result.setOffscreenCanvas(offscreenCanvas());
 }
 
 }  // namespace blink