Prime the landing pad for the new video rendering pipeline.

cc/layers/video_frame_provider.h

 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #ifndef CC_LAYERS_VIDEO_FRAME_PROVIDER_H_
 #define CC_LAYERS_VIDEO_FRAME_PROVIDER_H_
 
 #include "base/memory/ref_counted.h"
+#include "base/time/time.h"
 #include "cc/base/cc_export.h"
 
 namespace media {
 class VideoFrame;
 }
 
 namespace cc {
 
-// Threading notes: This class may be used in a multi threaded manner.
-// Specifically, the implementation may call GetCurrentFrame() or
-// PutCurrentFrame() from the compositor thread. If so, the caller is
-// responsible for making sure Client::DidReceiveFrame() and
-// Client::DidUpdateMatrix() are only called from this same thread.
+// VideoFrameProvider and VideoFrameProvider::Client define the relationship by
+// which video frames are exchanged between a provider and client.
+//
+// Threading notes: This class may be used in a multithreaded manner. However,
+// if the Client implementation calls GetCurrentFrame()/PutCurrentFrame() from
+// one thread, the provider must ensure that all client methods are called from
+// that thread (typically the compositor thread).
 class CC_EXPORT VideoFrameProvider {
  public:
-  virtual ~VideoFrameProvider() {}
-
   class CC_EXPORT Client {
    public:
-    // Provider will call this method to tell the client to stop using it.
+    // The provider will call this method to tell the client to stop using it.
     // StopUsingProvider() may be called from any thread. The client should
     // block until it has PutCurrentFrame() any outstanding frames.
     virtual void StopUsingProvider() = 0;
 
+    // Notifies the client that it should start or stop making regular Render()
+    // calls to the provider. No further calls to Render() will be made after
+    // StopRendering() returns. Clients should handle redundant calls.
+    virtual void StartRendering() = 0;
+    virtual void StopRendering() = 0;
+
     // Notifies the provider's client that a call to GetCurrentFrame() will
     // return new data.
+    // TODO(dalecurtis): Nuke this once VideoFrameProviderClientImpl is using a
+    // BeginFrameObserver based approach. http://crbug.com/336733
     virtual void DidReceiveFrame() = 0;
 
-    // Notifies the provider's client of a new UV transform matrix to be used.
+    // Notifies the client of a new UV transform matrix to be used.
     virtual void DidUpdateMatrix(const float* matrix) = 0;
 
    protected:
     virtual ~Client() {}
   };
 
   // May be called from any thread, but there must be some external guarantee
   // that the provider is not destroyed before this call returns.
   virtual void SetVideoFrameProviderClient(Client* client) = 0;
 
-  // This function places a lock on the current frame and returns a pointer to
-  // it. Calls to this method should always be followed with a call to
-  // PutCurrentFrame().
-  // Only the current provider client should call this function.
+  // Called by the client on a regular interval. Returns true if a new frame is

[xhwang, 2015/04/03 07:00:13]

nit: s/is/will be/ ?

[DaleCurtis, 2015/04/07 00:40:51]

Done.

+  // available via GetCurrentFrame() which should be displayed within the
+  // presentation interval [|deadline_min|, |deadline_max|].

[xhwang, 2015/04/03 07:00:13]

So I assume the "interval" will be the vsync interval? Is it possible that there
will be >1 frames available during this "interval"? In that case, should only
the newest frame be returned by GetCurrentFrame(), and all other frames be
dropped?

[DaleCurtis, 2015/04/03 17:14:52]

Yup, you can see what the mostly complete algorithm looks like here:
https://codereview.chromium.org/1021943002/ -- I still need to put it through
testing paces and resolve some TODOs, but the general strategy is finalized.

+  virtual bool Render(base::TimeTicks deadline_min,
+                      base::TimeTicks deadline_max) = 0;

[xhwang, 2015/04/03 07:00:13]

bikeshedding on naming: This really is something like UpdateCurrentFrame(). It
doesn't "Render" anything, and it doesn't even retrieve the current frame....

[DaleCurtis, 2015/04/03 17:14:52]

Yeah... I went back and forth on this: Either keep consistency with the sink
interface or with today's methods. Your comment has convinced me this should
likely be UpdateCurrentFrame, so I'll change it :)

[DaleCurtis, 2015/04/07 00:40:51]

Done.

+
+  // Returns the last frame that was rendered during Render(). A call to this
+  // method does not ensure that the frame will be rendered. A subsequent call
+  // to PutCurrentFrame() will be made if the frame is rendered.
   virtual scoped_refptr<media::VideoFrame> GetCurrentFrame() = 0;
 
-  // This function releases the lock on the video frame. It should always be
-  // called after GetCurrentFrame(). Frames passed into this method
-  // should no longer be referenced after the call is made. Only the current
-  // provider client should call this function.
-  virtual void PutCurrentFrame(
-      const scoped_refptr<media::VideoFrame>& frame) = 0;
+  // Indicates that the frame returned via GetCurrentFrame() was rendered. Must
+  // only occur after a previous call to GetCurrentFrame().
+  virtual void PutCurrentFrame() = 0;

[xhwang, 2015/04/03 07:00:13]

Here we have GetCurrentFrame/PutCurrentFrame. Then in the sink we have Render()
to return the frame. Is there any reason for the difference? Is it because
GetCurrentFrame() is also needed by
WebMediaPlayerImpl::GetCurrentFrameFromCompositor()?

https://code.google.com/p/chromium/codesearch#chromium/src/media/blink/webmed...

[DaleCurtis, 2015/04/03 17:14:52]

The compositor is what I'd call a two-phase process, it first "claims" the frame
via GetCurrentFrame() and then may or may not composite it later. If it does
composite the frame, PutCurrentFrame will be called.

The sink will receive an OnFrameDropped() call for new frames vended via
VRS::Render() which either don't get a GetCurrentFrame() or don't get a
PutCurrentFrame() before the next VRS::Render() call.

[xhwang, 2015/04/03 19:18:27]

Let me make sure I understand "dropped frame" count correctly, a frame is
dropped when:
1, A frame is already not the newest when Render() is called, or
2, A frame is promoted to "current_frame" in Render(), but GetCurrentFrame() is
not called before the next Render() call, or
3, A frame is returned by GetCurrentFrame(), but PutCurrentFrame() is not called
before the next Render() call.

Is this accurate? 

BTW, based on the comment, PutCurrentFrame() really should be something like
CurrentFrameRendered()...

[DaleCurtis, 2015/04/03 19:59:16]

Yes this is correct. I'd be okay with renaming it. WDYT Sunny?

+
+  // Called by the client when the visibility of the output layer changes. The
+  // client will not issue any Render() calls while the output layer has no
+  // visibility. It will not start any Render() calls before notifying the

[xhwang, 2015/04/03 07:00:13]

What about using web canvas to manipulate video? In that case, is the "output
layer" visible? If the Client doesn't issue Render(), the canvas won't be able
to get new frames...

[DaleCurtis, 2015/04/03 17:14:52]

Sunny, can you clarify on this? It's my expectation that we'll only stop
receiving vsync callbacks when the entire tab goes into the background, but not
if the tab is in the foreground but the VideoLayer is invisible.

[xhwang, 2015/04/03 19:18:27]

What if the JS app draw the video into a canvas, grab the pixels from the
canvas, and  store them or send them to the server? This has nothing to do with
visibility and it probably *should* work even if the tab is in background or the
whole browser is hidden.

[DaleCurtis, 2015/04/03 19:59:16]

I think if we update every say 100ms that should be fine. Once the app goes into
the background its timers will also be slowed down; requestAnimationFrame won't
fire and setTimeout, setInterval extend to once a second I think. Perhaps we
could detect paint() calls and switch to a higher frequency update.

+  // provider of visibility.
+  virtual void SetVisible(bool visible) = 0;
+
+ protected:
+  virtual ~VideoFrameProvider() {}
 };
 
 }  // namespace cc
 
 #endif  // CC_LAYERS_VIDEO_FRAME_PROVIDER_H_