Add an additional VDA::Flush() mode to return all allocated buffers.

media/video/video_decode_accelerator.h

 // Copyright (c) 2012 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #ifndef MEDIA_VIDEO_VIDEO_DECODE_ACCELERATOR_H_
 #define MEDIA_VIDEO_VIDEO_DECODE_ACCELERATOR_H_
 
 #include <stdint.h>
 
 #include <memory>
@@ skipping 96 matching lines @@
     // When set to ALLOCATE, the VDA is expected to allocate backing memory
     // for PictureBuffers at the time of AssignPictureBuffers() call.
     // When set to IMPORT, the VDA will not allocate, but after receiving
     // AssignPictureBuffers() call, it will expect a call to
     // ImportBufferForPicture() for each PictureBuffer before use.
     enum class OutputMode {
       ALLOCATE,
       IMPORT,
     };
 
+    // Specifies how the VDA should behave on VDA::Flush().
+    //
+    // - KEEP_OUTPUT_BUFFERS: the VDA may retain PictureBuffers that did not
+    // contain any decoded pictures at the end of flush for later use. In this
+    // mode, VDA returns via PictureReady() only the Pictures containing any
+    // remaining frames, decoded and/or released as a part of the flush, and
+    // ends the flush sequence by posting NotifyFlushDone().
+    //
+    // - RETURN_OUTPUT_BUFFERS: the VDA must return all PictureBuffers via
+    // PictureReady() before posting NotifyFlushDone(), including buffers not

[Owen Lin, 2016/04/13 09:21:39]

Actually, the whole VideoDecodeAccelerator runs in one thread. I mean the
methods of VDA as well as the callbacks in Client are called in this thread (and
we call it child thread). The fact that there is another decoder thread to help
sending/processing buffers is implementation details that each VDAImpl should
hide.

So the contract is simple, before Client::NotifyFlushDone() is called, all
buffers should be returned by NotifyPictureReady(). And after that, VDA should
start to use and hold any pictures sent by ReusePictureBuffer().

To make things clear I list two ideas in the shared document. Please take a look
and see if that is doable. But the point is to keep the above contract.

https://docs.google.com/a/google.com/document/d/1AuZXQDyDVcs1om5qzoQjBhtzKTEQ...

[Pawel Osciak, 2016/04/14 02:14:01]

That's true, however VDA is also an IPC interface and calls are IPC messages and
can be posted from and executed in separate worlds (e.g. different processes).
How it works on the GPU process side and its details is not applicable in all
cases. The only thing we can guarantee is the ordering of IPC messages, but not
when they arrive or when they execute and how.
The Client and VDA are in different processes. The only way for the Client to
guarantee that VDA will keep the buffers returned to it after flush sequence, is
to wait for NotifyFlushDone() before actually posting any ReusePictureBuffers().
There is no way for Client or the VDA to know or control when a message arrives
on the other side, apart from not posting it, or posting it only after receiving
the message that has to precede it.

The problem is, there is no notification Client->VDA that it consumed the
NotifyFlushDone, so VDA cannot govern this. Instead, the Client must assume that
any ReusePictureBuffer() call it posts after calling Flush() and before it gets
NotifyFlushDone() may behave in either way. Only if Client posts RPB() after
getting NotifyFlushDone(), it can guarantee VDA will keep the buffer.

[Owen Lin, 2016/04/14 02:57:17]

I see. You're right, I only think about the the use case for ArcGVDA which also
runs in the GPU process. However this make things difficult to handle. Client
must keep returning buffers since it doesn't know how many buffers are required
to finish the flush. On the other hand, those returned buffers could be hold
because VDA may already post NotifyFlushDone(). That leads us back to the
original issue: ArcGVDA may not have a buffer to send the EOS flag.

How about adding a new function like ResumeFromFlush() in VDA, before it is
called, VDA just keeps returning picture buffers. Otherwise, I would suggest we
just keeping one more picture buffer in ArcGVDA and use it to send back the EOS
flag as a temporary solution.

+    // containing decoded frames (if any). Buffers not containing any decoded
+    // data must have bitstream_buffer_id set to -1.
+    // Note: buffers returned to the VDA by the Client via ReusePictureBuffer()
+    // after NotifyFlushDone() is posted by the VDA (even if NotifyFlushDone()
+    // has not yet been serviced by the Client) can be kept by the VDA and
+    // reused after flush.
+    enum class FlushMode {
+      KEEP_OUTPUT_BUFFERS,
+      RETURN_OUTPUT_BUFFERS,
+    };
+
     Config() = default;
     Config(VideoCodecProfile profile);
     Config(const VideoDecoderConfig& video_decoder_config);
 
     std::string AsHumanReadableString() const;
 
     // |profile| combines the information about the codec and its profile.
     VideoCodecProfile profile = VIDEO_CODEC_PROFILE_UNKNOWN;
 
     // The flag indicating whether the stream is encrypted.
     bool is_encrypted = false;
 
     // The flag indicating whether the client supports deferred initialization
     // or not.
     bool is_deferred_initialization_allowed = false;
 
     // An optional graphics surface that the VDA should render to. For setting
     // an output SurfaceView on Android. It's only valid when not equal to
     // |kNoSurfaceID|.
     int surface_id = kNoSurfaceID;
 
     // Coded size of the video frame hint, subject to change.
     gfx::Size initial_expected_coded_size;
 
     OutputMode output_mode = OutputMode::ALLOCATE;
+
+    FlushMode flush_mode = FlushMode::KEEP_OUTPUT_BUFFERS;
   };
 
   // Interface for collaborating with picture interface to provide memory for
   // output picture and blitting them. These callbacks will not be made unless
   // Initialize() has returned successfully.
   // This interface is extended by the various layers that relay messages back
   // to the plugin, through the PPP_VideoDecoder_Dev interface the plugin
   // implements.
   class MEDIA_EXPORT Client {
    public:
@@ skipping 11 matching lines @@
     // larger than the value requested.
     virtual void ProvidePictureBuffers(uint32_t requested_num_of_buffers,
                                        uint32_t textures_per_buffer,
                                        const gfx::Size& dimensions,
                                        uint32_t texture_target) = 0;
 
     // Callback to dismiss picture buffer that was assigned earlier.
     virtual void DismissPictureBuffer(int32_t picture_buffer_id) = 0;
 
     // Callback to deliver decoded pictures ready to be displayed.
+    // If picture.bitstream_buffer_id is set to -1, the buffer does not contain
+    // any decoded data, but is being returned as a part of Flush().
     virtual void PictureReady(const Picture& picture) = 0;
 
     // Callback to notify that decoded has decoded the end of the current
     // bitstream buffer.
     virtual void NotifyEndOfBitstreamBuffer(int32_t bitstream_buffer_id) = 0;
 
     // Flush completion callback.
     virtual void NotifyFlushDone() = 0;
 
     // Reset completion callback.
@@ skipping 78 matching lines @@
   // for each buffer that has been processed so that decoder may know onto which
   // picture buffers it can write the output to.
   //
   // Parameters:
   //  |picture_buffer_id| id of the picture buffer that is to be reused.
   virtual void ReusePictureBuffer(int32_t picture_buffer_id) = 0;
 
   // Flushes the decoder: all pending inputs will be decoded and pictures handed
   // back to the client, followed by NotifyFlushDone() being called on the
   // client.  Can be used to implement "end of stream" notification.
+  // Exact behavior of Flush() depends on the selected FlushMode, set in the
+  // Config passed to Initialize().
   virtual void Flush() = 0;
 
   // Resets the decoder: all pending inputs are dropped immediately and the
   // decoder returned to a state ready for further Decode()s, followed by
   // NotifyResetDone() being called on the client.  Can be used to implement
   // "seek".
   virtual void Reset() = 0;
 
   // Destroys the decoder: all pending inputs are dropped immediately and the
   // component is freed.  This call may asynchornously free system resources,
@@ skipping 57 matching lines @@
 // Specialize std::default_delete so that scoped_ptr<VideoDecodeAccelerator>
 // uses "Destroy()" instead of trying to use the destructor.
 template <>
 struct MEDIA_EXPORT default_delete<media::VideoDecodeAccelerator> {
   void operator()(media::VideoDecodeAccelerator* vda) const;
 };
 
 }  // namespace std
 
 #endif  // MEDIA_VIDEO_VIDEO_DECODE_ACCELERATOR_H_