MJPEG acceleration for video capture using VAAPI

content/browser/renderer_host/gpu_jpeg_decoder.h

Index: content/browser/renderer_host/gpu_jpeg_decoder.h
diff --git a/content/browser/renderer_host/gpu_jpeg_decoder.h b/content/browser/renderer_host/gpu_jpeg_decoder.h
new file mode 100644
index 0000000000000000000000000000000000000000..cfd277617855cb785b74b61389fef9b0702a1dd8
--- /dev/null
+++ b/content/browser/renderer_host/gpu_jpeg_decoder.h
@@ -0,0 +1,123 @@
+// Copyright 2015 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_BROWSER_RENDERER_HOST_GPU_JPEG_DECODER_H_
+#define CONTENT_BROWSER_RENDERER_HOST_GPU_JPEG_DECODER_H_
+
+#include "base/memory/ref_counted.h"
+#include "base/memory/scoped_ptr.h"
+#include "content/common/content_export.h"
+#include "media/video/capture/video_capture_device.h"
+#include "media/video/jpeg_decode_accelerator.h"
+
+namespace base {
+class MessageLoopProxy;
+}
+
+namespace content {
+
+// Adapter to GpuJpegDecodeAccelerator for VideoCaptureDevice::Client. It takes
+// care of GpuJpegDecodeAccelerator creation, shared memory, and threading
+// issues. If decoding fails, it will fallback to software decode for current
+// frame and reject upcoming frames.
+//
+// There are two threads involved in this class. Ctor, dtor, Initialize(), and
+// DecodeCapturedData() are running on one thread (i.e., device thread for
+// Linux). The decoding responses (JpegDecodeAccelerator::Client functions) run
+// on IO thread.
+class CONTENT_EXPORT GpuJpegDecoder
+    : public media::JpegDecodeAccelerator::Client {
+ public:
+  // Return true if JPEG hardware decoding is supported on this device.

[mcasas, 2015/04/28 00:04:50]

s/Return/Returns/
(Comments are descriptive rather than imperative [1)

[1]
http://google-styleguide.googlecode.com/svn/trunk/cppguide.html#Function_Comm...

[kcwu, 2015/04/30 19:25:43]

Done.

+  static bool Supported();
+
+  // |device_client| is where we request VideoCaptureDevice::Client::Buffer as
+  // output buffer from and send decoded result to.
+  GpuJpegDecoder(media::VideoCaptureDevice::Client* device_client);
+  ~GpuJpegDecoder();
+
+  // Create and intialize decoder in GPU side. Return false if failed.
+  bool Initialize();
+
+  // Return true if in failed state.
+  bool IsFailed();
+
+  // Decode a JPEG picture. If this class is in failed state,
+  // DecodeCapturedData() will ignore all decoding requests.
+  void DecodeCapturedData(const uint8* data,
+                          int length,
+                          const media::VideoCaptureFormat& frame_format,
+                          int rotation,
+                          const base::TimeTicks& timestamp);
+
+  // JpegDecodeAccelerator::Client implementation.
+  // These will be called on IO thread.
+  virtual void VideoFrameReady(int32_t buffer_id) override;

[mcasas, 2015/04/28 00:04:50]

Suggest s/VideoFrameReady/VideoFrameReadyOnIOThread/
since this class seems to live mostly on Device Thread.

Same s/NotifyError/NotifiyErrorOnIOThread/

[kcwu, 2015/04/30 19:25:43]

These names are inheritance from JpegDecodeAccelerator::Client and thus cannot
be changed.
Or do you have suggestions to deal that?

+  virtual void NotifyError(int32_t buffer_id,
+                           media::JpegDecodeAccelerator::Error error) override;
+
+ private:
+  // Fail state indicates to use software decode. See comment in
+  // FallbackToSoftwareDecode for detail.
+  enum FailState { NOT_FAIL, FAILING, FAILED };
+
+  // For keeping parameters of DecodeCapturedData. In case we need to pass them
+  // to software decoder if hardware decoding failed.
+  struct CapturedData {
+    // If accessed inside DecodeCapturedData(), |data| is owned by capturer
+    // (i.e., v4l2 driver on linux). Otherwise, |data| is backed by
+    // |in_shared_memory_|.
+    uint8* data;
+    int length;
+    media::VideoCaptureFormat frame_format;
+    int rotation;
+    base::TimeTicks timestamp;
+  };
+
+  bool IsDecoding();
+  // Check |bitstream_buffer_id| from GPU process is expected.
+  bool IsExpectedDecodeResponse(int32 bitstream_buffer_id);
+  // Current frame decoding is done. Allow DecodeCapturedData() to accept next
+  // frame.
+  void DecodeDone();
+  // Fallback current frame to software decode. Set flag to reject future
+  // upcoming frames.
+  void FallbackToSoftwareDecode();
+
+  // The "customer" of GpuJpegDecoder.
+  media::VideoCaptureDevice::Client* device_client_;
+  // The main working thread.
+  const scoped_refptr<base::MessageLoopProxy> main_task_runner_;

[mcasas, 2015/04/28 00:04:50]

Hmm |main| can have another meaning in the Browser process.
Change it to "Device Thread", this thread 
exists on all platforms, although in most it is shared with 
"Audio Thread" and is labeled as such. Device Thread is also
called "Capture Thread", so I suggest |capture_task_runner_|.

[kcwu, 2015/04/30 19:25:43]

Done.

+  // The underlying JPEG decode accelerator.
+  scoped_ptr<media::JpegDecodeAccelerator> decoder_;
+
+  // Below fields are protected by |lock_|.
+  base::Lock lock_;
+
+  FailState fail_state_;
+
+  // Keep the captured data from camera. If hardware decode failed, pass this
+  // to software decoder as fallback.
+  CapturedData captured_data_;
+
+  // Next id for |in_buffer_|.
+  int32 next_bitstream_buffer_id_;
+
+  // Shared memory to store JPEG stream buffer. |in_buffer_| is backed by this.
+  scoped_ptr<base::SharedMemory> in_shared_memory_;
+  // JPEG stream buffer as input to JpegDecodeAccelerator.
+  media::BitstreamBuffer in_buffer_;
+
+  // Buffer to hold decoded output of JPEG decoder.
+  // |out_frame_| is backed by this.
+  scoped_refptr<media::VideoCaptureDevice::Client::Buffer> out_buffer_;
+  // VideoFrame to receive decoded output of JpegDecodeAccelerator.
+  scoped_refptr<media::VideoFrame> out_frame_;
+
+  DISALLOW_COPY_AND_ASSIGN(GpuJpegDecoder);
+};
+
+}  // namespace content
+
+#endif  // CONTENT_BROWSER_RENDERER_HOST_GPU_JPEG_DECODER_H_