Mojo Video Capture service in media/capture

media/capture/interfaces/video_capture.mojom

Index: media/capture/interfaces/video_capture.mojom
diff --git a/media/capture/interfaces/video_capture.mojom b/media/capture/interfaces/video_capture.mojom
new file mode 100644
index 0000000000000000000000000000000000000000..8c0dd1410641afa1af1e9acb6fa5adddec166287
--- /dev/null
+++ b/media/capture/interfaces/video_capture.mojom
@@ -0,0 +1,95 @@
+// Copyright 2016 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// This mojom defines the interface for enumerating Video Capture Devices and
+// selecting one for capturing. Such a VCD is manipulated via creation of a

[perkj_chrome, 2016/03/04 15:34:02]

Want to update this text with the new names.

[mcasas, 2016/03/04 19:59:45]

Done.

+// Stream, which is then Start()ed to produce FrameInfos to a StreamCient. In
+// this sense, this API mimics a UserMediaClient.
+
+module media.mojom;
+
+import "media/mojo/interfaces/media_types.mojom";
+import "ui/mojo/geometry/geometry.mojom";
+
+struct VideoCaptureDeviceInfo {
+  string device_id;
+  string label;

[perkj_chrome, 2016/03/04 15:34:02]

If this is from VideoCaptureDevice::Name - can we call it device_id and name
then.

[mcasas, 2016/03/04 19:59:45]

Corresponds to device_name.GetNameAndModel().

Done.

+  // TODO(mcasas): add VideoFacingMode facing_mode;
+};
+
+struct StreamOptions {

[perkj_chrome, 2016/03/04 15:34:02]

VideoCaptureDeviceOptions?  There are no Steams any longer which is nice. 

[mcasas, 2016/03/04 19:59:45]

These are not the VideoCaptureDevice
options, but the Capture options. 

Renamed to VideoCaptureOptions,
please instruct otherwise.

+  string device_id;
+
+  // Replicate the used fields of VideoCaptureParams.
+  mojo.Size capture_size;
+  double frame_rate;
+  // TODO(mcasas): add and wire PowerLineFrequency and ResolutionChangePolicy.
+};
+
+// Information provided to a VideoStreamClient about an available video frame.
+struct FrameInfo {
+  // Shared memory handle to allocated storage for the frame. This handle may
+  // be mapped by the client for read-only access.
+  // TODO(mcasas): investigate if sharing this buffer per frame has any impact
+  // in performance. If so, reinstaurate a sharing-buffer-preamble.
+  handle<shared_buffer> storage_handle;

[magjed_chromium, 2016/03/04 15:41:42]

|storage_handle| contains all the data, right? Don't we need plane offsets,
strides etc?

[mcasas, 2016/03/04 19:59:45]

Not yet. Offsets are not used in capture and 
the strides are the |coded_size|.

[magjed_chromium, 2016/03/07 15:36:24]

With plane offsets I mean: how do you calculate the pointer for the U and V
plane from |storage_handle|? When you say that the strides are in |coded_size|,
do you mean that everything is tightly packed? Otherwise strides are not the
same as |coded_size|, that's why we have both strides and coded_size in
VideoFrame.

[mcasas, 2016/03/07 20:57:49]

Yes.
Correct. But that capability of VideoFrame is not used
for capturing scenarios.

[magjed_chromium, 2016/03/09 14:11:18]

Acknowledged.

+  // Size in bytes of the allocated storage.
+  int32 storage_size;
+
+  // Pixel format of the frame data in storage.
+  media.interfaces.VideoFormat pixel_format;
+
+  // Size of the frame in pixels. This includes pixel data for the whole image;
+  // i.e. for YUV formats with subsampled chroma planes where the visible
+  // portion does not line up on a sample boundary, |coded_size| will be rounded
+  // up appropriately and pixel data will be provided for the odd pixels.
+  mojo.Size coded_size;
+
+  // Visible rectangle of pixels in the frame. This must be a subrect of
+  // |coded_rect|. May be odd with respect to the sample boundaries, e.g. for
+  // formats with subsampled chroma. This is used e.g. for letterboxing.
+  mojo.Rect visible_rect;

[perkj_chrome, 2016/03/04 15:34:02]

Do we use this now? Can't we add this if we ever need it with Video capture
devices?

[mcasas, 2016/03/04 19:59:45]

(Some) ChromeOS devices actually use this field
(I tried to remove it altogether a while ago).

+
+  // A coded version of base::TimeTicks::Now() at the time the frame data was
+  // generated by its source. For the sake of consistency this must always be
+  // interpreted using base::TimeTicks::FromInternalValue or some equivalent.
+  int64 timestamp;

[perkj_chrome, 2016/03/04 15:34:02]

Why can't this be TimeTicks then instead of int64?

[mcasas, 2016/03/04 19:59:45]

There are no marshalling primitives for TimeTicks
yet in mojo (think about this definition as an IPC
struct).

[perkj_chrome, 2016/03/07 23:58:27]

Acknowledged.

+};
+
+// The client interface for a VideoCaptureStream.
+interface VideoCaptureStreamClient {
+  // Informs the client that new frame data is available. |info| provides
+  // details about this individual frame.
+  FrameAvailable(FrameInfo info) => ();

[perkj_chrome, 2016/03/04 15:34:02]

Should this be OnFrameAvailable to be consistent with other callback interfaces?


Or is there another naming convention for mojo? 

Or maybe even better. use the same as 
	   // VideoCaptureDeviceClientImpl::Delegate
   void OnFrame(const scoped_refptr<MojoVideoFrame>& frame)    
   void OnError(const std::string& reason) override;

[mcasas, 2016/03/04 19:59:45]

Done.
Wouldn't that be confusing?

+
+  // Some Error has happened; the capture will in all likelihood be interrupted.

[perkj_chrome, 2016/03/04 15:34:02]

Please decide what will happen when an error occur. I think it should be stopped
and that this VideoCaptureStreamClient is no longer usable. 

[perkj_chrome, 2016/03/04 15:34:02]

nit: Suggest - An error has occurred.....

[mcasas, 2016/03/04 19:59:45]

The client doesn't need to know, because is
down to the implementation to decide if the
error is recoverable or not. That's why the
Client interface has only FrameAvailable()
and Error() methods. The client would only,
maybe, see that there are no more 
FrameAvailable() events.

[mcasas, 2016/03/04 19:59:45]

Done.

+  Error(string error);
+};
+
+// The VideoCaptureStream interface abstracts a distinct local source of video
+// frame data such as a WebCam.
+// TODO(mcasas): Consider extending to desktop capture and web view capture.
+interface VideoCaptureStream {
+  // Starts sending frames to |client|. Must only be called once and must be the
+  // first call on the Stream.
+  Start(VideoCaptureStreamClient client);
+
+  // Stops the capture, releasing all associated resources.
+  Stop();
+};
+
+// The primary top-level service interface exposed by the video_capture
+// component. This can used by a UserMediaClient implementation to get hold of a

[perkj_chrome, 2016/03/04 15:34:02]

If this is browser side only this will not be used by UserMediaClient.

[mcasas, 2016/03/04 19:59:45]

Done.

+// VideoCaptureStream that can be Start()ed to produce video frames. In a way,
+// it is a video-only flavour of WebUserMediaClient.
+// TODO(mcasas): Add a method to retrieve the capture capabilities of a given
+// |device_id|.
+interface VideoCaptureHandler {
+  EnumerateDevices()
+      => (array<VideoCaptureDeviceInfo> devices);
+
+  RequestVideoCaptureStream(StreamOptions options, string security_origin)

[perkj_chrome, 2016/03/04 15:34:02]

why security_origin if this is browser side only? 

[mcasas, 2016/03/04 19:59:45]

Reminder of where this all started, months and
months ago. Removed.

+      => (VideoCaptureStream? stream);
+};
+