Interfaces for encoded video sources

content/common/media/encoded_video_source_messages.h

+// Copyright 2013 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "base/shared_memory.h"
+#include "content/common/content_export.h"
+#include "ipc/ipc_message_macros.h"
+#include "ipc/ipc_message_start.h"
+#include "media/video/encoded_video_source.h"
+#include "media/video/video_encode_types.h"
+
+#undef IPC_MESSAGE_EXPORT
+#define IPC_MESSAGE_EXPORT CONTENT_EXPORT
+#define IPC_MESSAGE_START EncodedVideoSourceMsgStart
+
+IPC_ENUM_TRAITS(media::VideoCodec)
+
+IPC_ENUM_TRAITS(media::EncodedVideoBitstream::StatusCode)
+
+IPC_STRUCT_TRAITS_BEGIN(media::TemporalLayerParameters)
+  IPC_STRUCT_TRAITS_MEMBER(enabled)
+  IPC_STRUCT_TRAITS_MEMBER(target_bitrate)
+IPC_STRUCT_TRAITS_END()
+
+IPC_STRUCT_TRAITS_BEGIN(media::RuntimeVideoEncodingParameters)
+  IPC_STRUCT_TRAITS_MEMBER(frames_per_second)
+  IPC_STRUCT_TRAITS_MEMBER(max_bitrate)
+  IPC_STRUCT_TRAITS_MEMBER(max_qp)
+  IPC_STRUCT_TRAITS_MEMBER(temporal_layer_params)
+IPC_STRUCT_TRAITS_END()
+
+IPC_STRUCT_TRAITS_BEGIN(media::VideoEncodingParameters)
+  IPC_STRUCT_TRAITS_MEMBER(codec)
+  IPC_STRUCT_TRAITS_MEMBER(resolution)
+  IPC_STRUCT_TRAITS_MEMBER(runtime_params)
+IPC_STRUCT_TRAITS_END()
+
+IPC_STRUCT_TRAITS_BEGIN(media::BufferEncodingMetadata)
+  IPC_STRUCT_TRAITS_MEMBER(timestamp)
+  IPC_STRUCT_TRAITS_MEMBER(frame_type_flags)
+  IPC_STRUCT_TRAITS_MEMBER(temporal_layer_id)
+  IPC_STRUCT_TRAITS_MEMBER(layer_sync)
+  IPC_STRUCT_TRAITS_MEMBER(droppable)
+IPC_STRUCT_TRAITS_END()
+
+// Need to typedef BitstreamBufferMap in order to prevent the IPC message macros
+// from choking to the comma.

[Ami GONE FROM CHROMIUM, 2013/03/19 18:01:49]

s/to/on/
also, :'(

[vmr, 2013/03/20 12:09:14]

Done.

+typedef std::map<int, base::SharedMemoryHandle> BitstreamBufferMap;
+
+// Message from Renderer to Browser process to create a bitstream with specific
+// parameters. A successful request results in beginning of streaming and
+// EncoderVideoSourceMsg_BitstreamCreated message to Renderer. A failed request
+// triggers EncodedVideoSourceMsg_BitstreamDestroyed message. As this set of
+// messages is designed to work with existing device, device_id is determined
+// on the basis which device is providing the encoded video source
+// functionality. Renderer is responsible for generating unique stream_id within

[Ami GONE FROM CHROMIUM, 2013/03/19 18:01:49]

I think the sentence of l.53-56 is a convoluted way of saying "use the same
device_id as VideoCapture uses", which emphasizes that this message is
camera-specific (not generic encoder relevant).

A key observation, perhaps: when we have generic encoding, the CreateBitstream
message will probably go to the GPU process, not the browser process.  I
therefore wonder how generic this proposal is.

Perhaps while the API in media/video is left generic, this messages should
acknolwedge that they are in fact all about cameras, not encoders?  (and thus
move to live near the other VideoCapture messages)

(the same point, arrived at from the other direction: why do the handlers for
generic Encode APIs go through Video*Capture*MessageFilter?)

[vmr, 2013/03/20 12:09:14]

This is probably the least good part of this feature. Back in December in Taipei
we decided to try to define a set of messages that could be used with a encoder
as well as with a encoding camera and we came to conclusion that applications
are not interested in encoding functionality per se, but the output of the
encoding process which is encapsulated in the form of a Bitstream, whose
properties can be manipulated. As we have been working with a camera-based
implementation that perspective on the design got more attention and some
decisions might have been made otherwise to generalize it better.

Anyway the underlying idea that we could have abstracted "bitstream subscription
and control" interface available for applications is valid. I am not sure how to
bend this CL towards that direction right now, though.

+// its context that will be used to identify bitstreams in IPC.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceHostMsg_CreateBitstream,
+                     int /* device_id */,
+                     int /* stream_id */,
+                     media::VideoEncodingParameters /* params */)
+
+// Message from Renderer to Browser process to tell that the data within a
+// buffer has been processed and it can be reused to encode upcoming bitstream.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceHostMsg_BitstreamBufferConsumed,
+                     int /* device_id */,
+                     int /* stream_id */,
+                     int /* buffer_id */)
+
+// Message from Renderer to Browser process to stop streaming a bitstream. When
+// Browser has finalized the bitstream it will trigger
+// EncodedVideoSourceMsg_BitstreamDestroyed message back from Browser to
+// Renderer. Renderer must be prepared to receive
+// EncodedVideoSourceMsg_BitstreamReady messages until it has received the
+// EncodedVideoSourceMsg_BitstreamDestroyed message.
+IPC_MESSAGE_CONTROL2(EncodedVideoSourceHostMsg_DestroyBitstream,
+                     int /* device_id */,
+                     int /* stream_id */)
+
+// Message from Renderer to Browser process to set a stream's bitstream config.
+// Will always result in EncodedVideoSourceMsg_BitstreamConfigChanged containing
+// currently active parameters, regardless of whether this call succeeded or
+// not.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceHostMsg_TryConfigureBitstream,
+                     int /* device_id */,
+                     int /* stream_id */,
+                     media::RuntimeVideoEncodingParameters /* params */)
+
+// Message from Renderer to Browser process to request special frames from the
+// encoded video source. There will be no acknowledgement for this request and
+// the effect is only visible in the bitstream buffers passed to client
+// through the EncodedVideoSourceMsg_BitstreamReady message. This request is
+// served on a best-effort basis and client is not given any guarantees of the
+// realization or timing of the request. Flags parameter will be interpreted in
+// codec-specific manner using enumerations.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceHostMsg_RequestSpecialFrame,
+                     int /* device_id */,
+                     int /* stream_id */,
+                     int /* flags */)
+
+// Message from Browser to Renderer process to acknowledge a request to create
+// an encoded video bitstream. Encoding parameters on which the bitstream has
+// been created are contained in the message with a boolean value (is_mutable),
+// which informs whether the client can adjust the bitstream config using
+// TryConfigureBitstream. When this message occurs bitstream can be considered
+// to be streaming and Renderer should be ready to start accepting
+// EncodedVideoSourceMsg_BitstreamReady messages and buffers contained within
+// them. Shared memory buffers used to deliver the bitstream are assigned with
+// buffer ids as specified by the buffers parameter.
+IPC_MESSAGE_CONTROL5(EncodedVideoSourceMsg_BitstreamCreated,
+                     int /* device id */,
+                     int /* stream id */,
+                     media::VideoEncodingParameters /* params */,
+                     bool /* is_mutable */,
+                     BitstreamBufferMap /* buffers */)

[Ami GONE FROM CHROMIUM, 2013/03/19 18:01:49]

This requires you to decide how many buffers to allocate at bitstream creation
time, and gives you no way to react to fast/slow consumers of the bitstream.

Why not instead have BitstreamReady pass both a shmem handle and a
(browser-generated) buffer id and then have Consumed take the ID back?

[vmr, 2013/03/20 12:09:14]

This is fair point.
I was thinking about the cost of open+map of a shm handle (which I don't know
and should test). In current way of things they get opened and mapped once,
whereas your proposal requires opening and mapping them each time.

+
+// Message from Browser to Renderer process to acknowledge a request to destroy
+// an encoded video bitstream. Also used to signal renderer process that an
+// unrecoverable error has occurred and as a result bitstream has been
+// destroyed.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceMsg_BitstreamDestroyed,
+                     int /* device id */,
+                     int /* stream id */,
+                     media::EncodedVideoBitstream::StatusCode /* status */)
+
+// Message from Browser to Renderer process to inform the clients of the current
+// encoding parameters, regardless of whether the previous request to change
+// them has been successful or not. This is called usually as a response to
+// EncodedVideoSourceHostMsg_TryConfigureBitstream during runtime, but can occur
+// also as a result of config change initiated by encoder or other clients in
+// the system. E.g. if there are multiple clients and bitstream config change is
+// requested from one client, all clients should be prepared to handle the
+// configuration change.
+IPC_MESSAGE_CONTROL3(EncodedVideoSourceMsg_BitstreamConfigChanged,
+                     int /* device id */,
+                     int /* stream id */,
+                     media::RuntimeVideoEncodingParameters /* current_params */)
+
+// Message from Browser to Renderer process indicating that a bitstream buffer
+// is available for the stream.
+IPC_MESSAGE_CONTROL4(EncodedVideoSourceMsg_BitstreamReady,
+                     int /* device id */,
+                     int /* stream_id */,
+                     int /* buffer_id */,
+                     media::BufferEncodingMetadata /* metadata */)
+