Pepper: add Image Capture interfaces.

ppapi/api/ppb_image_capture.idl

Index: ppapi/api/ppb_image_capture.idl
diff --git a/ppapi/api/ppb_image_capture.idl b/ppapi/api/ppb_image_capture.idl
new file mode 100644
index 0000000000000000000000000000000000000000..5d2f49a9cbbc05c41b8469a8f978b6809d44719b
--- /dev/null
+++ b/ppapi/api/ppb_image_capture.idl
@@ -0,0 +1,122 @@
+/* Copyright 2014 The Chromium Authors. All rights reserved.
+ * Use of this source code is governed by a BSD-style license that can be
+ * found in the LICENSE file.
+ */
+
+/**
+ * Defines the <code>PPB_ImageCapture</code> interface. Used for taking
+ * a picture from the source of a MediaStream video track.
+ */
+
+[generate_thunk]
+
+label Chrome {
+  [channel=dev] M39 = 0.1
+};
+
+[version=0.1]
+interface PPB_ImageCapture {
+  /**
+   * Creates a PPB_ImageCapture resource.
+   *
+   * @param[in] media_stream_video_track A <code>PP_Resource</code> identifying
+   * one MediaStream video track.
+   *

[Justin Chuang, 2014/07/21 17:34:56]

What happens if the video track is Close()-ed?

[wuchengli, 2014/07/29 03:24:49]

Now I think video track and image capture should be decoupled more. We should
not force the applications to have an active video track when using image
capture. The constructor simply provides a way to use the same camera that video
tracks associates with. So the plugin can continue to use ImageCapture object if
the video track is closed.

Pawel. Does V4L2 require that some settings can only be controlled when
streaming is on? For example, can we control autofocus when streaming is off?

[Owen Lin, 2014/07/29 03:34:43]

I think we need stream on to take a photo. It need continuously to adjust the 3A
settings.

+   * @return A <code>PP_Resource</code> corresponding to a
+   * PPB_ImageCapture resource if successful, 0 if failed.
+   */
+  PP_Resource Create([in] PP_Resource media_stream_video_track);

[dmichael (off chromium), 2014/07/16 23:10:54]

I'm a little unsure about this. I like that it neatly takes care of permissions,
but it's a little strange. There's not an analogous JavaScript API exactly. This
is kind of a lower-level API, wanting to control stuff like AutoFocus.

[wuchengli, 2014/07/17 14:05:47]

I followed the design in Mediastream Image Capture draft. The idea is
MediaStreamVideoTrack is for preview and we want to use the same camera to take
a picture. Or we can take a video sourceId or MediaStreamTrack.id in the
constructor.

http://www.w3.org/TR/image-capture/#taking-a-picture-if-red-eye-reduction-is-...

[wuchengli, 2014/07/29 03:24:49]

I've changed the constructor argument to source ID.

+
+  /**
+   * Determines if a resource is an image capture resource.
+   *
+   * @param[in] resource The <code>PP_Resource</code> to test.
+   *
+   * @return A <code>PP_Bool</code> with <code>PP_TRUE</code> if the given
+   * resource is an image capture resource or <code>PP_FALSE</code>
+   * otherwise.
+   */
+  PP_Bool IsImageCapture([in] PP_Resource resource);
+
+  /**
+   * Sets the configuration of the image capture.
+   * If <code>SetConfig()</code> is not called, default settings will be used.
+   *
+   * @param[in] image_capture A <code>PP_Resource</code> corresponding to an
+   * image capture resource.
+   * @param[in] config A <code>PP_ImageCaptureConfig</code> object.
+   * @param[in] callback <code>PP_CompletionCallback</code> to be called upon
+   * completion of <code>SetConfig()</code>.
+   *
+   * @return An int32_t containing a result code from <code>pp_errors.h</code>.
+   * Returns <code>PP_ERROR_INPROGRESS</code> if there is a pending call of
+   * <code>SetConfig()</code> or <code>TakePhoto()</code>.
+   * If an error is returned, all configuration will not be changed.
+   */
+  int32_t SetConfig([in] PP_Resource image_capture,
+                    [in] PP_Resource config,
+                    [in] PP_CompletionCallback callback);
+
+  /**
+   * Gets the configuration of the image capture.
+   *
+   * @param[in] image_capture A <code>PP_Resource</code> corresponding to an
+   * image capture resource.
+   * @param[out] config A <code>PP_ImageCaptureConfig</code> for storing the
+   * current image capture config on success. Otherwise, the values will not be
+   * changed.
+   * @param[in] callback <code>PP_CompletionCallback</code> to be called upon
+   * completion of <code>GetConfig()</code>.
+   *
+   * @return An int32_t containing a result code from <code>pp_errors.h</code>.
+   */
+  int32_t GetConfig([in] PP_Resource image_capture,
+                    [out] PP_Resource config,
+                    [in] PP_CompletionCallback callback);
+
+  /**
+   * Gets the image capture capabilities of the camera.
+   *
+   * @param[in] image_capture A <code>PP_Resource</code> corresponding to an
+   * image capture resource.
+   * @param[out] capabilities A <code>PP_ImageCaptureCapabilities</code> for
+   * storing the image capture capabilities on success. Otherwise, the value
+   * will not be changed.
+   * @param[in] callback <code>PP_CompletionCallback</code> to be called upon
+   * completion of <code>GetCapabilities()</code>.
+   *
+   * @return An int32_t containing a result code from <code>pp_errors.h</code>.
+   */
+  int32_t GetCapabilities([in] PP_Resource image_capture,
+                          [out] PP_Resource capabilities,
+                          [in] PP_CompletionCallback callback);
+
+  /**
+   * Takes a photo from the camera.
+   *

[Justin Chuang, 2014/07/21 17:34:56]

Add comments that it may mute()/unmute() the paired video track object? It seems
important.

[wuchengli, 2014/07/29 03:24:50]

Done.

+   * @param[in] image_capture A <code>PP_Resource</code> corresponding to an
+   * image capture resource.
+   * @param[out] postview_frame A <code>PP_Resource</code> corresponding to a
+   * VideoFrame resource used to store the postview. The client can use this to
+   * show the captured photo. The format and the size are decided by the
+   * associated MediaStream video track.
+   * @param[out] photo A <code>PP_Resource</code> corresponding to a VideoFrame
+   * resource used to store the encoded picture. The default format is JPEG.

[Justin Chuang, 2014/07/21 17:34:56]

I suspect that we need to add a new PP_Resource. 

I'm not sure if PPB_ImageData fits here, which seems to be designed for
alpha-blending immediately (only premultiplied ARGB format). I don't know if
extending PPB_ImageData to support JPG will break the overall design.

[wuchengli, 2014/07/29 03:24:49]

PPB_ImageData applies to only ARGB so we should not use it. It looks like we can
add Jpeg to ppapi/api/ppb_video_frame.idl (not
ppapi/api/private/pp_video_frame_private.idl). So I use VideoFrame PP_Resource
for both postview and jpeg pictures.

+   * @param[in] callback A <code>PP_CompletionCallback</code> to be called when
+   * the picture has been taken and the camera can be moved without making the
+   * picture blurry. The client can play a shutter sound if needed.
+   * @param[in] postview_callback A <code>PP_CompletionCallback</code> to be
+   * called to store a postview video frame in <code>postview_frame</code>
+   * @param[in] photo_callback A <code>PP_CompletionCallback</code> to be
+   * called upon completion of TakePhoto(). <code>photo</code> will contain the
+   * encoded picture.

[Justin Chuang, 2014/07/21 17:34:56]

A general question: do we want to do something like
PPB_MediaStreamVideoTrack.RecycleFrame()? I'm worried about the internal memory
fragmentation, esp when the image size can go up to 10MB. 

I've never done real camera handling before, just a feeling that a frame ring
can be also helpful when supporting burst mode in the future. 

[wuchengli, 2014/07/29 03:24:49]

We may need it in the future. When I was on Android, taking pictures allocated
memory every time, caused lots of GC, and slowed down the image capture. But the
size of Jpeg is not fixed like YUV. So the buffer size has to be provided by the
driver. Let's make image capture work first and come back to optimization later.
We should only expose the minimum API that are required today. Let's worry about
this in the future.

+   *
+   * @return An int32_t containing a result code from <code>pp_errors.h</code>.
+   */
+  int32_t TakePhoto([in] PP_Resource image_capture,
+                    [out] PP_Resource postview_frame,
+                    [out] PP_Resource photo,
+                    [in] PP_CompletionCallback shutter_callback,
+                    [in] PP_CompletionCallback postview_callback,
+                    [in] PP_CompletionCallback photo_callback);

[dmichael (off chromium), 2014/07/16 23:10:54]

We can't use multiple completion callbacks for 1 function. It's supposed to be
possible to pass a blocking callback. It's not clear what would happen if one
callback was passed as blocking and two others weren't for instance. And the
return code is sometimes an indication of whether the callback will be run.

So it's unclear what PP_OK_COMPLETIONPENDING or PP_OK would mean...  which
callback(s) will still be run?

Two alternatives:
1) Split it in to multiple calls, each with its own completion callbakc.
2) Use a PPP-style interface (See PPB_Messaging::RegisterMessageHandler and
PPP_MessageHandler)

1 might be best, since it would allow you to block at each stage. Either lets
you get your result on a background thread.

[wuchengli, 2014/07/17 14:05:47]

You are right. I read the documentation of completion callback and it doesn't
make sense to have three of them for one function.

TakePhoto should be one function call and three callbacks. Making multiple calls
is not efficient. I'll try option 2.

[wuchengli, 2014/07/29 03:24:49]

I moved the callbacks to PPP_ImageCapture_Private. This is how all other pepper
APIs do like PPB_VideoDecoder/PPP_VideoDecoder.

+};