OLD | NEW |
(Empty) | |
| 1 # Image Capture API |
| 2 |
| 3 This folder contains the implementation of the [W3C Image Capture API](https://w
3c.github.io/mediacapture-image/). Image Capture was shipped in Chrome M59; ple
ase consult the [Implementation Status](https://github.com/w3c/mediacapture-imag
e/blob/master/implementation-status.md) if you think a feature should be availab
le and isn't. |
| 4 |
| 5 This API is structured around the [ImageCapture class](https://w3c.github.io/med
iacapture-image/#imagecaptureapi) _and_ a number of [extensions](https://w3c.git
hub.io/mediacapture-image/#extensions) to the `MediaStreamTrack` feeding it (let
's call them `theImageCapturer` and `theTrack`, respectively). |
| 6 |
| 7 ## API Mechanics |
| 8 |
| 9 ### `takePhoto()` and `grabFrame()` |
| 10 |
| 11 - `takePhoto()` returns the result of a single photographic exposure as a `Blob`
which can be downloaded, stored by the browser or displayed in an `img` element
. This method uses the highest available photographic camera resolution. |
| 12 |
| 13 - `grabFrame()` returns a snapshot of the live video in `theTrack` as an `ImageB
itmap` object which could (for example) be drawn on a `canvas` and then post-pr
ocessed to selectively change color values. Note that the `ImageBitmap` will onl
y have the resolution of the video track — which will generally be lower than th
e camera's still-image resolution. |
| 14 |
| 15 (_Adapted from the [Origin Trials Web Update post](https://developers.google.com
/web/updates/2016/12/imagecapture)_) |
| 16 |
| 17 ### Photo settings and capabilities |
| 18 |
| 19 The photo-specific options and settings are associated to `theImageCapturer` or
`theTrack` depending on whether a given capability/setting has an immediately re
cognisable effect on `theTrack`, in other words if it's "live" or not. For examp
le, changing the zoom level is instantly reflected on the `theTrack`, while conn
ecting the Red Eye Reduction, if available, is not. |
| 20 |
| 21 Object | type | retrieved by... | |
| 22 :--| :-- | --: | |
| 23 [`PhotoCapabilities`](https://w3c.github.io/mediacapture-image/##photocapabiliti
es-section) | non-live capabilities | `theImageCapturer.getPhotoCapabilities()`
| |
| 24 [`MediaTrackCapabilities`](https://w3c.github.io/mediacapture-image/#mediatrackc
apabilities-section) | live capabilities | `theTrack.getCapabilities()` | |
| 25 | | | |
| 26 [`PhotoSettings`](https://w3c.github.io/mediacapture-image/##photocapabilities-s
ection) | non-live settings | | |
| 27 [`MediaTrackSettings`](https://w3c.github.io/mediacapture-image/#mediatracksetti
ngs-section) | live settings | `theTrack.getSettings()` | |
| 28 |
| 29 ## Other topics |
| 30 |
| 31 ### Are `takePhoto()` and `grabFrame()` the same? |
| 32 |
| 33 These methods would not produce the same results as explained in [this issue com
ment](https://bugs.chromium.org/p/chromium/issues/detail?id=655107#c8): |
| 34 |
| 35 |
| 36 > Let me reconstruct the conversion steps each image goes through in CrOs/Linux
; |
| 37 > [...] |
| 38 > |
| 39 > a) Live video capture produces frames via `V4L2CaptureDelegate::DoCapture()`
[1]. |
| 40 > The original data (from the WebCam) comes in either YUY2 (a 4:2:2 format) or |
| 41 > MJPEG, depending if the capture is smaller than 1280x720p or not, respectivel
y. |
| 42 |
| 43 > b) This `V4L2CaptureDelegate` sends the capture frame to a conversion stage t
o |
| 44 > I420 [2]. I420 is a 4:2:0 format, so it has lost some information |
| 45 > irretrievably. This I420 format is the one used for transporting video frame
s |
| 46 > to the rendered. |
| 47 |
| 48 > c) This I420 is the input to `grabFrame()`, which produces a JS ImageBitmap, |
| 49 > unencoded, after converting the I420 into RGBA [3] of the appropriate endian-
ness. |
| 50 |
| 51 > What happens to `takePhoto()`? It takes the data from the Webcam in a) and |
| 52 > either returns a JPEG Blob [4] or converts the YUY2 [5] and encodes it to PNG |
| 53 > using the default compression value (6 in a 0-10 scale IIRC) [6]. |
| 54 |
| 55 > IOW: |
| 56 |
| 57 ``` |
| 58 - for smaller video resolutions: |
| 59 |
| 60 OS -> YUY2 ---> I420 --> RGBA --> ImageBitmap grabFrame() |
| 61 | |
| 62 +--> RGBA --> PNG ---> Blob takePhoto() |
| 63 |
| 64 - and for larger video resolutions: |
| 65 |
| 66 OS -> MJPEG ---> I420 --> RGBA --> ImageBitmap grabFrame() |
| 67 | |
| 68 +--> JPG ------------> Blob takePhoto() |
| 69 ``` |
| 70 |
| 71 |
| 72 > Where every conversion to-I420 loses information and so does the encoding to |
| 73 > PNG. Even a conversion `RGBA --> I420 --> RGBA` would not produce the origina
l |
| 74 > image. (Plus, when you show `ImageBitmap` and/or Blob on an `<img>` or `<canv
as>` |
| 75 > there are more stages of decoding and even colour correction involved!) |
| 76 |
| 77 > With all that, I'm not surprised at all that the images are not pixel |
| 78 > accurate! :-) |
| 79 |
| 80 |
| 81 ### Why are `PhotoCapabilities.fillLightMode` and `MediaTrackCapabilities.torch`
separated? |
| 82 |
| 83 Because they are different things: `torch` means flash constantly on/off whereas
`fillLightMode` means flash always-on/always-off/auto _when taking a photograph
ic exposure_. |
| 84 |
| 85 `torch` lives in `theTrack` because the effect can be seen "live" on it, whereas
`fillLightMode` lives in `theImageCapture` object because the effect of modifyi
ng it can only be seen after taking a picture. |
| 86 |
| 87 |
| 88 |
| 89 ## Testing |
| 90 |
| 91 Sensors layout tests are located in [`LayoutTests/imagecapture`](https://chromiu
m.googlesource.com/chromium/src/+/master/third_party/WebKit/LayoutTests/imagecap
ture/), [`LayoutTests/fast/imagecapture`](https://chromium.googlesource.com/chro
mium/src/+/master/third_party/WebKit/LayoutTests/fast/imagecapture/) and [`Layou
tTests/external/mediacapture-image`](https://chromium.googlesource.com/chromium/
src/+/master/third_party/WebKit/LayoutTests/external/wpt/mediacapture-image/). |
OLD | NEW |