Chromium Code Reviews
Help | Chromium Project | Gerrit Changes | Sign in
(3)

Issue 2121043002: 16 bpp video stream capture, render and WebGL usage - Realsense R200 & SR300 support.

Can't Edit
Can't Publish+Mail
Start Review
Created:
1 year, 3 months ago by aleksandar.stojiljkovic
Modified:
1 year ago
CC:
Aaron Boodman, abarth-chromium, ben+mojo_chromium.org, cc-bugs_chromium.org, chromium-reviews, darin (slow to review), darin-cc_chromium.org, feature-media-reviews_chromium.org, jam, mcasas+watch+vc_chromium.org, miu+watch_chromium.org, mkwst+moarreviews-renderer_chromium.org, mlamouri+watch-content_chromium.org, posciak+watch_chromium.org, qsr+mojo_chromium.org, viettrungluu+watch_chromium.org, yzshen+watch_chromium.org
Base URL:
https://chromium.googlesource.com/chromium/src.git@master
Target Ref:
refs/pending/heads/master
Project:
chromium
Visibility:
Public.

Description

16 bpp video stream capture, render and WebGL usage - Realsense R200 & SR300 support. It Supports 16 bit depth and infrared video streams from R200 and SR300. Verified to work on Windows 10 and Ubuntu 16.04. Rendering video element: ======================== 16bpp frames are uploaded to RG textures, respectively - for RG textures lower byte goes to 'R' and higher byte to 'G'. Rendered output is original RG texture. Screenshots of demo are available on crbug.com/624436 page. RG88 path required custom bilinear filter in shaders.cc. WebGL: ====== Upload video to texture: In all the textures, R, G, B components have the same value (as luminance). - RGBA UNSIGNED_BYTE: R, G and B components presents upper byte of 16 bit video, so, with precision loss. - RGB/RGBA FLOAT: Normalized 16 bit value in R, G and B component - no precision loss. Getting full 16 bit or float data to javascript: =============================================== 1. on start, create framebuffer, attach (COLOR_ATTACHMENT0) RGBA FLOAT texture to it. 2. In loop, when video is ready, after texImage2D video to the texture, bind the framebuffer and call readpixels to FloatArray. This way the data is accessible from javascript. Complete examples are available on Issue 624436 page. 2D canvas + getImageData returns 8 bit color components (higher bit value) in RGBA. BUG=624436 CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel

Patch Set 1 #

Total comments: 2

Patch Set 2 : 8 bpp support added. R200 camera supported. #

Total comments: 2

Patch Set 3 : bilinear filter for rg88, float and half float added. #

Total comments: 3

Patch Set 4 : WebGL video to texture support and readPixels from R16UI for CPU access #

Total comments: 10

Patch Set 5 : rebase #

Patch Set 6 : gpu memory buffers, reinterpret RG8->R32F on GPU, Linux and ChromeOS support added. #

Total comments: 3

Patch Set 7 : Tests: cc, skcanvas_video_renderer, wrtcrecorder... Fake capture supports Y16. #

Total comments: 15

Patch Set 8 : rebase #

Patch Set 9 : Review #36 fix. Thanks kbr@. #

Patch Set 10 : rebase after 2376293003 land. #

Patch Set 11 : scoping Y8 out. #

Total comments: 2

Patch Set 12 : createImageBitmap layout (content_browsertest) test. #

Patch Set 13 : rebase #

Unified diffs Side-by-side diffs Delta from patch set Stats (+1841 lines, -402 lines) Patch
M cc/BUILD.gn View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +2 lines, -0 lines 0 comments Download
M cc/ipc/cc_param_traits.cc View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +10 lines, -0 lines 0 comments Download
M cc/ipc/cc_param_traits_macros.h View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +7 lines, -0 lines 0 comments Download
M cc/ipc/cc_param_traits_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +5 lines, -0 lines 0 comments Download
M cc/ipc/quads.mojom View 1 2 3 4 5 6 7 2 chunks +7 lines, -0 lines 0 comments Download
M cc/ipc/quads_struct_traits.h View 1 2 3 4 5 6 7 5 chunks +29 lines, -0 lines 0 comments Download
M cc/ipc/quads_struct_traits.cc View 1 2 3 4 5 6 7 2 chunks +19 lines, -0 lines 0 comments Download
M cc/layers/video_layer_impl.cc View 1 2 3 4 5 6 7 2 chunks +16 lines, -0 lines 0 comments Download
M cc/output/ca_layer_overlay.cc View 1 2 3 4 5 6 7 2 chunks +3 lines, -0 lines 0 comments Download
M cc/output/gl_renderer.h View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +10 lines, -0 lines 0 comments Download
M cc/output/gl_renderer.cc View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +79 lines, -0 lines 0 comments Download
M cc/output/gl_renderer_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -0 lines 0 comments Download
M cc/output/renderer_pixeltest.cc View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +138 lines, -0 lines 0 comments Download
M cc/output/shader.h View 1 2 3 4 1 chunk +21 lines, -0 lines 0 comments Download
M cc/output/shader.cc View 1 2 3 4 5 6 7 8 9 1 chunk +52 lines, -0 lines 0 comments Download
M cc/output/software_renderer.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -0 lines 0 comments Download
M cc/quads/draw_quad.h View 2 1 chunk +2 lines, -1 line 0 comments Download
M cc/quads/draw_quad_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +46 lines, -0 lines 0 comments Download
M cc/quads/render_pass.cc View 1 2 3 4 5 6 7 2 chunks +4 lines, -0 lines 0 comments Download
A + cc/quads/y_video_draw_quad.h View 1 2 3 4 5 6 4 chunks +15 lines, -23 lines 0 comments Download
A cc/quads/y_video_draw_quad.cc View 1 2 3 4 5 6 1 chunk +63 lines, -0 lines 0 comments Download
M cc/raster/raster_buffer_provider.cc View 1 2 3 4 5 6 7 8 9 10 11 3 chunks +3 lines, -0 lines 0 comments Download
M cc/resources/platform_color.h View 1 2 3 1 chunk +1 line, -0 lines 0 comments Download
M cc/resources/platform_color_unittest.cc View 1 2 3 1 chunk +1 line, -0 lines 0 comments Download
M cc/resources/resource_format.h View 1 2 3 4 5 6 7 1 chunk +2 lines, -1 line 0 comments Download
M cc/resources/resource_format.cc View 1 2 3 4 5 6 7 8 9 10 6 chunks +7 lines, -0 lines 0 comments Download
M cc/resources/resource_format_utils.cc View 1 2 3 1 chunk +1 line, -0 lines 0 comments Download
M cc/resources/resource_provider.h View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +5 lines, -0 lines 0 comments Download
M cc/resources/resource_provider.cc View 1 2 3 4 5 6 7 8 9 10 11 6 chunks +11 lines, -0 lines 0 comments Download
M cc/resources/video_resource_updater.h View 1 2 3 4 5 6 7 1 chunk +1 line, -0 lines 0 comments Download
M cc/resources/video_resource_updater.cc View 1 2 3 4 5 6 7 8 9 10 11 11 chunks +29 lines, -12 lines 0 comments Download
A cc/test/data/intersecting_light_dark_squares_video.png View 1 2 3 4 5 6 Binary file 0 comments Download
M content/browser/renderer_host/media/video_capture_buffer_pool_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -0 lines 0 comments Download
M content/browser/renderer_host/media/video_capture_controller.cc View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +5 lines, -3 lines 0 comments Download
M content/browser/renderer_host/media/video_capture_controller_event_handler.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -0 lines 0 comments Download
M content/browser/renderer_host/media/video_capture_controller_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 14 chunks +57 lines, -57 lines 0 comments Download
M content/browser/renderer_host/media/video_capture_device_client_unittest.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +2 lines, -1 line 0 comments Download
M content/browser/renderer_host/media/video_capture_host.cc View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +3 lines, -2 lines 0 comments Download
M content/browser/renderer_host/media/video_capture_manager.cc View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +8 lines, -2 lines 0 comments Download
M content/browser/webrtc/webrtc_getusermedia_browsertest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +13 lines, -0 lines 0 comments Download
M content/renderer/media/renderer_gpu_video_accelerator_factories.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -1 line 0 comments Download
M content/renderer/media/renderer_gpu_video_accelerator_factories.cc View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +6 lines, -1 line 0 comments Download
M content/renderer/media/video_capture_impl.cc View 1 2 3 4 5 6 7 8 9 10 11 12 1 chunk +4 lines, -3 lines 0 comments Download
A content/test/data/media/depth_stream_test_utilities.js View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +84 lines, -0 lines 0 comments Download
M content/test/data/media/getusermedia.html View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +16 lines, -0 lines 0 comments Download
M media/base/video_frame.cc View 1 2 3 4 5 6 7 8 9 10 11 5 chunks +28 lines, -14 lines 0 comments Download
M media/capture/BUILD.gn View 1 2 3 4 5 6 7 8 9 10 2 chunks +10 lines, -2 lines 0 comments Download
D media/capture/video/blob_utils.h View 1 2 3 4 5 6 7 8 9 10 1 chunk +0 lines, -23 lines 0 comments Download
D media/capture/video/blob_utils.cc View 1 2 3 4 5 6 7 8 9 10 1 chunk +0 lines, -69 lines 0 comments Download
M media/capture/video/fake_video_capture_device.h View 1 2 3 4 5 6 2 chunks +4 lines, -1 line 0 comments Download
M media/capture/video/fake_video_capture_device.cc View 1 2 3 4 5 6 7 8 9 10 11 12 5 chunks +55 lines, -16 lines 0 comments Download
M media/capture/video/fake_video_capture_device_factory.cc View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +16 lines, -6 lines 0 comments Download
M media/capture/video/fake_video_capture_device_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 12 7 chunks +33 lines, -22 lines 0 comments Download
M media/capture/video/linux/v4l2_capture_delegate.cc View 1 2 3 4 5 6 7 8 9 10 11 12 3 chunks +24 lines, -4 lines 0 comments Download
M media/capture/video/video_capture_device_client.cc View 1 2 3 4 5 6 7 8 9 10 11 6 chunks +46 lines, -12 lines 0 comments Download
M media/capture/video/video_capture_device_unittest.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +19 lines, -0 lines 0 comments Download
A + media/capture/video/video_capture_utils.h View 1 2 3 4 5 6 7 8 9 10 2 chunks +9 lines, -3 lines 0 comments Download
A + media/capture/video/video_capture_utils.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +15 lines, -1 line 0 comments Download
M media/capture/video/win/sink_input_pin_win.cc View 1 2 3 4 5 6 7 8 9 10 11 3 chunks +10 lines, -2 lines 0 comments Download
M media/capture/video/win/video_capture_device_factory_win.cc View 1 2 3 4 1 chunk +3 lines, -1 line 0 comments Download
M media/capture/video/win/video_capture_device_mf_win.cc View 1 2 3 4 5 6 7 8 9 10 2 chunks +5 lines, -0 lines 0 comments Download
M media/capture/video/win/video_capture_device_win.cc View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +4 lines, -1 line 0 comments Download
M media/renderers/gpu_video_accelerator_factories.h View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +4 lines, -3 lines 0 comments Download
M media/renderers/mock_gpu_video_accelerator_factories.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -1 line 0 comments Download
M media/renderers/skcanvas_video_renderer.h View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +56 lines, -1 line 0 comments Download
M media/renderers/skcanvas_video_renderer.cc View 1 2 3 4 5 6 7 8 9 10 11 9 chunks +356 lines, -11 lines 0 comments Download
M media/renderers/skcanvas_video_renderer_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 3 chunks +58 lines, -0 lines 0 comments Download
M media/video/gpu_memory_buffer_video_frame_pool.cc View 1 2 3 4 5 6 7 8 9 10 11 23 chunks +97 lines, -39 lines 0 comments Download
M media/video/gpu_memory_buffer_video_frame_pool_unittest.cc View 1 2 3 4 5 6 7 8 9 10 11 4 chunks +48 lines, -1 line 0 comments Download
M third_party/WebKit/Source/core/html/HTMLVideoElement.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +14 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/core/html/HTMLVideoElement.cpp View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +19 lines, -2 lines 0 comments Download
M third_party/WebKit/Source/modules/webgl/WebGLRenderingContextBase.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -0 lines 0 comments Download
M third_party/WebKit/Source/modules/webgl/WebGLRenderingContextBase.cpp View 1 2 3 4 5 6 7 8 9 10 11 12 11 chunks +80 lines, -48 lines 0 comments Download
M third_party/WebKit/Source/platform/graphics/ImageBuffer.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +2 lines, -0 lines 0 comments Download
M third_party/WebKit/Source/platform/graphics/ImageBuffer.cpp View 1 2 3 4 5 6 7 8 9 10 11 12 2 chunks +0 lines, -5 lines 0 comments Download
M third_party/WebKit/Source/platform/graphics/gpu/DrawingBuffer.cpp View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +1 line, -1 line 0 comments Download
M third_party/WebKit/Source/platform/graphics/gpu/Extensions3DUtil.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +4 lines, -4 lines 0 comments Download
M third_party/WebKit/Source/platform/graphics/gpu/Extensions3DUtil.cpp View 1 2 3 4 5 6 7 8 9 10 11 2 chunks +10 lines, -2 lines 0 comments Download
M third_party/WebKit/public/platform/WebMediaPlayer.h View 1 2 3 4 5 6 7 8 9 10 11 1 chunk +17 lines, -0 lines 0 comments Download
Commit queue not available (can’t edit this change).

Messages

Total messages: 60 (31 generated)
aleksandar.stojiljkovic
Patch 1 is fully functioning proof of concept code. Continuing work on it but early ...
1 year, 3 months ago (2016-07-04 18:09:55 UTC) #3
dshwang
nice progress https://codereview.chromium.org/2121043002/diff/1/cc/layers/video_layer_impl.cc File cc/layers/video_layer_impl.cc (right): https://codereview.chromium.org/2121043002/diff/1/cc/layers/video_layer_impl.cc#newcode323 cc/layers/video_layer_impl.cc:323: YVideoDrawQuad* y_video_quad = IMO, existing TextureDrawQuad can ...
1 year, 3 months ago (2016-07-04 18:23:17 UTC) #4
aleksandar.stojiljkovic
patchset 2: 8 bpp and Realsense R200 support added. Works with depth and infrared device ...
1 year, 3 months ago (2016-07-05 19:54:18 UTC) #7
aleksandar.stojiljkovic
https://codereview.chromium.org/2121043002/diff/20001/content/browser/renderer_host/media/video_capture_device_client.cc File content/browser/renderer_host/media/video_capture_device_client.cc (right): https://codereview.chromium.org/2121043002/diff/20001/content/browser/renderer_host/media/video_capture_device_client.cc#newcode172 content/browser/renderer_host/media/video_capture_device_client.cc:172: memcpy(buffer->data(), data, length); normalized unsigned short -| half_float conversion ...
1 year, 3 months ago (2016-07-11 22:15:14 UTC) #8
aleksandar.stojiljkovic
Patch #3: bilinear filter for rg88, float and half float added. Patch is used for ...
1 year, 3 months ago (2016-07-15 12:29:06 UTC) #10
aleksandar.stojiljkovic
+zmo (to review approach in third_party/WebKit/Source/modules/webgl, Video skcanvas_video_renderer.cc and VideoMediaPlayer(MS) added in this patch). The ...
1 year, 2 months ago (2016-08-15 21:10:23 UTC) #13
Ken Russell (switch to Gerrit)
It's encouraging to see your progress on this. A few questions. https://codereview.chromium.org/2121043002/diff/60001/cc/resources/resource_provider.cc File cc/resources/resource_provider.cc (right): ...
1 year, 2 months ago (2016-08-16 00:22:27 UTC) #15
aleksandar.stojiljkovic
On 2016/08/16 00:22:27, Ken Russell wrote: > It's encouraging to see your progress on this. ...
1 year, 2 months ago (2016-08-16 12:02:31 UTC) #16
Ken Russell (switch to Gerrit)
https://codereview.chromium.org/2121043002/diff/60001/media/renderers/skcanvas_video_renderer.cc File media/renderers/skcanvas_video_renderer.cc (right): https://codereview.chromium.org/2121043002/diff/60001/media/renderers/skcanvas_video_renderer.cc#newcode775 media/renderers/skcanvas_video_renderer.cc:775: data = frame->visible_data(0); On 2016/08/16 12:02:31, aleksandar.stojiljkovic wrote: > ...
1 year, 2 months ago (2016-08-16 18:27:37 UTC) #17
aleksandar.stojiljkovic
On 2016/08/16 18:27:37, Ken Russell wrote: > https://codereview.chromium.org/2121043002/diff/60001/media/renderers/skcanvas_video_renderer.cc > File media/renderers/skcanvas_video_renderer.cc (right): > > https://codereview.chromium.org/2121043002/diff/60001/media/renderers/skcanvas_video_renderer.cc#newcode775 ...
1 year, 2 months ago (2016-08-18 22:29:14 UTC) #21
aleksandar.stojiljkovic
Patch Set 6 : gpu memory buffers support, reinterpret RG8->R32F on GPU, Linux and ChromeOS ...
1 year, 1 month ago (2016-09-20 12:22:55 UTC) #26
enne (OOO)
Where is the testing for this feature? I would have expected to see a layout ...
1 year, 1 month ago (2016-09-20 19:03:45 UTC) #27
aleksandar.stojiljkovic
On 2016/09/20 19:03:45, enne wrote: > Where is the testing for this feature? I would ...
1 year, 1 month ago (2016-09-20 20:04:06 UTC) #28
enne (OOO)
Ok! Sorry, I misunderstood your previous message. It looked like you were CCing a bunch ...
1 year, 1 month ago (2016-09-20 20:12:05 UTC) #29
jbauman
https://codereview.chromium.org/2121043002/diff/100001/media/renderers/skcanvas_video_renderer.cc File media/renderers/skcanvas_video_renderer.cc (right): https://codereview.chromium.org/2121043002/diff/100001/media/renderers/skcanvas_video_renderer.cc#newcode951 media/renderers/skcanvas_video_renderer.cc:951: rg8_to_red_program_ = gl->CreateProgram(); I don't think the lifetime of ...
1 year, 1 month ago (2016-09-20 20:51:22 UTC) #30
aleksandar.stojiljkovic
On 2016/09/20 20:51:22, jbauman wrote: > https://codereview.chromium.org/2121043002/diff/100001/media/renderers/skcanvas_video_renderer.cc > File media/renderers/skcanvas_video_renderer.cc (right): > > https://codereview.chromium.org/2121043002/diff/100001/media/renderers/skcanvas_video_renderer.cc#newcode951 > ...
1 year, 1 month ago (2016-09-20 21:30:08 UTC) #31
aleksandar.stojiljkovic
Patch Set 7 : Tests: cc, skcanvas_video_renderer, wrtcrecorder... Fake capture supports Y16 The patch adds ...
1 year ago (2016-09-30 14:26:32 UTC) #35
Ken Russell (switch to Gerrit)
There were quite a few issues in the command buffer changes so I stopped after ...
1 year ago (2016-09-30 23:01:30 UTC) #36
aleksandar.stojiljkovic
Patch Set 9 : Review #36 fix. Thanks kbr@. Resolved by reverting the code. Decided ...
1 year ago (2016-10-03 10:56:53 UTC) #37
Ken Russell (switch to Gerrit)
Thanks for revising this patch. I only reviewed a few parts; gpu/command_buffer, ui/gl, and third_party/WebKit; ...
1 year ago (2016-10-05 01:27:28 UTC) #38
aleksandar.stojiljkovic
Patch Set 10 : rebase after 2376293003 land. On 2016/10/05 01:27:28, Ken Russell wrote: > ...
1 year ago (2016-10-05 03:57:33 UTC) #39
aleksandar.stojiljkovic
Capture (content&media) patch: https://codereview.chromium.org/2398463003 On 2016/10/05 03:57:33, aleksandar.stojiljkovic wrote: > Patch Set 10 : rebase ...
1 year ago (2016-10-05 04:08:00 UTC) #40
Ken Russell (switch to Gerrit)
On 2016/10/05 04:08:00, aleksandar.stojiljkovic wrote: > Capture (content&media) patch: https://codereview.chromium.org/2398463003 > > On 2016/10/05 03:57:33, ...
1 year ago (2016-10-05 19:38:40 UTC) #41
aleksandar.stojiljkovic
Patch Set 11 : scoping Y8 out. On 2016/10/05 19:38:40, Ken Russell wrote: > On ...
1 year ago (2016-10-07 15:10:05 UTC) #43
Daniele Castagna
https://codereview.chromium.org/2121043002/diff/200001/media/video/gpu_memory_buffer_video_frame_pool.cc File media/video/gpu_memory_buffer_video_frame_pool.cc (right): https://codereview.chromium.org/2121043002/diff/200001/media/video/gpu_memory_buffer_video_frame_pool.cc#newcode162 media/video/gpu_memory_buffer_video_frame_pool.cc:162: VideoPixelFormat output_format_; This is changing slightly with crrev.com/2382443008 and ...
1 year ago (2016-10-10 18:42:31 UTC) #46
aleksandar.stojiljkovic
https://codereview.chromium.org/2121043002/diff/200001/media/video/gpu_memory_buffer_video_frame_pool.cc File media/video/gpu_memory_buffer_video_frame_pool.cc (right): https://codereview.chromium.org/2121043002/diff/200001/media/video/gpu_memory_buffer_video_frame_pool.cc#newcode162 media/video/gpu_memory_buffer_video_frame_pool.cc:162: VideoPixelFormat output_format_; On 2016/10/10 18:42:31, Daniele Castagna wrote: > ...
1 year ago (2016-10-11 18:29:54 UTC) #47
aleksandar.stojiljkovic
ccameron@'s non-lgtm on misusing RG88 for 16 bit plane in YUV : https://crrev.com/2122573003 ccameron@ raised ...
1 year ago (2016-10-18 20:45:33 UTC) #48
Ken Russell (switch to Gerrit)
On 2016/10/18 20:45:33, aleksandar.stojiljkovic wrote: > ccameron@'s non-lgtm on misusing RG88 for 16 bit plane ...
1 year ago (2016-10-20 00:47:29 UTC) #59
aleksandar.stojiljkovic
1 year ago (2016-10-20 21:37:06 UTC) #60
On 2016/10/20 00:47:29, Ken Russell wrote:
> On 2016/10/18 20:45:33, aleksandar.stojiljkovic wrote:
> > ccameron@'s non-lgtm on misusing RG88 for 16 bit plane in YUV :
> > https://crrev.com/2122573003
> > 
> > ccameron@ raised important concern affecting this and split patches too:
> > 
> > On 2016/10/17 22:28:31, ccameron wrote:
> > > This approach does not lgtm. We should not simulate 1=channel images using
> > > 2-channel images.
> > > 
> > > It is possible to create R16 unorm GMBs. We should use these. I verified
> this
> > on
> > > Mac with IOSurfaces, and I can send you a test application which
> demonstrates
> > > this.
> > > 
> > > Alternatively, unless we prove that the short<->half-float conversion is
the
> > > critical bottleneck for a critical use-case, then we should consider using
> > R16F
> > > (half-float single channel).
> > 
> > Could you please clarify the concern, why it is not ok to use RG88 for 16
> > bit plane and in which cases it could be considered as a hack?
> > Is is about lower layers loosing the semantics or the need for custom
bilinear
> > filtering?
> > 
> > Measurements: 
> > Did some measurements here, comparing CPU, GPU power consumption for
> conversion,
> > glTexImage and rendering for float, half_float and RG88.
> > https://bugs.chromium.org/p/chromium/issues/detail?id=624436#c28
> > 
> > And the conclusion was that RG88 used for Depth16 internally only is
> acceptable
> > (hack) if we hide this as implementation detail from end user...
> > End user shouldn't see the value split to color components. So, this patch
is
> > handling that, too,... I mean converting RG88 to float on GPU in
> > SkCanvasVideoRenderer or doing it on CPU when shared memory data gets
> texImage2D
> > to float texture.
> > 
> > In a way, float would be preferred as users could use WebGL1 with no
precision
> > loss to get the data available on GPU (FLOAT RGBA texture) or CPU (from
> > javascript accessible through gl.readPixels from texture color-attached to
> > framebuffer).
> > 
> > IOSurface R16 or R16UI is not usable here - OSX is not a target platform now
> for
> > Realsense cameras or Kinect2 camera. In later plases, once libusb backend is
> > added to capture, we could use it.
> > 
> > If using RG88 for 16 bit depth is not fine, this patch would get simplified
> and
> > implemented on the same way on all of the target platforms (Win, ChromeOS,
> > Linux) using shared memory CPU buffers. I'll do it that way than - split
this
> > patch to minimal support (using CPU shared memory) and consider GPU buffer
> > format as an optimization for patches.
> > Thanks ccameron@.
> 
> I'd like to focus on the highest performance path. If developers can keep all
of
> the data on the GPU by using RG88 textures then I think we should enable that
> possibility. I pushed back on ccameron's review on
> https://codereview.chromium.org/2122573003 . Let's see what the resolution is.

Thanks a lot.
In order to get the patch reviewed faster, I've split the CPU/shared memory
support and tests here:
https://crrev.com/2428263004: 16 bpp video stream capture, render and WebGL
usage using (CPU) shared memory buffers

There is significantly less code to review than here and the code is simpler.
Since there is no GPU buffer support anyway on Windows, I hope we can get the
code in sooner with https://crrev.com/2428263004.
The plan is to to rebase this one later to it.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld 81bcdb8aa