Add support for Tegra V4L2 VDA

content/common/gpu/media/v4l2_video_decode_accelerator.cc

 // Copyright 2014 The Chromium Authors. All rights reserved.
 // Use of this source code is governed by a BSD-style license that can be
 // found in the LICENSE file.
 
 #include <dlfcn.h>
 #include <errno.h>
 #include <fcntl.h>
 #include <libdrm/drm_fourcc.h>
 #include <linux/videodev2.h>
 #include <poll.h>
@@ skipping 352 matching lines @@
   attrs[5] = DRM_FORMAT_NV12;
   for (size_t i = 0; i < output_buffer_map_.size(); ++i) {
     DCHECK(buffers[i].size() == frame_buffer_size_);
     OutputRecord& output_record = output_buffer_map_[i];
     attrs[7]  = output_record.fds[0];
     attrs[9]  = 0;
     attrs[11] = frame_buffer_size_.width();
     attrs[13] = output_record.fds[1];
     attrs[15] = 0;
     attrs[17] = frame_buffer_size_.width();
-    EGLImageKHR egl_image = eglCreateImageKHR(
-        egl_display_, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, NULL, attrs);
+    EGLImageKHR egl_image = device_->CreateEGLImage(
+        egl_display_, attrs, buffers[i].texture_id(), i);
     if (egl_image == EGL_NO_IMAGE_KHR) {
       DLOG(ERROR) << "AssignPictureBuffers(): could not create EGLImageKHR";

[Ami GONE FROM CHROMIUM, 2014/02/07 09:09:30]

already logged in the Device method; unnecessary

       NOTIFY_ERROR(PLATFORM_FAILURE);
       return;
     }
 
-    glBindTexture(GL_TEXTURE_EXTERNAL_OES, buffers[i].texture_id());
-    glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, egl_image);

[Pawel Osciak, 2014/02/10 06:36:17]

I would like to understand the big picture here please.

We strive to stay as close as possible to using (and/or creating)
platform-independent standards where we can, like the sequence above, instead of
providing custom calls for each platform. Removing this from here and TVDA is a
step into an opposite direction, and I would like to understand what technical
difficulties force us to do this first.

Binding textures to EGLImages also serves to keep track of ownership. There are
multiple users of the shared buffer, the renderer, the GPU that renders the
textures, this class and the HW codec. How is ownership/destruction managed and
how is it ensured that the buffer is valid while any of the users are still
referring to/using it (both in userspace and in kernel)?

What happens if the renderer crashes and the codec is writing to the textures?
What happens when this class is destroyed, but the texture is in the renderer?
What happens when the whole Chrome crashes, but the HW codec is using a buffer
(i.e. kernel has ownership)?

Could you please explain how is ownership managed for shared buffers on Tegra?

[shivdasp, 2014/02/10 13:31:17]

The decoder's output buffers are created when REQBUFS(x) is called on
CAPTURE_PLANE. These buffers are hardware buffers which can be shared with the
AVP processor for decoder to write into.
Then V4L2VDA triggers ProvidePictureBuffers() on which textures are created and
sent back in AssignPictureBuffers().
Now V4L2VDA creates EglImages from these textures and sends each EglImage handle
to library using the QUERYBUF (but can use a custom call too). The tegrav4l2
library cannot create EglImages from DMABUFS like in Exynos since there is no
such extension. We create EglImage from this texture itself so there is a
binding between texture and eglImage.
Now when this EglImage is sent to libtegrav4l2, it is mapped with the
corresponding decoder buffer created in REQBUF() call.
This way there is one map of EglImage, texture and decoder buffer.
When any buffer is enqueued in QBUF, the library sends it down to the decoder.
Once the decoder buffer is ready, the library uses graphics apis to populate the
corresponding EglImage with the RGB data and then pushes into a queue thereby
making it available for DQBUF after which this buffer can be used only when it
is back in QBUF call.
This way the buffer ownership is managed.
So in summary the library uses queues and does all the buffer management between
decoder and the graphics stack for conversion.

On 2014/02/10 06:36:17, Pawel Osciak wrote:

[Pawel Osciak, 2014/02/12 09:15:13]

By decoder do you mean V4L2VDA class?
Sounds like the eglCreateImage extension taking offsets I described in the
comment in tegra_v4l2_video_device.cc could work for this?
My understanding is you mean the buffer is bound to a texture? If so, then it
also seems like we could use the current bind texture to eglimage calls?
What happens when this class calls REQBUFS(0), but the corresponding textures
are being rendered to the screen?
How will the buffers be freed if the GPU process crashes without calling
REQBUFS(0)?
What happens when the bound textures are deleted, but the HW codec is still
using them?

[shivdasp, 2014/02/12 10:11:55]

No I meant the decoder entity within the library.
Unfortunately there is no such extension today.
The libtegrav4l2 talks to another internal library which actually creates the
YUV buffer. This is what is given to the AVP and where the decoded output is
actually filled.
There is a corresponding RGB buffer created when the EGLImage is called, this is
owned by the graphics library. While enqueuing buffers for CAPTURE PLANE, there
is a conversion performed to do YUV to RGB.
I guess I am missing something here. I did not understand "REQBUFS(0) is called
but corresponding textures are being rendered ?". Doesn't DestroyOutputBuffers()
call guarantee that buffers on CAPTURE plane are no longer used.
I will confirm about the buffer freeing in gpu process crash scenario.
The last scenario (bound texture are deleted but HW codec is still using them)
is taken care by the conversion step performed using the library. The texture is
bound to the EGlImage. So that binding will fail. Since the libtegrav4l2 has the
EglImage backed by a RGB buffer the conversion can happen. How can I test this
scenario ?

[Pawel Osciak, 2014/02/13 10:42:54]

So the YUV buffers are tied to the textures somehow?
The underlying memory can still be used as textures in the client of VDA class.
It only guarantees that they are not used anymore by the codec class as
v4l2_buffers.
Thanks.
This is just a case where there is a bug in the code, but my point is that the
ownership should be shared with the kernel as well, so if the userspace (Chrome)
dies, the kernel will properly clean up.

[shivdasp, 2014/02/14 03:06:45]

We send texture_id to eglCreateImageKHR and bind it there. And eglImage is sent
to the library which maps it to its YUV buffer.
My subsequent patch will probably make this clearer.
If the EGLimage is destroyed I think the texture becomes unbound. I was
debugging some scenario and I get errors as "texture not bound or texture id 0"
kind of errors from gles2_cmd_decoder.cc. These I guess represent this crash
scenario. So it is taken care of already while validating the texture before
rendering ?
And I observe similar kind of logs on Exynos too.
Do you have a test case or steps of validating this ? Will killing gpu process
while video playback validate this path ?

[Pawel Osciak, 2014/02/14 07:36:10]

Wait, where do you send texture_id to eglCreateImageKHR? I don't see that in the
code above.
Do you have an extension for eglCreateImageKHR to also accept texture ids and
bind during creation? Why not do this in the standard way, i.e. by using
GL_OES_EGL_image_external? I would expect your EGL implementation already has it
for other things (and it's an extension created by NVIDIA too)...
Or did you mean some other function?
If you are getting those errors, then there is definitely something wrong going
on.
That is even more worrying. Could you please submit a bug for Exynos with repro
steps?
It should.

[shivdasp, 2014/02/14 09:18:58]

The texture_id is sent in eglCreateImageKHR parameter. See TegraV4L2Device
implementation of CreateEGLImage().

I will submit a bug with my findings and repro steps.

     picture_buffers_ref->picture_buffers.push_back(
         PictureBufferArrayRef::PictureBufferRef(egl_image, buffers[i].id()));
   }
   decoder_thread_.message_loop()->PostTask(
       FROM_HERE,
       base::Bind(&V4L2VideoDecodeAccelerator::AssignPictureBuffersTask,
                  base::Unretained(this),
                  base::Passed(&picture_buffers_ref)));
 }
 
@@ skipping 1240 matching lines @@
   }
 }
 
 bool V4L2VideoDecodeAccelerator::GetFormatInfo(struct v4l2_format* format,
                                                  bool* again) {
   DCHECK_EQ(decoder_thread_.message_loop(), base::MessageLoop::current());
 
   *again = false;
   memset(format, 0, sizeof(*format));
   format->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
-  if (HANDLE_EINTR(device_->Ioctl(VIDIOC_G_FMT, format)) != 0) {
+
+  if (HANDLE_EINTR(device_->Ioctl(VIDIOC_G_FMT, format) != 0)) {

[Ami GONE FROM CHROMIUM, 2014/02/07 09:09:30]

This is a bug!

     if (errno == EINVAL) {
       // EINVAL means we haven't seen sufficient stream to decode the format.
       *again = true;
       return true;
     } else {
       DPLOG(ERROR) << "DecodeBufferInitial(): ioctl() failed: VIDIOC_G_FMT";
       NOTIFY_ERROR(PLATFORM_FAILURE);
       return false;
     }
   }
 
+  // Since the underlying library at the moment is not updated this hack

[Ami GONE FROM CHROMIUM, 2014/02/07 09:09:30]

This comment is opaque to me.
Also, if something is a hack, usually there should be a TODO/crbug to go along
with it.

[Pawel Osciak, 2014/02/10 06:36:17]

Could this hack be moved into the library itself then?
This class should not have any awareness of device-specific issues.

[shivdasp, 2014/02/10 13:31:17]

Apologies. This code was not meant to be here. Will remove it.
On 2014/02/10 06:36:17, Pawel Osciak wrote:

+  LOG(ERROR) << "Hardcoding the values here ";
+  format->fmt.pix_mp.num_planes = 2;
+  format->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_NV12M;
+
   return true;
 }
 
 bool V4L2VideoDecodeAccelerator::CreateBuffersForFormat(
     const struct v4l2_format& format) {
   DCHECK_EQ(decoder_thread_.message_loop(), base::MessageLoop::current());
   CHECK_EQ(format.fmt.pix_mp.num_planes, 2);
   frame_buffer_size_.SetSize(
       format.fmt.pix_mp.width, format.fmt.pix_mp.height);
   output_buffer_pixelformat_ = format.fmt.pix_mp.pixelformat;
@@ skipping 113 matching lines @@
 
   DVLOG(3) << "CreateOutputBuffers(): ProvidePictureBuffers(): "
            << "buffer_count=" << output_buffer_map_.size()
            << ", width=" << frame_buffer_size_.width()
            << ", height=" << frame_buffer_size_.height();
   child_message_loop_proxy_->PostTask(FROM_HERE,
                                       base::Bind(&Client::ProvidePictureBuffers,
                                                  client_,
                                                  output_buffer_map_.size(),
                                                  frame_buffer_size_,
-                                                 GL_TEXTURE_EXTERNAL_OES));
+                                                 device_->GetTextureTarget()));
 
   return true;
 }
 
 void V4L2VideoDecodeAccelerator::DestroyInputBuffers() {
   DVLOG(3) << "DestroyInputBuffers()";
   DCHECK(child_message_loop_proxy_->BelongsToCurrentThread());
   DCHECK(!input_streamon_);
 
   for (size_t i = 0; i < input_buffer_map_.size(); ++i) {
@@ skipping 119 matching lines @@
 
 void V4L2VideoDecodeAccelerator::PictureCleared() {
   DVLOG(3) << "PictureCleared(). clearing count=" << picture_clearing_count_;
   DCHECK_EQ(decoder_thread_.message_loop(), base::MessageLoop::current());
   DCHECK_GT(picture_clearing_count_, 0);
   picture_clearing_count_--;
   SendPictureReady();
 }
 
 }  // namespace content