New issue
Advanced search Search tips

Issue 850851 link

Starred by 4 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Implement a V4L2 virtual codec driver

Project Member Reported by tfiga@chromium.org, Jun 8 2018

Issue description

This is for tracking a V4L2 virtual codec driver. The idea is similar to existing upstream vivid and vim2m drivers, but implementing V4L2 Codec API.
 

Comment 1 by tfiga@chromium.org, Jun 8 2018

Description: Show this description
Status: Started (was: Assigned)
Cc: hiroh@chromium.org tfiga@chromium.org
As the first step, I tried to run Hans's virtual codec driver[1] on betty.
This driver supports a fake codec called "Fast Walsh Hadamard Transform" (FWHT).
By applying patch sets related to crrev.com/c/1125889 , it worked on betty.

To test vicodec device easily, I also prepared an ebuild file for latest v4l-utils with some patches by crrev.com/c/1127560 and crrev.com/c/1127572.

After applying them, I could test vicodec by the following way:
# on betty VM
$ modprobe vicodec
# Generate test image
$ v4l2-ctl --stream-mmap --stream-out-mmap --stream-to-hdr out.comp --stream-count=100 --stream-out-pattern=4 --stream-out-hor-speed=1 --stream-out-vert-speed=1
# Decode out.cmp to out.yuv by vicodec
$ v4l2-ctl -d1 --stream-out-mmap --stream-mmap --stream-from-hdr out.comp --stream-to out.yuv

Then, I could play out.yuv as a normal YUV video on my workstation.
$ mplayer -demuxer rawvideo -rawvideo format=i420:w=1280:h=720:fps=20 -loop 0 out.yuv

I think the next step is to add a test case in VDA unittest for FWHT videos.

[1] https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=vicodec
We need to implement multi-planar APIs in Hans's vicodec driver since Chrome requres them.

The required APIs are listed here:
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/planar-apis.html#calls-that-distinguish-between-single-and-multi-planar-apis
I added multi-planar API support to Hans's vicodec driver in [1].
Then, with some ad-hoc hacks in Chrome [2], I succeeded to run (not pass) video_decode_accelerator_unittest on betty with vicodec drivers by the following way:

```
$ stop ui
$ modprobe vicodec multiplanar
$ ln -s /dev/video1 /dev/video-dec
$ v4l2-ctl --stream-mmap --stream-out-mmap --stream-to-hdr test-20fps.fwht --stream-count=100 --stream-out-pattern=4 --stream-out-hor-speed=1 --stream-out-vert-speed=1
$ video_decode_accelerator_unittest --disable_rendering --disable_ozone --single-process-tests --test_video_data=test-20fps.fwht:320:240:250:258:35:150:-1 --vmodule=*=4
```

Currently, it passed 5 cases and failed on 17 cases.
To make it pass them in correct way (not in hacky way), at least, we need the followings:
(i) Define constants like |FWHTPROFILE_ANY| and |kCodecFWHT|
  - For this, we might have to change a lot of files. For example, the CL adding AV1 decoder[3] changed 25+ files.
(ii) Write correct handler in EncodedDataHelper::GetBytesForNextData [4] for FWHT video file
  - At least, it's necessary to parse the frame header defined in [5]
(iii) Correctly determine which format is supported in GenericV4L2Device::CanCreateEGLImageFrom[6]

Also, some changes in video_decode_accelerator_unittest might be needed.

---

At first, I wondered if we can use this FWHT virtual driver on Chrome waterfall as the first step.
Now, however, I don't think that it is worth doing because the changes would be too large, especially (i) above.
Since what we really want is not FWHT virtual driver but H264/VPx driver, these changes will be revereted finally.
Thus, it might be better to give up using FWHT virtual driver on Chrome waterwall and develop a virtual driver with fake decoder (or user-space codec daemon) for real codec, H264 and VPx, as originally planned.
Even in this case, we can develop it based on Hans's virtual codec driver.

In addition, video_decode_accelerator_unittest is so large that it is difficult to change it. Fortunately, we have a plan to create new VDA integration tests. So, I think it is also possible to make a virtual driver work with the new one instead of the current video_decode_accelerator_unittest.

So, I think I shouldn't spend much time for this issue for now.
After Hans makes his virtual codec driver stable and new VDA integration tests start working, I will be back.

[1] Support multi-planar API: crrev.com/c/1136459 (I'm planning to send this patch to Hans)
[2] Ad-hoc patch for VDA unittest: crrev.com/c/1141749
[3] CL for adding AV1: crrev.com/c/762359
[4] https://chromium-review.googlesource.com/c/chromium/src/+/1141749/2/media/gpu/test/video_decode_accelerator_unittest_helpers.cc#162
[5] crrev.com/c/1127572
[6] https://chromium-review.googlesource.com/c/chromium/src/+/1141749/2/media/gpu/v4l2/generic_v4l2_device.cc#205

Sign in to add a comment