New issue
Advanced search Search tips

Issue 853185 link

Starred by 3 users
Status: Started
Owner:
sprang@chromium.org
Cc:
posciak@chromium.org
ilnik@chromium.org
mcasas@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature



Sign in to add a comment

Add support for VP8 temporal layers in vaapi video encode accelerator

Project Member Reported by sprang@chromium.org, Jun 15 2018 
For the WebRTC use-case, we want the vaapi-based VP8 encode acceleration to be able to support temporal layers for scalability and resilience.

This will require several things:
* Signaling to the accelerator how many temporal layers to use, and how the bitrate should be allocated across temporal layers.
* Add functionality to actually configure the encoder with appropriate temporal layer settings and reference buffer flags, etc.
* Add metadata to the encoded buffers, so that rtp packetization can correctly mark the packets.

See also https://bugs.chromium.org/p/chromium/issues/detail?id=794608

Comment 1 by xhw...@chromium.org, Jun 18 2018

Components: -Internals>Media>Hardware -Internals>Media -Internals>Media>Video
Project Member

Comment 2 by bugdroid1@chromium.org, Jun 29 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f169a5709a02da5e6dcbbbe3d9abd294b48120af

commit f169a5709a02da5e6dcbbbe3d9abd294b48120af
Author: Erik Språng <sprang@chromium.org>
Date: Fri Jun 29 08:44:29 2018

Add VideBitrateAllocation support to MojoVideoEncodeAccelerator.

The bitrate allocation class needs to be serialized in order to be
passed over ipc.
Also optimize the VideoBitrateAllocation to make it cheaper to get the
sum, which is now being called in more and more places.

Bug: chromium:853185
Change-Id: Id61fcfa5c5fe448cea0b1fef3283f3c414c7c5a4
Reviewed-on: https://chromium-review.googlesource.com/1102470
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Commit-Queue: Erik Språng <sprang@chromium.org>
Cr-Commit-Position: refs/heads/master@{#571428}
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/base/video_bitrate_allocation.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/base/video_bitrate_allocation.h
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/clients/mojo_video_encode_accelerator.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/clients/mojo_video_encode_accelerator.h
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/clients/mojo_video_encode_accelerator_unittest.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/interfaces/video_encode_accelerator.mojom
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/interfaces/video_encode_accelerator.typemap
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/interfaces/video_encode_accelerator_typemap_traits.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/interfaces/video_encode_accelerator_typemap_traits.h
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/services/mojo_video_encode_accelerator_service.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/services/mojo_video_encode_accelerator_service.h
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/services/mojo_video_encode_accelerator_service_unittest.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/mojo/test/mojo_video_encode_accelerator_integration_test.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/video/fake_video_encode_accelerator.cc
[modify] https://crrev.com/f169a5709a02da5e6dcbbbe3d9abd294b48120af/media/video/fake_video_encode_accelerator.h
Project Member

Comment 3 by bugdroid1@chromium.org, Jul 10 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e24828d26cccbcac5a51d17f58c8d4d256cacd46

commit e24828d26cccbcac5a51d17f58c8d4d256cacd46
Author: Erik Språng <sprang@chromium.org>
Date: Tue Jul 10 11:33:15 2018

Add BitstreamBufferMetadata struct and add vp8-specific part.

This CL refactors the signature for
media::VideoEncodeAccelerator::Client::BitstreamBufferReady()
Instead of adding several new parameters needed for VP8 temporal layer
support, I merged them into a struct so we can more easily update it in
the future.

Bug: chromium:853185
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I93e8a12537c3bbfc598f5609439a66cd0026ab59
Reviewed-on: https://chromium-review.googlesource.com/1107982
Commit-Queue: Erik Språng <sprang@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Emircan Uysaler <emircan@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Reviewed-by: Lambros Lambrou <lambroslambrou@chromium.org>
Cr-Commit-Position: refs/heads/master@{#573682}
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/components/arc/video_accelerator/gpu_arc_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/components/arc/video_accelerator/gpu_arc_video_encode_accelerator.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/media/webrtc/rtc_video_encoder.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/media/webrtc/rtc_video_encoder_unittest.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/media_recorder/vea_encoder.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/media_recorder/vea_encoder.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/pepper/pepper_video_encoder_host.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/pepper/pepper_video_encoder_host.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/content/renderer/pepper/video_encoder_shim.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/cast/sender/external_video_encoder.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/android/android_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/v4l2/v4l2_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/vaapi/accelerated_video_encoder.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/vaapi/accelerated_video_encoder.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/vaapi/vaapi_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/video_encode_accelerator_unittest.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/vt_video_encode_accelerator_mac.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/gpu/windows/media_foundation_video_encode_accelerator_win.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/clients/mojo_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/clients/mojo_video_encode_accelerator_unittest.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/interfaces/video_encode_accelerator.mojom
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/interfaces/video_encode_accelerator.typemap
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/interfaces/video_encode_accelerator_typemap_traits.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/interfaces/video_encode_accelerator_typemap_traits.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/services/mojo_video_encode_accelerator_service.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/services/mojo_video_encode_accelerator_service.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/services/mojo_video_encode_accelerator_service_unittest.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/mojo/test/mojo_video_encode_accelerator_integration_test.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/video/fake_video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/video/video_encode_accelerator.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/media/video/video_encode_accelerator.h
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/remoting/codec/webrtc_video_encoder_gpu.cc
[modify] https://crrev.com/e24828d26cccbcac5a51d17f58c8d4d256cacd46/remoting/codec/webrtc_video_encoder_gpu.h
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 17 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8413c2fc42156ec7ec7c00c258382978795ec79c

commit 8413c2fc42156ec7ec7c00c258382978795ec79c
Author: Chih-Yu Huang <akahuang@google.com>
Date: Tue Jul 17 03:22:05 2018

v4l2_vea: Fix timestamp of BitstreamMetadata.

Time::FromTimeVal() method will add the offset of UNIX epoch,
and breaks the timestamp check at video_encode_accelerator_unittest.
This CL change it to Time::FromMicroseconds().

BUG=chromium:853185
TEST=pass video_encode_accelerator_unittest at Kevin device

Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: Ia13027141062015f904be74b9323a988697fea3a
Reviewed-on: https://chromium-review.googlesource.com/1133612
Commit-Queue: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Hirokazu Honda <hiroh@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Commit-Position: refs/heads/master@{#575537}
[modify] https://crrev.com/8413c2fc42156ec7ec7c00c258382978795ec79c/media/gpu/v4l2/v4l2_video_encode_accelerator.cc
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 3 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f5420d17fdf394d32cae5d391613e7dbcc6acd3d

commit f5420d17fdf394d32cae5d391613e7dbcc6acd3d
Author: Erik Språng <sprang@chromium.org>
Date: Mon Sep 03 09:09:40 2018

Add support for configuring content type for VideoEncodeAccelerators

Bug: chromium:853185
Change-Id: I5e0a76e83318d7761d9d010b0083abc048f563e4
Reviewed-on: https://chromium-review.googlesource.com/1183665
Commit-Queue: Erik Språng <sprang@chromium.org>
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588358}
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/mojo/clients/mojo_video_encode_accelerator_unittest.cc
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/mojo/interfaces/video_encode_accelerator.mojom
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/mojo/interfaces/video_encode_accelerator_mojom_traits.cc
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/mojo/interfaces/video_encode_accelerator_mojom_traits.h
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/video/video_encode_accelerator.cc
[modify] https://crrev.com/f5420d17fdf394d32cae5d391613e7dbcc6acd3d/media/video/video_encode_accelerator.h
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 10 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6498431e2138bed86d4f8c5ba5bba60e10498ef2

commit 6498431e2138bed86d4f8c5ba5bba60e10498ef2
Author: Erik Språng <sprang@chromium.org>
Date: Mon Sep 10 10:22:44 2018

Disable vp8 screen content for vaapi

Bug: chromium:853185, b:111781384
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I7bc0dc619503bbb5d0b1ed5420187f0338e160a4
Reviewed-on: https://chromium-review.googlesource.com/1201859
Commit-Queue: Erik Språng <sprang@chromium.org>
Reviewed-by: Emircan Uysaler <emircan@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Commit-Position: refs/heads/master@{#589874}
[modify] https://crrev.com/6498431e2138bed86d4f8c5ba5bba60e10498ef2/content/renderer/media/webrtc/rtc_video_encoder.cc
[modify] https://crrev.com/6498431e2138bed86d4f8c5ba5bba60e10498ef2/media/gpu/vaapi/vaapi_video_encode_accelerator.cc
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 13

Labels: merge-merged-3538
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bd32f02a65c700f0df259c3d63d4d6ca884ad555

commit bd32f02a65c700f0df259c3d63d4d6ca884ad555
Author: Erik Språng <sprang@chromium.org>
Date: Thu Sep 13 16:27:29 2018

Add support for configuring content type for VideoEncodeAccelerators

Bug: chromium:853185,  chromium:883436 
Change-Id: I5e0a76e83318d7761d9d010b0083abc048f563e4
Reviewed-on: https://chromium-review.googlesource.com/1183665
Commit-Queue: Erik Språng <sprang@chromium.org>
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#588358}(cherry picked from commit f5420d17fdf394d32cae5d391613e7dbcc6acd3d)
Reviewed-on: https://chromium-review.googlesource.com/1224110
Cr-Commit-Position: refs/branch-heads/3538@{#370}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/mojo/clients/mojo_video_encode_accelerator_unittest.cc
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/mojo/interfaces/video_encode_accelerator.mojom
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/mojo/interfaces/video_encode_accelerator_mojom_traits.cc
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/mojo/interfaces/video_encode_accelerator_mojom_traits.h
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/video/video_encode_accelerator.cc
[modify] https://crrev.com/bd32f02a65c700f0df259c3d63d4d6ca884ad555/media/video/video_encode_accelerator.h
Project Member

Comment 8 by bugdroid1@chromium.org, Sep 14 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/55ef3690e6f8306f4d26f2a5506ded25e1a6b4c3

commit 55ef3690e6f8306f4d26f2a5506ded25e1a6b4c3
Author: Erik Språng <sprang@chromium.org>
Date: Fri Sep 14 12:34:40 2018

Disable vp8 screen content for vaapi

Bug: chromium:853185,  chromium:883436 , b:111781384
Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel
Change-Id: I7bc0dc619503bbb5d0b1ed5420187f0338e160a4
Reviewed-on: https://chromium-review.googlesource.com/1201859
Commit-Queue: Erik Språng <sprang@chromium.org>
Reviewed-by: Emircan Uysaler <emircan@chromium.org>
Reviewed-by: Pawel Osciak <posciak@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#589874}(cherry picked from commit 6498431e2138bed86d4f8c5ba5bba60e10498ef2)
Reviewed-on: https://chromium-review.googlesource.com/1224552
Cr-Commit-Position: refs/branch-heads/3538@{#402}
Cr-Branched-From: 79f7c91a2b2a2932cd447fa6f865cb6662fa8fa6-refs/heads/master@{#587811}
[modify] https://crrev.com/55ef3690e6f8306f4d26f2a5506ded25e1a6b4c3/content/renderer/media/webrtc/rtc_video_encoder.cc
[modify] https://crrev.com/55ef3690e6f8306f4d26f2a5506ded25e1a6b4c3/media/gpu/vaapi/vaapi_video_encode_accelerator.cc
►

Sign in to add a comment