Accelerated video encoder framework |
|||||
Issue descriptionSome hardware encoders may require additional support from their clients for high-level, video codec-specific encoding steps, such as reference frame management, generating stream headers, etc. The AcceleratedVideoEncoder framework is intended to support such hardware encoders. An AcceleratedVideoEncoder (AVE) is a platform-independent, codec-specific video encoder that performs high-level, platform-independent encoding process tasks, such as managing codec state, reference frames, etc., and requires support from an external Accelerator (typically a hardware accelerator) to offload the actual image encoding process, using the parameters that an AVE prepares beforehand. Clients of AVEs are expected to provide an Accelerator class instance appropriate for current platform and/or scenario. Each video codec-specific AVE specifies its own Accelerator interface (e.g. a H264Encoder AVE defines an H264-specific Accelerator).
,
Dec 15 2017
,
Dec 15 2017
,
Mar 19 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53 commit ec6e21b6e5159fee87fc595a225e9d3abd7a9a53 Author: Pawel Osciak <posciak@chromium.org> Date: Mon Mar 19 09:13:06 2018 AcceleratedVideoDecoder: introduce CodecPicture and remove VaapiDecodeSurface Generic, platform-independent decoders - {H264,VP8,VP9}Decoder - operate on generic Picture classes - {H264,VP8,VP9}Picture - and use them to communicate with platform-specific Accelerators. Accelerators augment these Picture classes with platform-specific metadata in order to process them. This includes platform-specific representations of buffers used as targets for decoding a given picture, such as VASurface for VaapiVDA. This platform-specific metadata is encapsulated in Accelerator-specific {Vaapi,V4L2}DecodeSurface classes. To tie {Vaapi,V4L2}DecodeSurface to each type of Picture class, a set of classes: {Vaapi,V4L2}{H624,VP8,VP9}Picture is introduced. They are subclasses of {H264,VP8,VP9}Picture classes and own {Vaapi,V4L2}DecodeSurface classes. For VaapiVDA, the relationship is: {H264,VP8,VP9}Picture ^ | | -[owns] -> H264/VP8/VP9 specific metadata [is a] | Vaapi{H624,VP8,VP9}Picture | -[owns]-> VaapiDecodeSurface | -[owns]-> common Picture metadata(visible_rect,...} platform-specific metadata(VASurface) In case of V4L2VDA: {H264,VP8,VP9}Picture ^ | | -[owns] -> H264/VP8/VP9 specific metadata [is a] | V4L2{H624,VP8,VP9}Picture | -[owns]-> V4L2DecodeSurface | -[owns]-> common Picture metadata(visible_rect,...) platform-specific metadata: ({input,output}_record, config_store...) However, as {Vaapi,V4L2}{H624,VP8,VP9}Picture are platform-specific already, we can eliminate one level of ownership indirection by removing {Vaapi,V4L2}DecodeSurface classes, as well as avoid having the same common Picture metadata in each of the 6 {Vaapi,V4L2}{H624,VP8,VP9}Picture classes to further simplify the design: CodecPicture ^ | | -[owns] -> common Picture metadata [is a] | {H264,VP8,VP9}Picture ^ | | -[owns] -> H264/VP8/VP9-specific metadata [is a] | Vaapi{H624,VP8,VP9}Picture | -[owns] -> platform-specific metadata(VASurface) Moreover, this will further allow us to reuse the picture classes on all levels of the hierarchy (including {V4L2,Vaapi}{H624,VP8,VP9}Picture classes) for the upcoming, new AcceleratedVideoEncoder framework, without having to create a new set of encoder-only {Vaapi,V4L2}EncodeSurface classes. Removal of V4L2DecodeSurface is left for a follow-up CL. BUG= 703988 TEST=VDA unittests, video playback in Chrome, video playback in ARC++ Cq-Include-Trybots: luci.chromium.try:linux_optional_gpu_tests_rel;master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Change-Id: I86f30adf4b5754fffb89eb3769b11d01d8e04814 Reviewed-on: https://chromium-review.googlesource.com/947545 Commit-Queue: Pawel Osciak <posciak@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Cr-Commit-Position: refs/heads/master@{#543995} [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/BUILD.gn [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/accelerated_video_decoder.h [add] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/codec_picture.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/h264_decoder.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/h264_decoder.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/h264_decoder_unittest.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/h264_dpb.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/BUILD.gn [add] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_common.cc [add] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_common.h [delete] https://crrev.com/f96879cc8dc1472de34cf6772702fa1e78da5d03/media/gpu/vaapi/vaapi_decode_surface.cc [delete] https://crrev.com/f96879cc8dc1472de34cf6772702fa1e78da5d03/media/gpu/vaapi/vaapi_decode_surface.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_h264_accelerator.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_h264_accelerator.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_video_decode_accelerator.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_video_decode_accelerator.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_video_decode_accelerator_unittest.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_vp8_accelerator.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_vp8_accelerator.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_vp9_accelerator.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vaapi/vaapi_vp9_accelerator.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp8_decoder.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp8_decoder.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp8_picture.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp8_picture.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp9_decoder.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp9_decoder.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp9_picture.cc [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/vp9_picture.h [modify] https://crrev.com/ec6e21b6e5159fee87fc595a225e9d3abd7a9a53/media/gpu/windows/d3d11_video_decoder_impl.cc
,
May 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e08e8bdfa1754c8c5e13feaa3608b348f8750f05 commit e08e8bdfa1754c8c5e13feaa3608b348f8750f05 Author: Pawel Osciak <posciak@chromium.org> Date: Mon May 21 00:42:17 2018 Add AcceleratedVideoEncoder API, H264Encoder impl, and a VA-API accelerator. An AcceleratedVideoEncoder (AVE) is a video encoder that performs high-level, platform-independent encoding process steps, such as managing codec state, reference frames, etc., but requires support from an external Accelerator (typically a hardware accelerator) to offload the actual image encoding process, using the parameters that AVE prepares beforehand. Clients of AVEs are expected to provide an Accelerator class instance appropriate for current platform and/or scenario. Each video codec-specific AVE specifies its own Accelerator interface (e.g. a H264Encoder AVE defines an H264-specific Accelerator). This CL adds the AVE API, a platform-independent implementation of AVE for H264 - H264Encoder, as well as a VA-API implementation of H264Encoder::Accelerator to provide an accelerator for it. BUG= 703988 , 794608 TEST=veaunittest,apprtc,vdaunittest Cq-Include-Trybots: luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: Iaf50f3a83df4ed51c81bb8a4c0e1152af82515e1 Reviewed-on: https://chromium-review.googlesource.com/918701 Commit-Queue: Pawel Osciak <posciak@chromium.org> Reviewed-by: Xiaohan Wang <xhwang@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Reviewed-by: Kuang-che Wu <kcwu@chromium.org> Cr-Commit-Position: refs/heads/master@{#560213} [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/content/renderer/media/webrtc/rtc_video_encoder.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/filters/h264_bitstream_buffer.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/filters/h264_bitstream_buffer.h [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/filters/h264_bitstream_buffer_unittest.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/v4l2/v4l2_video_encode_accelerator.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/BUILD.gn [add] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/accelerated_video_encoder.cc [add] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/accelerated_video_encoder.h [add] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/h264_encoder.cc [add] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/h264_encoder.h [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/vaapi_video_encode_accelerator.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/vaapi_video_encode_accelerator.h [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/vaapi_wrapper.cc [modify] https://crrev.com/e08e8bdfa1754c8c5e13feaa3608b348f8750f05/media/gpu/vaapi/vaapi_wrapper.h
,
May 21 2018
,
May 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/13de15aef0201ceae4ea203aceb0e0986ab5a8bf commit 13de15aef0201ceae4ea203aceb0e0986ab5a8bf Author: Pawel Osciak <posciak@chromium.org> Date: Mon May 21 05:48:25 2018 Add an AcceleratedVideoEncoder for VP8 and a VA-API accelerator for it. Add VP8Encoder, an implementation of AcceleratedVideoEncoder interface for VP8, and a VA-API implementation of VP8Encoder::Accelerator. Also introduce Vp8ReferenceFrameVector, to share reference frame management code between encoder and decoder classes. BUG= 703988 , 794608 TEST=veaunittest,apprtc,vdaunittest Cq-Include-Trybots: luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I7aa9bd9c110bf936728c2a5dab85649ca3328ed6 Reviewed-on: https://chromium-review.googlesource.com/918702 Commit-Queue: Pawel Osciak <posciak@chromium.org> Reviewed-by: Dale Curtis <dalecurtis@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Cr-Commit-Position: refs/heads/master@{#560234} [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/filters/vp8_parser.cc [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/filters/vp8_parser.h [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/BUILD.gn [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/BUILD.gn [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vaapi_video_encode_accelerator.cc [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vaapi_video_encode_accelerator.h [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vaapi_vp8_accelerator.cc [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vaapi_vp8_accelerator.h [add] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vp8_encoder.cc [add] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vaapi/vp8_encoder.h [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vp8_decoder.cc [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vp8_decoder.h [modify] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vp8_picture.cc [add] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vp8_reference_frame_vector.cc [add] https://crrev.com/13de15aef0201ceae4ea203aceb0e0986ab5a8bf/media/gpu/vp8_reference_frame_vector.h
,
May 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/267ef3465088d4d1bd324c0cc85cfb40819e4ff8 commit 267ef3465088d4d1bd324c0cc85cfb40819e4ff8 Author: Miguel Casas <mcasas@chromium.org> Date: Wed May 23 17:54:36 2018 VP8Decoder: simplify logic This Cl follows up on a discussion on [1] by simplifying the logic of a conditional. [1] https://chromium-review.googlesource.com/c/chromium/src/+/918702/10/media/gpu/vp8_decoder.cc#137 Bug: 703988 , 794608 Cq-Include-Trybots: luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel Change-Id: I890ab7fa742ff63e6b8ccf57221e2b90648a1fd7 Reviewed-on: https://chromium-review.googlesource.com/1069189 Reviewed-by: Daniele Castagna <dcastagna@chromium.org> Commit-Queue: Miguel Casas <mcasas@chromium.org> Cr-Commit-Position: refs/heads/master@{#561151} [modify] https://crrev.com/267ef3465088d4d1bd324c0cc85cfb40819e4ff8/media/gpu/vp8_decoder.cc
,
Aug 31
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by posciak@chromium.org
, Jun 6 2017