New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 702017 link

Starred by 3 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature



Sign in to add a comment

Support VEA temporal layers.

Project Member Reported by pbos@chromium.org, Mar 15 2017

Issue description

Hardware encoders do not support temporal layers, this effort encompasses extracting the temporal-layer strategies, bitrate allocation (between layers), error resilience and buffer references etc. so they can be used with non-libvpx encoders.

After they are less tied to vp8_impl.cc, it'll need to be wired up from RTCVideoEncoder -> VEA -> V4L2 -> Kernel drivers -> HW encoders, and back up, so that RTP headers can properly set temporal layers/layer sync bits and.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Mar 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/e6fe2a9a7f489ae6cf2a1f9df214394221965369

commit e6fe2a9a7f489ae6cf2a1f9df214394221965369
Author: pbos <pbos@webrtc.org>
Date: Wed Mar 22 20:44:15 2017

Simplify default temporal layers.

Moves towards separating which layers may be referenced instead of
referencing libvpx flags directly. This will make strategies easier to
extract and usable from hardware encoders (RTCVideoEncoder, for
instance).

BUG=chromium:702017, webrtc:7349
R=brandtr@webrtc.org, marpan@webrtc.org. sprang@webrtc.org

Review-Url: https://codereview.webrtc.org/2747123005
Cr-Commit-Position: refs/heads/master@{#17349}

[modify] https://crrev.com/e6fe2a9a7f489ae6cf2a1f9df214394221965369/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.cc
[modify] https://crrev.com/e6fe2a9a7f489ae6cf2a1f9df214394221965369/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.h

Project Member

Comment 2 by bugdroid1@chromium.org, Mar 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/1436c83cd11f636b49eacdbada2e7f6d17eb6d62

commit 1436c83cd11f636b49eacdbada2e7f6d17eb6d62
Author: Peter Boström <pbos@webrtc.org>
Date: Mon Mar 27 19:01:49 2017

Base screenshare layers on TemporalReferences.

Decouples encode flags and calculates them the same for both default and
screencast temporal layers.

With this change encoders could start using TemporalReferences for
temporal-layers flags, but they can not be used by asynchronous encoders
(hardware encoders) yet.

Also removes 'timestamp' as a dead parameter to FrameEncoded().

BUG=chromium:702017, webrtc:7349
R=marpan@google.com, sprang@webrtc.org, marpan@webrtc.org

Review-Url: https://codereview.webrtc.org/2769263002 .
Cr-Commit-Position: refs/heads/master@{#17397}

[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.cc
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.h
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.cc
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.h
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/screenshare_layers_unittest.cc
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/temporal_layers.h
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
[modify] https://crrev.com/1436c83cd11f636b49eacdbada2e7f6d17eb6d62/webrtc/modules/video_coding/utility/simulcast_rate_allocator_unittest.cc

Cc: brandtr@chromium.org

Comment 4 by pbos@chromium.org, Apr 3 2017

Labels: -M-59 M-60
M59 is not realistic.
Project Member

Comment 5 by bugdroid1@chromium.org, May 2 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0cd36b0f1fa2b9778ec33ec9db4ad9bcba094c7e

commit 0cd36b0f1fa2b9778ec33ec9db4ad9bcba094c7e
Author: vabr <vabr@chromium.org>
Date: Tue May 02 09:50:42 2017

Revert of Remove FrameView::Root (patchset #2 id:20001 of https://codereview.chromium.org/2849383002/ )

Reason for revert:
This broke compilation (details later).

Today's sheriff

Original issue's description:
> Remove FrameView::Root
>
> BUG= 637460 
>
> Review-Url: https://codereview.chromium.org/2849383002
> Cr-Commit-Position: refs/heads/master@{#468588}
> Committed: https://chromium.googlesource.com/chromium/src/+/561e38ff8dc24f23f0293c54777bbc2eda99a2b5

TBR=haraken@chromium.org,dcheng@chromium.org,joelhockey@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 637460 

Review-Url: https://codereview.chromium.org/2857523002
Cr-Commit-Position: refs/heads/master@{#468589}

[modify] https://crrev.com/0cd36b0f1fa2b9778ec33ec9db4ad9bcba094c7e/third_party/WebKit/Source/core/frame/FrameView.cpp
[modify] https://crrev.com/0cd36b0f1fa2b9778ec33ec9db4ad9bcba094c7e/third_party/WebKit/Source/core/frame/FrameView.h
[modify] https://crrev.com/0cd36b0f1fa2b9778ec33ec9db4ad9bcba094c7e/third_party/WebKit/Source/web/WebInputEventConversion.cpp

Project Member

Comment 6 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/18ad1d4008ce5a1909310bafdd50897cfefc2de7

commit 18ad1d4008ce5a1909310bafdd50897cfefc2de7
Author: pbos <pbos@webrtc.org>
Date: Thu May 04 12:04:46 2017

Derive current layer from TL frame config.

Partially removes "current frame" state from TemporalLayers, aiming to
tie more state to the frame config being encoded. This is necessary for
having several outstanding frames being encoded.

Also renames several structs, since TemporalReferences contains more
than references it's renamed TemporalLayers::FrameConfig.

BUG=chromium:702017, webrtc:7349
R=sprang@webrtc.org

Review-Url: https://codereview.webrtc.org/2853073004
Cr-Commit-Position: refs/heads/master@{#18012}

[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.cc
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.h
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers_unittest.cc
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.cc
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.h
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/screenshare_layers_unittest.cc
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/temporal_layers.h
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/codecs/vp8/vp8_impl.h
[modify] https://crrev.com/18ad1d4008ce5a1909310bafdd50897cfefc2de7/webrtc/modules/video_coding/utility/simulcast_rate_allocator_unittest.cc

Comment 8 by pbos@chromium.org, May 29 2017

Labels: -M-60 M-61

Comment 9 by pbos@chromium.org, Jun 16 2017

Status: Available (was: Started)
Project Member

Comment 10 by bugdroid1@chromium.org, Jul 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/1777c5fec5f14b7ecc882e359e36cfe89903cbfb

commit 1777c5fec5f14b7ecc882e359e36cfe89903cbfb
Author: pbos <pbos@webrtc.org>
Date: Thu Jul 20 00:04:02 2017

Move temporal-layer properties to FrameConfig.

Removes keying on pattern_idx inside TemporalLayers implementations for
several properties that are different between screencast temporal layers
and normal/default temporal layers.

This is a step towards sharing PopulateCodecSpecific between the layer
patterns and code deduplication to longer term be able to separate the
packetizer step from encoder settings, so that temporal patterns can be
used for asynchronous hardware encoders where there may be outstanding
frames.

BUG=chromium:702017, webrtc:7349
R=brandtr@webrtc.org

Review-Url: https://codereview.webrtc.org/2924993002
Cr-Commit-Position: refs/heads/master@{#19097}

[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.cc
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers.h
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/default_temporal_layers_unittest.cc
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.cc
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/screenshare_layers.h
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/screenshare_layers_unittest.cc
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/temporal_layers.h
[modify] https://crrev.com/1777c5fec5f14b7ecc882e359e36cfe89903cbfb/webrtc/modules/video_coding/codecs/vp8/vp8_impl.cc

Comment 11 by pbos@chromium.org, Aug 8 2017

Labels: -M-61

Comment 12 by pbos@chromium.org, Sep 5 2017

Cc: emir...@chromium.org mnilsson@chromium.org
Owner: sprang@chromium.org
sprang@ I won't be finishing this work. I think it's important to have quality parity (temporal layers supported) on hardware encoders. There is backing hardware support for it in several cases. Can you prioritize or reassign as appropriate?

When WebRTC exposes temporal layer information and RtcVideoEncoder makes use of it (referencing correct layers, using correct bitrate allocator, having correct target bitrates per layer), posciak@ can probably take this over from a VEA perspective, to make sure that this information reaches the drivers (and gets reported back properly when buffers are encoded).

Comment 13 by pbos@chromium.org, Sep 5 2017

Cc: -jmcgill@google.com pbos@chromium.org
Status: Assigned (was: Available)

Sign in to add a comment