Artifacts in WebRTC video |
||||
Issue descriptionSmall, black, rectangular boxes appear when during WebRTC session. This has only been seen on ChromeOS, Intel platforms. Specifically eve and soraka. Only occurs when the video has been place on an overlay/underlay. Only seen when using vaapi with hw codecs. It is present for both h.264 and vp8 Is exacerbated by poor network. By causing WebRTC to drop 1% of the packets this is quite repeatable.
,
Nov 27
Changing the underlay/overlay changes how the video is processed. When using an underlay/overlay the video frame is decoded by vaapi and then sent directly to the display controller. Otherwise the video is composited on the gpu and then sent to the display controller. There currently isn't a way to force a sync between the display controller and vaapi. vaSyncSurface is currently a no-op on Intel : https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/master/x11-libs/libva-intel-driver/files/no_explicit_sync_in_va_sync_surface.patch Reverting that patch does not make the artifacts go away.
,
Nov 27
https://cdn.rawgit.com/uysalere/js-demos/master/pc1bw.html will show the artifacts when using callfactory.cc patch to drop packets.
,
Nov 27
The artifacts appear to be a result of a cache coherency issue. Frames of video are decoded by vaapi and then passed to the display controller. It looks like the cache is not getting flushed before display. The artifacts appear predominantly at the top of the screen. If scan out occurred from the top of the image down this would correspond to the cache not getting flushed at the start of the frame, but being flushed by the end of the frame. Dropping frames exacerbates the issue. This could be because normally there is a buffer of frames. In that case the cache for the video frame would be flushed before it is time to display it. In the case of packet drop the frame is dropped and a new frame is requested, which would need to be displayed right away, without sitting in a queue for any period of time. There should be an implicit fence occurring between the video decode and vaapi, but this doesn't appear to be happening. There currently isn't a way to have an explicit fence for the vaapi frames.
,
Dec 4
,
Dec 5
,
Dec 6
Hi frkoenig, I can't reproduce this WebRtC frame drop issue on Soraka with R72.11307 by https://cdn.rawgit.com/uysalere/js-demos/master/pc1bw.html, could you share more detail repo steps for this issue.
,
Dec 6
Did you modify callfactory.cc to drop packets? Or are you running stock R72.11307? Is the video getting sent to a plane, or is it getting composited? Did you fullscreen the video? If you press Enter it will show the decoded video fullscreen. This makes it easier to spot the artifacts.
,
Dec 7
Now I can repo it on Soraka R72.11307 after pressed Enter to fullscreen without changed callfactory.cc code. Meanwhile, I did some testing as the following. precondition: Disabled WebRTC HW h264/vp8 video encoding 1. Enabled WebRTC HW video decoding flag -> Appear artifacts 2. Disabled WebRTC HW video decoding flag -> No artifacts 3. Run on Nami with same Mesa 18.2.0 and Enabled HW decoding -> No artifacts 4. Run "WebRTC Samples - AppRTC video chat client" with HW video decoding -> No artifacts Per above testing result, I think the issue is not from Intel Mesa and i915 driver. Is there any different HW decoding config between pc1w.html and AppRTC video client? Is any test sample can run fullscreen video playback in WebRTC HW decoding without camera engagement?
,
Dec 7
I can also reproduce issue, but from libva decode surfaces dump, there is no artifacts on decoded buffer. That means the HW decode function is good. So the artifacts looks like is an incomplete frame but consuming directly by display controller, hence expose the artifacts shows on screen. Need further check why video & display not in sync.
,
Dec 7
Off topic, but you don't need to modify callfactory.cc, you can configure packet loss from command line. On cros that is a bit cumbersome but still easier than making custom build. See https://www.chromium.org/developers/how-tos/run-chromium-with-flags A command line for 3% sent packet loss and 5% received packet loss is: --force-fieldtrials=WebRTCFakeNetworkSendLossPercent/3/WebRTCFakeNetworkReceiveLossPercent/5/
,
Dec 7
Re #10: yes, that's accurate. We suspect that it's perhaps the synchronization on the video decode ring buffer that's not working right. I believe this is the first case of video decode directly to a scanout buffer for the Intel Linux graphics stack, so perhaps it never worked. Normally we pageflip to 3d engine buffers (from the compositor). I think i915.ko internally disinguishes between execution (ie either video decode or 3d rendering) and subsequent cache flushing. Perhaps we're waiting on the execution to finish, but failing to wait for the flushing to finish, which would explain the cache tile sizes black flicker.
,
Jan 18
(5 days ago)
Any progress on the Intel side of things with this?
,
Jan 18
(5 days ago)
Hi, I can't reproduce this issue on latest R73.11602 build (compared with buggy image R72.11307) on Soraka, could you help to confirm?
,
Jan 18
(4 days ago)
You are right, on R73.11602, https://cdn.rawgit.com/uysalere/js-demos/master/pc1bw.html it does not show the problem. I have an internal example that still shows the failure. I can confirm that it is failing in the same was as R72.11307 I will try to find another external example that fails for R73 |
||||
►
Sign in to add a comment |
||||
Comment 1 by frkoenig@chromium.org
, Nov 27Forcing WebRTC to drop packets on ChromeOS can be done by modifying callfactory.cc diff --git a/call/callfactory.cc b/call/callfactory.cc index ab057be028..4e0615472e 100644 --- a/call/callfactory.cc +++ b/call/callfactory.cc @@ -62,6 +62,11 @@ absl::optional<webrtc::BuiltInNetworkBehaviorConfig> ParseDegradationConfig( } configured |= ParseConfigParam(exp_prefix + "AvgBurstLossLength", &config.avg_burst_loss_length); + if (!send) { + configured = true; + config.loss_percent = 1; + } + return configured ? absl::optional<webrtc::BuiltInNetworkBehaviorConfig>(config) : absl::nullopt;