New issue
Advanced search Search tips

Issue 642663 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

Cannot clone/forward hardware decoded video tracks

Project Member Reported by jansson@chromium.org, Aug 31 2016

Issue description

Discovered after using mediaStream.clone() in a recent AppRTC loopback patch which implements proper loopback using two peerConnections. This needs to be fixed before re-landing it.

niklase@ not sure if you are the right owner for this?

Just adding a preliminary milestone, please feel free to adjust.
 
Note: I can bring up a dev instance of AppRTC that uses the new loopback logic for testing/dev work if required.
Labels: -M-55 M-56
It didn't make it into M55 so moving to M56.
Cc: -emir...@chromium.org niklase@chromium.org
Owner: emir...@chromium.org
Status: Started (was: Assigned)
Cc: jansson@chromium.org
jansson@, what you mentioned on #2 would be very useful to test. Is there a page I can use currently? 
https://loopback-dot-apprtc.appspot.com/?debug=loopback.

Note that there are some errors in the console about keys etc, do not worry about it, should work for the loopback use case anyway.
Project Member

Comment 6 by bugdroid1@chromium.org, Oct 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e0a575d620207471068e04b133f7e24ff84b23bd

commit e0a575d620207471068e04b133f7e24ff84b23bd
Author: emircan <emircan@chromium.org>
Date: Fri Oct 28 23:05:01 2016

Add callback to copy texture backed frames in WebRtcVideoFrameAdapter

This CL adds a callback to copy texture backed frames in WebRtcVideoFrameAdapter
so that hardware decoded video tracks can be cloned or forwarded. The callback is
assigned by WebRtcVideoCapturerAdapter and runs in main renderer thread.

BUG= 642663 
TEST=Ran https://loopback-dot-apprtc.appspot.com/?debug=loopback&vsc=h264 on Mac.

Review-Url: https://codereview.chromium.org/2456443002
Cr-Commit-Position: refs/heads/master@{#428535}

[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/gpu/rtc_video_decoder.cc
[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/webrtc/webrtc_video_capturer_adapter.cc
[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/webrtc/webrtc_video_capturer_adapter.h
[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/webrtc/webrtc_video_capturer_adapter_unittest.cc
[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/webrtc/webrtc_video_frame_adapter.cc
[modify] https://crrev.com/e0a575d620207471068e04b133f7e24ff84b23bd/content/renderer/media/webrtc/webrtc_video_frame_adapter.h

Status: Fixed (was: Started)
jansson@, I added the texture copy path for HW decoded frames. However, the performance is terrible as expected. On my Mac running H264 on you AppRTC fork, I get ~5 fps. I think it makes sense to keep the older version when HW decode is involved.

The problems behind performance is basically due to the webrtc calling this method on some jingle thread, posting it back on renderer main thread, roundtrip via gpu context and blocking until it is done. Unfortunately, there isn't a cheaper way to this that I know of. In addition, I tried a hacky approach where I asynchronously download all hw decoded frame before going to webrtc but even that reaches only ~10 fps. Th patch is here for the referenceat, https://codereview.chromium.org/2459613004/. 
Status: Verified (was: Fixed)
Verified in M56 56.0.2924.18 in Mac
Tested https://loopback-dot-apprtc.appspot.com/?debug=loopback&vsc=h264 using hardware decoding 
Frame rate received stayed at 30 fps
Frame rate sent was between 10-20 fps, occasionally spiking to 30 fps

Sign in to add a comment