Issue metadata
Sign in to add a comment
|
Hardware decoded 'low latency' h.264 video has ~500ms delay over software decoded
Reported by
br...@openrov.com,
Jan 10 2017
|
||||||||||||||||||||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2976.0 Safari/537.36 Steps to reproduce the problem: 1. Ensure hardware decoded video is enabled 2. Using MSE APIs watch segmented mp4 live video feed, 1080p30 3. Measure video.seekable.end(0) - video.currentTime 4. Disable hardware decoding 5. Watch video and compare the measured video.seekable.end(0) - video.currentTime What is the expected behavior? With hardware decoded video: The measured duration of video in the buffer should be <.07 ms or so. Snapping your fingers should seem almost instant on the video What went wrong? On OS/X (both 55.0.28839.95 and latest canary 57.0.2976.0 Hardware decoded video has ~500ms delay compared to software video decoding On Windows both stable and canary Hardware decoded videos has ~200ms delay compared to software video decoding Did this work before? Yes ~Sep 2016 Does this work in other browsers? N/A Chrome version: 57.0.2976.0 Channel: n/a OS Version: OS X 10.11.6 Flash Version: Shockwave Flash 24.0 r0 We have been testing across 3 different computers. With a default Chrome install that has hardware decoding, we used to get consistent low latency live stream video using MSE with a reported 70ms or less of video stored in the buffer by measuring video.seekable.end(0) - video.currentTime. Video appeared snappy. Around September perhaps of 2016, noticed on OSX Canary at first, a huge spike in latency when watching the video. Turning off hardware decoded video fixes the issue. Around December of 2016 we noticed that the issue now existing in standard chrome as well, including on windows PCs, just not as pronounced. I understand this may be difficult without a repo case so we are actively working on trying to find a way help do that. Bug its taking time and we wanted to get this out in the open since we have seen it spread in to the stable builds. If I can get access to older versions of chrome I'd be happy to run a bisec to help try to figure out where the behavior changed. I hope to have a repo solution together, hopefully no later than the end of this week.
,
Jan 13 2017
,
Jan 16 2017
,
Jan 24 2017
brian@openrov.com, when I run your test url in c#1, I see Gaptime < 0.07 when use S/W decoding. Gaptime is always around 0.5~0.8 when use H/W decoding. is this the issue you mentioned? what are the expected Gaptime in H/W and S/W decoding?
,
Jan 24 2017
Yes, that is the issue. It affects low latency video streaming. The Hardware decoder is suffering a 500ms or so lag between appending the frames to the decoder buffer and when they play where as the software decoder has only 1/10th of that. They used to be nearly identical at less than 70ms.
,
Jan 24 2017
,
Jan 24 2017
Your H.264 stream sets neither constraint_set3_flag nor bitstream_restriction_flag, therefore the decoder must assume that up to the max DPB size of frames can be reordered (16 frames in this case). It looks like you are not using reordering at all, so setting bistream_restriction_flag to 1 and max_num_reorder_frames to 0 in the SPS will probably fix your issue.
,
Jan 26 2017
sande, thanks we are working on setting those right now and will test again. Out of curiosity, do you guys have a tool you like that dumps all of the SPS settings?
,
Jan 26 2017
I think the "standard" free tool is https://sourceforge.net/projects/h264bitstream/, but I've found it to be unstable. For quick analysis, I usually convert the stream to non-fragmented MP4 format with ffmpeg (ffmpeg -i frag.mp4 -c:v copy unfrag.mp4), then load it into Intel Video Pro Analyzer (trial version is enough for SPS analysis). Hopefully their next version will support fragmented MP4 directly. There are a variety of other H.264 analysis tools out there, I have not tried enough of them to make a recommendation.
,
Feb 3 2017
Thank you for providing more feedback. Adding requester "yiningc@chromium.org" for another review and adding "Needs-Review" label for tracking. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Feb 3 2017
Just to close the loop, those NAL flags made the difference. Thanks!
,
Feb 3 2017
Glad to hear this worked for you! (Closing as WontFix; working as intended.) |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by br...@openrov.com
, Jan 11 2017