During Muxed Audio Video Content Playback via MSE, Audio Sync Error between Representations accumulates on ABR switches, ultimately resulting in Error
Reported by
hi.a...@gmail.com,
Nov 9 2016
|
||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36 Example URL: Steps to reproduce the problem: 1. Play muxed content having multiple representations and slight audio video sync differences between fragments 2. Make a few ABR switches 3. See that ultimately the Audio Sync Error gets large enough to throw a Audio Splicing Failed error What is the expected behavior? The Audio Sync Error between multiple representations, should not build up. What went wrong? Audio splicing failed: coded frame timestamp differs from expected timestamp 726545386us by 95980us, more than threshold of +/-50ms. Similar issue is not encountered in Firefox in such a scenario. Did this work before? N/A Is it a problem with Flash or HTML5? HTML5 Does this work in other browsers? Yes Chrome version: 54.0.2840.71 Channel: n/a OS Version: OS X 10.11.6 Flash Version: Shockwave Flash 23.0 r0 Following is a comparison of Audio and Video Timestamp differences between corresponding fragments of 2 of the Reps Fragment Rep_1_Video_Time Rep_1_Audio_Time Rep_2_Video_Time Rep_2_Audio_Time 1 240.24 240.2853667 240.24 240.252 2 246.24 246.3013667 246.246 246.268 3 252.25 252.2853667 252.252 252.252 4 258.25 258.3013667 258.258 258.268 5 264.26 264.3173667 264.264 264.284 6 270.27 270.3333667 270.27 270.3 7 276.27 276.3173667 276.276 276.284 8 282.28 282.3333667 282.282 282.3 9 288.28 288.3493667 288.288 288.316 10 294.29 294.3333667 294.294 294.3 It is not possible for me to share the Player code and Stream with anyone, due to my company policy.
,
Nov 10 2016
Hi hi.ansh, The problem should be eliminated if you can make the segment start times match accross representations. Chrome's media clock is derived by summing up decoded audio samples. The little gaps in your fragments are not considered in this calculation, so the math veers further from expected time as more and more fragment gaps are observed. We've written the audio clock this way because timestamps from metadata are often imprecise. We might consider veri-speed stretching/shrinking of audio to cover these gaps, but this isn't currently in the roadmap. Best advice is to just re-encode your segments to make them match. Can you share what tools your using to make these segments? Perhaps they can be improved to be more precise.
,
Nov 30 2016
hi.ansh@gmail.com, friendly ping. I believe this is working as intended, but to make sure my explanation in comment 2 unblocks you before I close. Also, can you share what tools you're using to make the segments?
,
Dec 1 2016
chcunningham@chromium.org, Thanks for your explanation in comment 2. We at Adobe have built an SDK, which can play HLS and DASH streams via MSE. The stream in question here is an HLS stream and is from one of our customers, and I am trying to find the tool that they are using to create the stream. But it is taking some time to find out this information. But, we generally don't have control over source of the streams and so would really appreciate if Chrome can handle mismatch of segment start times across representations. One point that is not clear to me, is why is this issue specific to muxed buffers. If I play the same stream, with separate audio and video buffers, then it keeps on playing for a long time without any Error.
,
Dec 7 2016
I'll continue to monitor the frequency of this issue and brainstorm ideas! Chrome can attempt to cover it up, but it has maintenance/complexity tradeoffs that aren't great. The best long term cross-platform solution is to improve tooling so that developers can chunk up their media in a way that's precise. Happy to see what we can do there. |
||
►
Sign in to add a comment |
||
Comment 1 by dalecur...@chromium.org
, Nov 9 2016