New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 727872 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android , Windows , Chrome , Mac
Pri: 2
Type: Bug



Sign in to add a comment

MediaStream Video and Audio chunks have different timestamp origins

Project Member Reported by mcasas@chromium.org, May 30 2017

Issue description

[1] described a situation in which the video and audio chunks being
transported in the respective MediaStreams do not have the same clock
origin, which caused a freeze upon playback.

 https://crbug.com/597034 , where this issue was uncovered, patched 
MediaRecorder's WebmMuxer code by keeping a rolling "last seen timestamp"
but that was a temporary solution: MR produces reproducible files,
but the video and the audio could be out of synch.


This was the interchange:

>On 2016/04/06 00:49:01, miu wrote:
>> lgtm, HOWEVER: The reason the timestamps are far apart is because there is
>> more/less playout buffering going on for audio versus video.  The timestamps
>> should be reflecting the playout time at which each frame should be displayed
>> on screen (or, for audio, when each frame should come out the speaker).
>> 
>> With this change, I'm willing to bet A/V sync is now a problem.  At least, it
>> will be on some platforms.  Solving this may involve having your WebM muxer
>> queuing-up either audio or video (or both), and then only output both to the
>> WebM container once they've been aligned w.r.t. their playout timestamps.
>> 
>> So, I'm lgtm'ing this, and leave it up to you to decide whether this change
>> should be committed as-is, and whether the A/V sync performance aspects of
>> your feature should be addressed now or later.
>
>I don't think this is a case of audio/video sync adjustments. I 
>printed out the base::TimeTicks timestamps of the incoming video
>and audio recorded frames and I got the following numbers. 
>
>First a bunch of Video Frames arrive with correlative timestamps
>
>media::WebmMuxer::OnEncodedVideo - 59928B @ 2149924 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo: delaying until audio track ready.
>media::WebmMuxer::OnEncodedVideo - 108B @ 2183258 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo: delaying until audio track ready.
>media::WebmMuxer::OnEncodedVideo - 143B @ 2216591 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo: delaying until audio track ready.
>
>and now the first Audio arrives with completely different timestamps:
>
>media::WebmMuxer::OnEncodedAudio - 386B @ 1269455645930 bogo-microseconds
>media::WebmMuxer::AddAudioTrack format: 1 channel_layout: 2 channels: 1
>sample_rate: 48000 bits_per_sample: 16 frames_per_buffer: 2880 effects: 0
>mic_positions:
>
>and now they start mingling, each with their own reference.
>
>media::WebmMuxer::OnEncodedVideo - 214B @ 2249924 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 217B @ 2283258 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 308B @ 2316591 bogo-microseconds
>media::WebmMuxer::OnEncodedAudio - 383B @ 1269455771097 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 348B @ 2349924 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 309B @ 2383257 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 640B @ 2416591 bogo-microseconds
>media::WebmMuxer::OnEncodedAudio - 383B @ 1269455878194 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 809B @ 2449924 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 977B @ 2483257 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 1118B @ 2516591 bogo-microseconds
>media::WebmMuxer::OnEncodedAudio - 383B @ 1269455988642 bogo-microseconds
>media::WebmMuxer::OnEncodedVideo - 1296B @ 2549924 bogo-microseconds
>
>(I also checked that these timestamps correlate with
>those coming from the Tracks, the recorders did not
>interfere with them significantly).







[1] https://bugs.chromium.org/p/chromium/issues/detail?id=597034#c12
 

Comment 1 by mcasas@chromium.org, May 30 2017

Cc: srnarayanan@chromium.org
Project Member

Comment 2 by sheriffbot@chromium.org, May 31 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Available (was: Untriaged)

Sign in to add a comment