New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 826668 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

Fill out missing input audio data with silence on Windows

Project Member Reported by grunell@chromium.org, Mar 28 2018

Issue description

If we have missing input audio data from the audio engine/audio device, we should fill out with silence. Otherwise we'll have a delay jump as seen from the echo canceller (i.e. the delay output to input which can harm its performance) and the missing data will cause a speed-up of the audio. Silence is probably better in the latter case too.

We need a reliable glitch detection for this. Can we trust the device position?
 
Description: Show this description
I'm not sure the discontinuity flag can be trusted either. I've seen it set when there is seemingly no glitch. Perhaps combine it with device position?

Comment 3 by ossu@chromium.org, Mar 28 2018

Yeah. I think we'd want to use the discontinuity flag and the device position to decide this. I could imagine we'd want to ignore very short discontinuities as well. Say the driver calculates something poorly and we every now and then get a single sample of discontinuity. Perhaps better to just eat that - or find a better place to extend with zeroes. Like when it's silent. 

Comment 4 by peah@chromium.org, Mar 29 2018

Cc: hlundin@chromium.org tlegrand@chromium.org
The tricky part with filling out missing data is that for it to make sense it needs to be exactly correct in duration. Otherwise, it may actually be better to not fill in the missing data.

If the inserted number of zeros does not exactly match the number of samples that were missed, the echo canceller will anyway need to re-converge. How bad that reconvergence is differs but in some cases it is better to just avoid inserting the zeros and let the echo canceller handle that.

What should definitely be avoided is to insert too many zeros as that will lead to a delay buildup (assuming this is on the capture side). If there is any risk of that, it is better to not insert zeros.

Comment 5 by peah@chromium.org, Mar 29 2018

Cc: gustaf@chromium.org

Comment 6 by ossu@chromium.org, Apr 5 2018

peah@: In understand your considerations are specific to when we're using an echo canceller. I'm not sure I really understand them, though. How sensitive is this? Why would, say, half a second of missing samples be better than a few extra samples?

While digging into another bug, we found that not delivering any audio during discontinuities caused an even worse audio experience than necessary. Not only were we missing bits of the audio - the bits we had were compressed over time, and played back much faster than recorded.

I'll be looking at probably adding some silent blocks of audio to the Mac implementation as well, during device reconfiguration. I'd like to understand your concerns better, so I don't do anything stupid. :)

Comment 7 by peah@google.com, Apr 5 2018

For the AEC, it is very sensitive and the number of samples inserted needs to be exact to what was missing. Otherwise it is better not to do anything but just continue. 

If I understand you correctly, there is another usecase for inserting zeros? It sounds a bit strange that inserting zeros would avoid that. 

In general, I strongly doubt that one should try to patch the signals with zeros for where data is lost. If that goes wrong, it can very easily cause other problems such as poor NetEQ behavior, delay buildup in the audio pipeline (note related to the AEC), and possibly more.


Mac counterpart in issue 610270.

Sign in to add a comment