Assigning to hlundin@ as I will be OOO from 6/18.

aleloi@, please, take a look at this bug.

gyzhou@, is there any reason to believe that this is unique to Windows 7? Did you try to reproduce on any other platform?

The application that uses AudioTrackSinkInterface is for windows OS. I don't whether other platforms will have this issue or not.

Trying to reproduce here https://codereview.webrtc.org/2083903003, so far no luck.

Hi!

I think we now know the reason for this.

What we see (and this is reproduced by both gyzhou@ and me), is that the first frames received from AudioTrackSinkInterface have 1 channel, 16kHz and are filled with zeros. The number of such frames seems to vary between different runs and different machines. On my latest debug run it was 52.

The audio data comes from the NetEq component, which gets audio data from the network. It takes a while for data to go through the network, even in a loopback configuration. When there is no network data, NetEq just makes something up. When there hasn't been any traffic at all, it sends zero frames.

The rate returned is the rate that NetEq has been initialized with. The default rate is 16k (link: https://cs.chromium.org/chromium/src/third_party/webrtc/modules/audio_coding/neteq/include/neteq.h?sq=package:chromium&rcl=1470626560&l=78). It is not modified after a NetEq::Config is created (here: https://cs.chromium.org/chromium/src/third_party/webrtc/modules/audio_coding/include/audio_coding_module.h?sq=package:chromium&rcl=1470626560&l=67), nor when a NetEq instance is created from the config (here: https://cs.chromium.org/chromium/src/third_party/webrtc/modules/audio_coding/include/audio_coding_module.h?sq=package:chromium&rcl=1470626560&l=67).

The number of channels is set in the NetEqImpl constructor (here: https://cs.chromium.org/chromium/src/third_party/webrtc/modules/audio_coding/neteq/neteq_impl.cc?sq=package:chromium&rcl=1470626560&l=124) and is set to 1 by default.

Thus, if there is a poll for audio before the audio has got through the network, NetEq will return 16kHz, 1 channel. This should be possible to reproduce on other platforms. The fact that my attempt didn't work may be explained by running it on a very high-end Goobuntu desktop. I will run it on a laptop to see the difference.

Deleted: recording_data.txt 6.7 MB

recording_data.txt 6.7 MB View Download

Forgot to mention: the attached text file is a python/json log of a short recording (I count 1, 2, 3.. in Swedish). The first 48kHz/2 channel frame is #53 IIRC.

Also, you may not agree with me re my WontFix resolution of this. My motivation is that (from my imperfect understanding of the code) there is nothing else NetEq could possibly do before receiving any network packets. On the other hand, we could document that this is the intended behavior.

Finally, sorry for my slow handling of this. I had lots of windows trouble in the beginning and expected it to continue.

It is nice that you got the root cause.

I believe this need to be fixed. Because typical user will decide the number of channels and sampling rate from the first callback of audio sink. Asking them to take care of this issue in user side seems not reasonable.

As long as you get the root cause, this doesn't seem like a very difficult one to fix. You may cancel any callback before receiving network packet. It is better to deliver no package than deliver zero package before receiving network packet.

gyzhou@, I agree that we have a usability problem here. The current design makes it difficult and undesirable to just not send out any audio until the parameters are known. It may be possible to send out fake data at the rate that the user asks for. Alternatively, a simple 'fix' is to set 48kHz/stereo as the default instead of 8kHz/mono. Or claim some bogus rate and channel count until the real ones are known.

I think "Won't Fix" is appropriate here. The AudioTrackSinkInterface is a way to side-chain audio from a stream that's being received, and as the interface clearly indicates, each frame may in theory have different sample rate and channel layout. The implementor must be able to handle that (by doing appropriate conversions or something else).

It is not only this warm-up case which can cause the audio properties to change here - if a new codec is picked during a session, if the sound card output changes, that may trigger a change too.