New issue
Advanced search Search tips

Issue 737733 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

When setting the position of a PannerNode, the node's position is only updated when the sound starts, causing glitches when the sound starts as it moves from previous location to the current one

Reported by weis...@googlemail.com, Jun 28 2017

Issue description

UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36

Steps to reproduce the problem:
1. Create audio context, set up a panner node in HRTF panning model.
2. Set panner position to -1, 0, 0. Play the sound. After it has finished, set the panner position to 1, 0, 0. Play the sound again.

What is the expected behavior?
When the sound has stopped and the panner is updated, the next sound to be played through the panner should come immediately from the updated position.

What went wrong?
The sound plays fine when positioned for the first time. When the sound stops and I update the panner, then start the audio, the panner will only start crossfading the sound while playback is happening, creating a very quick but very noticeable slide effect as it moves quickly from left to right to set the new position, even though I had set the position while the sound was inactive, so it should have been set immediately.

Did this work before? N/A 

Does this work in other browsers? Yes

Chrome version: 58.0.3029.110  Channel: n/a
OS Version: 10.0
Flash Version: 

I'm working on an audio only game which needs precise positioning to be playable. When I change the position of a sound while it is not playing and then play it at a later time, the sound can sometimes take quite a while to get where it's supposed to, because the updated position only takes affect after the sound has started.
For an example on how this should sound like, you may try Mozilla Firefox. It does not seem to have this problem, which is why I suspect this to be a bug.
 
chrome-sound-test.zip
12.9 KB Download

Comment 1 by rtoy@chromium.org, Jun 28 2017

Status: Available (was: Unconfirmed)
Thanks for the repro test. When I press "Play left" and then "Play right" (middle button, not the right-most), I do hear briefly sound on the left then right.

This is not supposed to happen; I thought we removed all of the position dezippering from the panner node.

Comment 2 by rtoy@chromium.org, Jun 28 2017

Ok.  Since you set up the panner to use HRTF mode, conceptually you should always hear sound from both left and right, even if the source is full left or right because sound will travel through your head to other ear. How much leaks through depends on the responses we're using.  That would require additional investigation to see if the amount that leaks through is right.

If you truly expect only sound on the right when the source is full right, I suggest setting the panningModel to 'equalpower' which does this.  And it's a lot less CPU hungry than the HRTF panner.

In addition, the HRTF panner has memory so the motion will cause some audio transition between locations due to this memory.  
Hello,

I do know that this is the case and that also while a sound is playing it needs to transition from one location to another one to prevent clicks and audible noises. However, the sound is completely stopped when my location is changed, and it doesn't matter how much time passes between the changing of the panner node and the starting of the sound, the glitch at the beginning where the sound swiftly moves from the old location to the new one is always audible, and depending on the distance between the two points might even take more time.
This problem does not occur in Firefox. Try the same example script in that browser and the audio glitching between the two positions is not audible there.

The HRTF sounds fine to me. I can reliably determine where a sound is coming from. If it helps, I can record a demo of the audio glitches happening in my game as an audio file. It's quite a big project already so I didn't think posting it would be very helpful, especially since it features absolutely no graphics and thus might be confusing. So I created a little test script to try to replicate the error as easily as possible, and I can tell the quick zooming of the sound after clicking play left, waiting for it to finish and clicking play right. And this might be my error, but I don't hear that glitch in Firefox, and even my game sounds fine there.
I've just tried the newest Chrome Canary build to confirm the update hadn't fixed the problem. I should probably have done this beforehand, however the behavior is still the same there. Whenever I setPosition on the panner node, no matter when a source is played through it, it will always quickly move after having set the position even if there's no audio playing at the time of setting the position. So I could be setting a new position, wait 10 seconds and play the sound, and it would still zoom from the old location to the new one. 
Sorry for the comment spam, however I am not sure how to edit a post to add an attachment.
I've changed my reproduction test a little bit. I've made the sound mono so it's location can be easier determined. I've also changed it so the three buttons, set left, set middle and set right only move the panner, and the play button triggers the sound. This way you can click one of the set buttons, the first of which pans fine without problems. Wait for the sound to finish completely, click another button, most noticeable with setleft and setright, and for a few milliseconds at the beginning you hear a click on the left side before the sound quickly moves to the other side when you start playback of the source, no matter how much time has since passed after clicking one of the set buttons.
When I open this example in Firefox and try the above thing, the sound does not have the audible click at the beginning where the sound moves to change position, no matter how much time I wait between clicking the set button and playing the sound.
chrome-sound-test_v2.zip
19.6 KB Download

Comment 6 by rtoy@chromium.org, Jul 20 2017

Thanks for the updated version.  This does make it clearer (except the shuffling example has it's own "clicks" which makes it somewhat difficult).

I tried to reproduce this issue using hoch.github.com/canopy with this script, based on your example, but using oscillators:

// @channels 2
// @duration 1
// @sampleRate 48000

let s0 = context.createOscillator();
s0.type = 'sine';
s0.frequency.value = 440;
let s1 = context.createOscillator();
s1.type = 'sawtooth';
s1.frequency.value = 880;

let p = context.createPanner();
p.panningModel = 'HRTF';
p.distanceModel = 'inverse';
p.refDistance = 1;
p.maxDistance = 10000;
p.rolloffFactor = 1;
p.coneInnerAngle = 360;
p.coneOuterAngle = 0;
p.coneOuterGain = 0;

p.connect(context.destination);

context.listener.setOrientation(0,0,-1,0,1,0);

s0.connect(p);

let stime = 0.25;

p.setPosition(-1, 0, 0);
s0.start();
s0.stop(stime);

context.suspend(stime + 128 / context.sampleRate)
.then(() => {
        p.setPosition(1,0,0);
    })
.then(() => context.resume());

context.suspend(stime + 0.25)
.then(() => {
        p.setPosition(1,0,0);
        s1.connect(p);
        s1.start();
    })
.then(() => context.resume());

There are a couple of issues I can see.  One is that the sine wave oscillator leaks into the second part where the second oscillator starts.  This is a known issue, and we'll be fixing that soon. (We fail to flush out the memory of the HRTF panner when the source goes away.)

Second, as you can see from the amplitudes, the first (left) channel has a high amplitude that quickly decreases, and the second (right) channel has a low amplitude that quickly increases.  This gives the effect of the sound moving from left to right.  The click comes from the sine oscillator leaking into the second part where the sine wave starts at full amplitude.

Our above fix reduces this effect of the amplitude changes, but it's still there. We'll need to examine further to find out why it is. (Part of the amplitude change has to be there because the HRTF panner has filters so there will be some ramp up of the output.)

Comment 7 by rtoy@chromium.org, Jul 28 2017

Some conclusions after doing some more experiments and looking at the code.

One issue is that the click is louder than expected because we don't properly flush out the memory of the HRTF panner.  This will get fixed soon.

Second, you should hear a click because you're suddenly starting a sound that starts off with a non-zero (or very rapidly goes from 0 to non-zero) value.

Third, when the position changes, we cross fade the HRTF responses between the old and new positions.  This takes about 45 ms to finish the cross-fade.  This is why the sound appears to move from left to right.  To make the sound transition smoothly, we need to do something as we switch from one impulse response to another due to the change in position.

So, except for item 1, we're doing what we should.  The cross-fade might be a little long but is otherwise doing what it should do.  
Hi,

I know that fades must be done, it is when the sound is not actually playing and the panner node is updated when the fade confuses me since it doesn't seem to happen in any other browser but chrome, as well as other sound engines off of the web that handle hrtf audio. Thanks a lot for looking into this issue! :)
I'm by no means an expert on binaural audio, but while a sound is stopped and nothing is playing through the panner node, the fade could be nearly instant? Even if I set it to 0, 0, 0, play a sound, wait for it to finish and completely stop, set it to 5, 5, 5, wait, 10, 10, 10, wait, 15, 10, 10, play, it seems to fades from 0, 0, 0 to 15, 10, 10. This takes a very noticeable time. I suppose this is the memory issue you were mentioning? 

Comment 9 by rtoy@chromium.org, Jul 31 2017

The problem where it seems to fade from 0,0,0 to 15,10,10 is due to how we implement processing of the node.  Currently for a panner node, if there is no input connected, no processing happens.  This means that all the calls to setPosition update some internal state, but the actual HRTF panner processor isn't informed of this change, so it looks to it as if the last time the position that was set is 0,0,0, not 10,10,10.

This is something we need to fix somehow.

Comment 10 by rtoy@chromium.org, Aug 8 2017

This will very likely get fixed when we change setPosition() to call setValueAtTime() for each component.  This is required by the spec but Chrome hasn't implemented this yet.
Hi, I wanted to quickly ask of a status update of this issue? It was suspected that with setValueAtTime() etc this might be fixed, however it seems to have gotten worse. It should be possible to update the position of a sound that's not playing and not only sounds that currently output sounds. It's really irritating  if you need to locate a sound directly and it zips around. :(

Comment 12 by rtoy@chromium.org, Mar 30 2018

Nothing yet.  We haven't forgotten about this.

Comment 13 by rtoy@chromium.org, Mar 30 2018

Do you have an example that shows it's gotten worse?  With ToT chromium (or canary), the c#6 no longer shows the first oscillator leaking into the start of the second.  It's what I expect.
Hi, I've updated my example to reflect setValueAtTime. I've also included a recording of the project I'm working on where the problem occurs most noticeably. It's really bad. If you put on headphones you can hear the step sounds pan from left to right. The character walks strictly along the X axis, yet sometimes you can hear the steps completely miss the location they're supposed to play at. I assume it must be the fact that positions of sources that do not play don't get updated and then swish over when they play the next time. It's a big codebase, so I decided not to include it here, but the recording perfectly demonstrates my problem. The test, also, still leaks from channel to channel. I've tried this in the latest Chrome Canary build. 
chrome-sound-test.zip
826 KB Download

Comment 15 by rtoy@chromium.org, Apr 3 2018

The reason for the glitches you hear is because the HRTF panner has memory of the previous position.  It uses the previous position and the current position to select the impulse response.  This makes for nice smooth transitions when moving.

Your example in chrome-sound-test breaks these assumptions.  You play a source at some position, then play a different source at a radically different position but use the same panner.  Because of the memory, you get some sound from the old position before transitioning to the new.

I think the best practice is to use a different panner for each source. This is how the box2d demo works (each block gets its own panner) (See http://googlechromelabs.github.io/web-audio-samples/samples/audio/box2d-js/box2d-audio.html)

The behavior of the HRTF panner is not specified in the webaudio spec, so implementations can do many different, incompatible, things.
Right I will try to create new panner nodes for everytime I play the sounds. The reason I thought this was a bad idea is because I could have around 50 sounds all coming together to create a sound scape, with sounds quickly changing and triggering, so I thought it would be a good idea to re-use sounds that have already been played once and stopped since. Like footsteps. If I have 6 different step sounds and I trigger them randomly, if one step sound has already been set up with panner etc., I thought it would be better for performance to just quickly move that source and play it again rather than re-creating the panner node and re-connecting everything. But if it's not possible to simply snap sources to the new position while they're actually stopped, where crossfading need not occur, I will just setup new panner nodes everytime a sound needs to play at a different position. 

Comment 17 by rtoy@chromium.org, Apr 4 2018

Let us know how it goes. It seems to me that HRTF panners are overkill in this scenario of short sounds like footsteps.  An equalpower panner would work just as well.  Plus, I think you could get away with just one because this panner doesn't have any memory of your previous position.
Right, the scenario is I'm making an audio-only game geared towards people with blindness or visual impairment, so sound is the absolute main focus of the game and locating them exactly is key to being successful or not, that's why I put so much emphasis on them. Using new panners for each sound instance works fine for now, not sure if that will not cause performance problems down the line though? I understand that for smoothe movement, sounds need to have a memory of where they last were, but need that be the case for sounds that are completely stopped and re-emerge at another location? Just curious. Since the sound is stopped and not outputting any data, it should not be audible if the panner is moved quickly before the sound is started? This way works for now however, thank you.

Comment 19 by rtoy@chromium.org, Apr 4 2018

Ah, I see.  That makes sense. 

We tried to think of some ways to do this magically in the panner node, but it was all too magical and would probably cause issues in other scenarios.

HRTF panners are fairly expensive, require 2 convolutions and a fair amount of computation to figure out which responses to use.

An alternative approach is to use just the one panner as you did before, but move the panner to a new position about 45 ms *before* you start playing the sound.  (Comments in the code say it takes about 45 ms to move from the old position to the new.) 
Right, but it doesn't matter how early I move the panner, the position only gets updated as long as sound is put through it, so if it's updated while it's silent the position doesn't seem to change and the glitch it still there as soon as the sound starts. I do update the panner on every position change and the glitches still occur. 

Comment 21 by rtoy@chromium.org, Apr 4 2018

Ah, ok.  The gross hack right now is to connect a ConstantSourceNode with offset 0 to the panner, move the panner and stop the node.

But this is something we can possibly do in the code.  If the panner moves the HRTF panner internals should be updated with the new position as needed.  Maybe.  I'll have to take a closer look.
I'll give that a try, thanks. 

Comment 23 by rtoy@chromium.org, Apr 4 2018

That's a really horrible hack and I wouldn't want to do that in production.

We will look into seeing how to update the HRTF panner internals when things locations change.  This is something that I think we can do and is the right thing.
Right, recreating the panner seems to also work and it seems to be the easier temporary workaround for now so I'll develop with that until the issue can be resolved. :)

Sign in to add a comment