AudioWorkletProcessor process method receives a few incorrect buffers when starting
Reported by
l...@grame.fr,
Apr 17 2018
|
||
Issue description
UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15
Example URL:
Steps to reproduce the problem:
1. Open the given "vumeter.htm" test page which contains a Faust generated vumeter
2. Output signal is incorrect.
3.
What is the expected behavior?
Vumeter page should display the dB level of input audio signal.
What went wrong?
Debugging the code (in process method in vumeter-processor.js file) shows that some incorrect buffers are delivered to the process function when starting.
See lines 347-362:
/*
- here we get a few incorrect buffers (like 4) when processing starts
- processing produces incorrect outputs...
- un-commenting the next section will "fix" the problem
*/
/*
// Check inputs/outputs
if (input === undefined
|| output === undefined
|| input[0][0] === undefined
|| output[0][0] === undefined) {
console.log("Process error");
return true;
}
*/
Did this work before? N/A
Is it a problem with Flash or HTML5? N/A
Does this work in other browsers? Yes
Chrome version: <Copy from: 'about:version'> Channel: n/a
OS Version: OS X 10.11.6
Flash Version:
Contents of chrome://gpu:
,
Apr 17 2018
Do you have a smaller repro case?
,
Apr 17 2018
"Do you have a smaller repro case?" not really, this is already a simple case, can you reproduce the issue?
,
Apr 17 2018
What do you mean by "a few incorrect buffers (like 4)"? - excessive render quanta? - excessive input count? - excessive channel count? - or something else? I don't see any error in the console from the repro case. Do I have to do something to break it?
,
Apr 17 2018
I get several several cycles where "input[0][0] === undefined" happens (line 357). It seems the buffers are not correct during a few cycles. Testing on Canary 68.0.3398.0
,
Apr 17 2018
Here's one suggestion: Can you remove the microphone input and put the oscillator instead? If this works for you, fetching buffers from the microphone somehow takes longer than expected on your system. (...and this becomes getUserMedia issue.) I am running the same version with you, but I don't see the errors so far.
,
Apr 17 2018
Here is a version of the vumeter with oscillator at 440 Hz as input. Same issue, see screenshot ("process input error" happing 6 times)
,
Apr 17 2018
First, the repro case code is really difficult to follow. I still appreciate that you're kindly sharing the example with me, but it takes time to dig into the code and understand what's going on.
From my perspective, I don't think this is an error. Here's what your code does:
A. Loads the worklet code.
A.1. Creates AudioWorkletNode upon the promise resolution.
B. Waits for the resolution of getUserMedia()
B.1. Call getDevice() upon the promise resolution: here you connect the oscillator.
The resolution of A.1 and B.1 are completely orthogonal. I bet A.1 would be much faster than the other. In that case, the AudioWorkletNode will run even without any source connected into it. Thus you get no input (a Float32Array with zero length, which means "unconnected") until the B.1 happens. So this is a correct behavior.
If you want to fix this, I suggest you to synchronize the resolution of multiple promises:
Promise.all([addModulePromise, getUserMediaPromise]).then(() => {
// Start making connections and processing audio.
});
,
Apr 17 2018
OK, I see. So I guess the better way is to have more robust AudioWorkletProcessor "process" code, since we cannot be always sure that the AudioWorkletProcessor is actually connected when "process" method starts to be called right ? (I mean in the general case, since we generate the AudioWorkletNode /AudioWorkletProcessor JavaScript code to be used by client code that may or may not correctly follow the Promise.all model)
,
Apr 17 2018
If your job is to make process() code more robust, yes. I think the code needs to cover several corner cases around the input and the output buffers. Thanks to the *dynamic* nature of Web Audio API, that's what we have been doing for a while - covering tons of corner cases. But with the AudioWorklet now it will become developer's burden. "With great power, comes great responsibility." Right? :) Are you okay with closing this issue?
,
Apr 17 2018
- "with the AudioWorklet now it will become developer's burden." : is this documented anywhere? - OK for closing the issue
,
Apr 17 2018
I don't think it needs to be documented. The AudioWorklet exposes the low-level guts of callback function. What you do in there becomes your responsibility. That's what I meant. Closing as WAI. |
||
►
Sign in to add a comment |
||
Comment 1 by rtoy@chromium.org
, Apr 17 2018