New issue
Advanced search Search tips

Issue 833750 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Apr 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

AudioWorkletProcessor process method receives a few incorrect buffers when starting

Reported by l...@grame.fr, Apr 17 2018

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15

Example URL:

Steps to reproduce the problem:
1.  Open the given "vumeter.htm" test page which contains a Faust generated vumeter

2. Output signal is incorrect. 

3. 

What is the expected behavior?
Vumeter page should display the dB level of input audio signal.

What went wrong?

Debugging the code (in process method in vumeter-processor.js file) shows that some incorrect buffers are delivered to the process function when starting.

See lines 347-362:

         /*
        	- here we get a few incorrect buffers (like 4) when processing starts
        	- processing produces incorrect outputs...
     	 	- un-commenting the next section will "fix" the problem
     	*/ 

        /*
        // Check inputs/outputs
        if (input === undefined
            || output === undefined
            || input[0][0] === undefined
            || output[0][0] === undefined) {
            console.log("Process error");
            return true;
        }
    	*/

Did this work before? N/A 

Is it a problem with Flash or HTML5? N/A

Does this work in other browsers? Yes

Chrome version: <Copy from: 'about:version'>  Channel: n/a
OS Version: OS X 10.11.6
Flash Version: 

Contents of chrome://gpu:
 
vumeter-processor.js
18.8 KB View Download
vumeter.html
285 KB View Download

Comment 1 by rtoy@chromium.org, Apr 17 2018

Components: -Internals>Media Blink>WebAudio

Comment 2 by rtoy@chromium.org, Apr 17 2018

Do you have a smaller repro case?

Comment 3 by l...@grame.fr, Apr 17 2018

"Do you have a smaller repro case?" not really, this is already a simple case, can you reproduce the issue?
What do you mean by "a few incorrect buffers (like 4)"?

- excessive render quanta?
- excessive input count?
- excessive channel count?
- or something else?

I don't see any error in the console from the repro case. Do I have to do something to break it?

Comment 5 by l...@grame.fr, Apr 17 2018

I get several several cycles where "input[0][0] === undefined" happens (line 357). It seems the buffers are not correct during a few cycles. Testing on Canary 68.0.3398.0 
Here's one suggestion:

Can you remove the microphone input and put the oscillator instead? If this works for you, fetching buffers from the microphone somehow takes longer than expected on your system. (...and this becomes getUserMedia issue.)

I am running the same version with you, but I don't see the errors so far.

Comment 7 by l...@grame.fr, Apr 17 2018

Here is a version of the vumeter with oscillator at 440 Hz as input. Same issue, see screenshot ("process input error" happing 6 times)
vumeter.html
285 KB View Download
vumeter-processor.js
18.7 KB View Download
vumeter error.png
860 KB View Download
First, the repro case code is really difficult to follow. I still appreciate that you're kindly sharing the example with me, but it takes time to dig into the code and understand what's going on.

From my perspective, I don't think this is an error. Here's what your code does:

A. Loads the worklet code.
  A.1. Creates AudioWorkletNode upon the promise resolution.

B. Waits for the resolution of getUserMedia()
  B.1. Call getDevice() upon the promise resolution: here you connect the oscillator.

The resolution of A.1 and B.1 are completely orthogonal. I bet A.1 would be much faster than the other. In that case, the AudioWorkletNode will run even without any source connected into it. Thus you get no input (a Float32Array with zero length, which means "unconnected") until the B.1 happens. So this is a correct behavior.

If you want to fix this, I suggest you to synchronize the resolution of multiple promises:

Promise.all([addModulePromise, getUserMediaPromise]).then(() => {
 // Start making connections and processing audio.
});

Comment 9 by l...@grame.fr, Apr 17 2018

OK, I see. 

So I guess the better way is to have more robust AudioWorkletProcessor "process" code, since we cannot be always sure that the AudioWorkletProcessor is actually connected when "process" method starts to be called right ? (I mean in the general case, since we generate the AudioWorkletNode /AudioWorkletProcessor JavaScript code to be used by client code that may or may not correctly follow the Promise.all model)


If your job is to make process() code more robust, yes. I think the code needs to cover several corner cases around the input and the output buffers.

Thanks to the *dynamic* nature of Web Audio API, that's what we have been doing for a while - covering tons of corner cases. But with the AudioWorklet now it will become developer's burden.

"With great power, comes great responsibility." Right? :)

Are you okay with closing this issue?

Comment 11 by l...@grame.fr, Apr 17 2018

- "with the AudioWorklet now it will become developer's burden." : is this documented anywhere?

- OK for closing the issue
Status: WontFix (was: Unconfirmed)
I don't think it needs to be documented. The AudioWorklet exposes the low-level guts of callback function. What you do in there becomes your responsibility. That's what I meant.

Closing as WAI.

Sign in to add a comment