New issue
Advanced search Search tips

Issue 789183 link

Starred by 0 users

Issue metadata

Status: WontFix
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 469639



Sign in to add a comment

Resolution order of audioWorklet.addModule().then() and WebAssembly.instantiate().then().

Project Member Reported by hongchan@chromium.org, Nov 28 2017

Issue description

Experimental AudioWorklet feature:
https://googlechromelabs.github.io/web-audio-samples/audio-worklet/

Repro link:
https://faust.grame.fr/modules-worklet/CMajDryHarp.html

The page fails with the error message of:
DOMException: Failed to construct 'AudioWorkletNode': AudioWorkletNode cannot be created: The node name 'CMajDryHarp' is not defined in AudioWorkletGlobalScope.
  at new CMajDryHarpNode (https://faust.grame.fr/modules-worklet/CMajDryHarp.html:5445:9)
  at https://faust.grame.fr/modules-worklet/CMajDryHarp.html:5554:19
  at <anonymous>

The failing code snippet:

// Main thread
audioWorklet.addModule("ChromaticSoftHarp-processor.js")
    .then(function () {
         let context = new AudioContext();
         callback(new ChromaticSoftHarpNode(context, {}));
    });

// AudioWorkletGlobalScope
WebAssembly.instantiate(faust.atob(getBase64CodeCMajDryHarp()), faust.importObject)
            .then(dsp_module => {
                  faust.CMajDryHarp_instance = dsp_module.instance;
                  registerProcessor('CMajDryHarp', CMajDryHarpProcessor);
            });

So the ‘registerProcessor’ call is doe when the WebAssembly.instantiate promise is finished. So it seems the Node side doing the AWContext.addModule, then new ChromaticSoftHarpNode(audio_context, {})); does no « wait » until the WebAssembly.instantiate then the registerProcessor has been done.
 
From what I see, this is a timing issue between the promise resolutions: addModule().then() and instantiate.then(). I don't think what you're doing is wrong, but here's my suggestion: how about instantiating the WASM module in the main scope and send it to the processor via MessagePort?
Cc: l...@grame.fr

Comment 3 by l...@grame.fr, Nov 28 2017

"how about instantiating the WASM module in the main scope and send it to the processor via MessagePort?"

So "Main thread" would do:

- WebAssembly.instantiate to create the wasm module
- when promise resolves, do audioWorklet.addModule to create the CMajDryHarpProcessor 
- post the created WASM module (using this.port.posMessage(...))
- but then CMajDryHarpProcessor constructor already needs the WASM module... But the CMajDryHarpProcessor would set its "this.port.onmessage" handler inside its constructor right ? And this handler is going to be called in an asynchronous way? (relative to the CMajDryHarpProcessor constructor itself), 

So I don't see how it can work.

> then CMajDryHarpProcessor constructor already needs the WASM module

I don't know how processor and WASM module are structured. I thought the WASM module would be a simple function call inside of process() method. Until the module is ready, the processor can bypass the input. When sending a WASM module from the main thread to the processor is completed, the process() kernel can be replaced with the module.

Another question: does adding setTimeout() can be a workaround? If so, it is a definitely a timing issue. Otherwise, my suspicion is registerProcessor() might be failing silently.
This is what I get from the console:

AudioWorklet.addModule done      <== (A)
CMajDryHarp.html:5558 DOMException: Failed to construct 'AudioWorkletNode': AudioWorkletNode cannot be created: The node name 'CMajDryHarp' is not defined in AudioWorkletGlobalScope.
    at new CMajDryHarpNode (http://localhost:3030/bugs/789183-resolution-order/CMajDryHarp.html:5445:9)
    at http://localhost:3030/bugs/789183-resolution-order/CMajDryHarp.html:5556:19
    at <anonymous>
CMajDryHarp.html:5558 Faust CMajDryHarp cannot be loaded or compiled     <== (C)
CMajDryHarp-processor.js:342 WebAssembly.instantiate done    <== (B)

This log suggests (A) finishes earlier than the completion of (B).
We also have (C) that means the invocation of addModule() was not successful. (e.g. error in the source code)

In this sense, the log actually makes sense. Are we absolutely certain that processor.js file does not have any error in it?
nhiroki@ 2 questions.

Does ThreadedWorkletObjectProxy::DidEvaluateModuleScript() wait for the completion of WorkletGlobalScope::registerFoo() methods?

What if the script code in the global scope has a promise inside? Then does it wait for the resolution of promise before it fires DidEvaluateModuleScript()?
CMajDryHarp-processor.js
69.9 KB View Download
CMajDryHarp.html
337 KB View Download

Comment 7 by l...@grame.fr, Nov 28 2017

"I don't know how processor and WASM module are structured." ==> look at the CMajDryHarp-processor.js code ((-;

As you can see the CMajDryHarpProcessor constructor does some initialization (preparation of audio buffers to be used later on in "process", init the DSP instance...) This code has to be done once. Postponing it inside "process" would be quite ugly....

Comment 8 by l...@grame.fr, Nov 28 2017

"Are we absolutely certain that processor.js file does not have any error in it?"

Just to check, I tried to change the order (in CMajDryHarp-processor.js file)

registerProcessor('CMajDryHarp', CMajDryHarpProcessor);

WebAssembly.instantiate(faust.atob(getBase64CodeCMajDryHarp()), faust.importObject)
            .then(dsp_module => {
             ///faust.CMajDryHarp_instance = dsp_module.instance;
             console.log("WASM compiled");
            })
            .catch(function(error) { console.log(error); console.log("Faust CMajDryHarp cannot be loaded or compiled"); });

and the console.log("WASM compiled"); is done (even if at the end the CMajDryHarpProcessor class cannot run correctly, since the constructor cannot access the WASM module and so on...).

The same kind of logic was working before the https, and change to access of AudioWorklet. So the only change I've done is :

var AWContext = window.audioWorklet || BaseAudioContext.AudioWorklet;

in Node (main thread) side.
> var AWContext = window.audioWorklet || BaseAudioContext.AudioWorklet;

We can get this one out of the picture. The transition of AudioWorklet from window to BaseAudioContext has not happened yet. So it is irrelevant to this issue.

Comment 10 by l...@grame.fr, Nov 28 2017

And the same generated Faust/WASM code is deployed and works in ScripProcessor mode here: http://faust.grame.fr/modules/
(just recompiled all pages with faust 2.5.9 and pushed them on faust.grame.fr)

Comment 11 by l...@grame.fr, Nov 28 2017

A version that works: https://faust.grame.fr/modules-worklet/ChromaticSoftHarp.html

- doing 'registerProcessor' first (outside of the WebAssembly.instantiate promise)

- adding 'setTimeout' after the 'addModule(...)' on Node side

But obviously this is not really satisfactory.
Re #11:

I think you just confirmed a very important fact. If setTimeout() fixes this issue, then we don't have to worry about the malfunction in JS parsing and evaluation side anymore.

This is strictly a timing issue now; the issue that I described in the comment #6 might be the actual case.
letz@

Please take a look at:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Instance

In this way, you can create an instance synchronously. For AudioWorklet purpose, I think the synchronous instantiation makes sense.
I changed processor.js (l.345~351) like this and the demo loads okay.

```
let wasmModule = new WebAssembly.Module(faust.atob(getBase64CodeCMajDryHarp()));
let dspInstance = new WebAssembly.Instance(wasmModule, faust.importObject);
faust.CMajDryHarp_instance = dspInstance.instance;
registerProcessor('CMajDryHarp', CMajDryHarpProcessor);
```
In short, the problem here is the async waiting on promise.

1. The parse/eval starts in AudioWorkletGlobalScope.
2. The eval process ended when the parse reaches the end of script code.
3. Technically the code eval is finished, so it invokes "AudioWorkletObjectProxy::DidEvaluateModuleScript" without waiting on the promise.
4. AudioWorklet.addModule().then() fired.
5. Later the promise is resolved on WebAssembly.initiate().then().

Thus at the step 4, the class definition is not ready for sure because registerProcessor() has not even called yet.

Comment 16 by l...@grame.fr, Nov 29 2017

The proposed solution fix the problem. Thanks.
Status: WontFix (was: Assigned)
Closing this per #16.

I do not believe this will be addressed by Worklet infrastructure either.

Sign in to add a comment