Resolution order of audioWorklet.addModule().then() and WebAssembly.instantiate().then(). |
|||
Issue descriptionExperimental AudioWorklet feature: https://googlechromelabs.github.io/web-audio-samples/audio-worklet/ Repro link: https://faust.grame.fr/modules-worklet/CMajDryHarp.html The page fails with the error message of: DOMException: Failed to construct 'AudioWorkletNode': AudioWorkletNode cannot be created: The node name 'CMajDryHarp' is not defined in AudioWorkletGlobalScope. at new CMajDryHarpNode (https://faust.grame.fr/modules-worklet/CMajDryHarp.html:5445:9) at https://faust.grame.fr/modules-worklet/CMajDryHarp.html:5554:19 at <anonymous> The failing code snippet: // Main thread audioWorklet.addModule("ChromaticSoftHarp-processor.js") .then(function () { let context = new AudioContext(); callback(new ChromaticSoftHarpNode(context, {})); }); // AudioWorkletGlobalScope WebAssembly.instantiate(faust.atob(getBase64CodeCMajDryHarp()), faust.importObject) .then(dsp_module => { faust.CMajDryHarp_instance = dsp_module.instance; registerProcessor('CMajDryHarp', CMajDryHarpProcessor); }); So the ‘registerProcessor’ call is doe when the WebAssembly.instantiate promise is finished. So it seems the Node side doing the AWContext.addModule, then new ChromaticSoftHarpNode(audio_context, {})); does no « wait » until the WebAssembly.instantiate then the registerProcessor has been done.
,
Nov 28 2017
,
Nov 28 2017
"how about instantiating the WASM module in the main scope and send it to the processor via MessagePort?" So "Main thread" would do: - WebAssembly.instantiate to create the wasm module - when promise resolves, do audioWorklet.addModule to create the CMajDryHarpProcessor - post the created WASM module (using this.port.posMessage(...)) - but then CMajDryHarpProcessor constructor already needs the WASM module... But the CMajDryHarpProcessor would set its "this.port.onmessage" handler inside its constructor right ? And this handler is going to be called in an asynchronous way? (relative to the CMajDryHarpProcessor constructor itself), So I don't see how it can work.
,
Nov 28 2017
> then CMajDryHarpProcessor constructor already needs the WASM module I don't know how processor and WASM module are structured. I thought the WASM module would be a simple function call inside of process() method. Until the module is ready, the processor can bypass the input. When sending a WASM module from the main thread to the processor is completed, the process() kernel can be replaced with the module. Another question: does adding setTimeout() can be a workaround? If so, it is a definitely a timing issue. Otherwise, my suspicion is registerProcessor() might be failing silently.
,
Nov 28 2017
This is what I get from the console:
AudioWorklet.addModule done <== (A)
CMajDryHarp.html:5558 DOMException: Failed to construct 'AudioWorkletNode': AudioWorkletNode cannot be created: The node name 'CMajDryHarp' is not defined in AudioWorkletGlobalScope.
at new CMajDryHarpNode (http://localhost:3030/bugs/789183-resolution-order/CMajDryHarp.html:5445:9)
at http://localhost:3030/bugs/789183-resolution-order/CMajDryHarp.html:5556:19
at <anonymous>
CMajDryHarp.html:5558 Faust CMajDryHarp cannot be loaded or compiled <== (C)
CMajDryHarp-processor.js:342 WebAssembly.instantiate done <== (B)
This log suggests (A) finishes earlier than the completion of (B).
We also have (C) that means the invocation of addModule() was not successful. (e.g. error in the source code)
In this sense, the log actually makes sense. Are we absolutely certain that processor.js file does not have any error in it?
,
Nov 28 2017
nhiroki@ 2 questions. Does ThreadedWorkletObjectProxy::DidEvaluateModuleScript() wait for the completion of WorkletGlobalScope::registerFoo() methods? What if the script code in the global scope has a promise inside? Then does it wait for the resolution of promise before it fires DidEvaluateModuleScript()?
,
Nov 28 2017
"I don't know how processor and WASM module are structured." ==> look at the CMajDryHarp-processor.js code ((-; As you can see the CMajDryHarpProcessor constructor does some initialization (preparation of audio buffers to be used later on in "process", init the DSP instance...) This code has to be done once. Postponing it inside "process" would be quite ugly....
,
Nov 28 2017
"Are we absolutely certain that processor.js file does not have any error in it?"
Just to check, I tried to change the order (in CMajDryHarp-processor.js file)
registerProcessor('CMajDryHarp', CMajDryHarpProcessor);
WebAssembly.instantiate(faust.atob(getBase64CodeCMajDryHarp()), faust.importObject)
.then(dsp_module => {
///faust.CMajDryHarp_instance = dsp_module.instance;
console.log("WASM compiled");
})
.catch(function(error) { console.log(error); console.log("Faust CMajDryHarp cannot be loaded or compiled"); });
and the console.log("WASM compiled"); is done (even if at the end the CMajDryHarpProcessor class cannot run correctly, since the constructor cannot access the WASM module and so on...).
The same kind of logic was working before the https, and change to access of AudioWorklet. So the only change I've done is :
var AWContext = window.audioWorklet || BaseAudioContext.AudioWorklet;
in Node (main thread) side.
,
Nov 28 2017
> var AWContext = window.audioWorklet || BaseAudioContext.AudioWorklet; We can get this one out of the picture. The transition of AudioWorklet from window to BaseAudioContext has not happened yet. So it is irrelevant to this issue.
,
Nov 28 2017
And the same generated Faust/WASM code is deployed and works in ScripProcessor mode here: http://faust.grame.fr/modules/ (just recompiled all pages with faust 2.5.9 and pushed them on faust.grame.fr)
,
Nov 28 2017
A version that works: https://faust.grame.fr/modules-worklet/ChromaticSoftHarp.html - doing 'registerProcessor' first (outside of the WebAssembly.instantiate promise) - adding 'setTimeout' after the 'addModule(...)' on Node side But obviously this is not really satisfactory.
,
Nov 28 2017
Re #11: I think you just confirmed a very important fact. If setTimeout() fixes this issue, then we don't have to worry about the malfunction in JS parsing and evaluation side anymore. This is strictly a timing issue now; the issue that I described in the comment #6 might be the actual case.
,
Nov 29 2017
letz@ Please take a look at: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WebAssembly/Instance In this way, you can create an instance synchronously. For AudioWorklet purpose, I think the synchronous instantiation makes sense.
,
Nov 29 2017
I changed processor.js (l.345~351) like this and the demo loads okay.
```
let wasmModule = new WebAssembly.Module(faust.atob(getBase64CodeCMajDryHarp()));
let dspInstance = new WebAssembly.Instance(wasmModule, faust.importObject);
faust.CMajDryHarp_instance = dspInstance.instance;
registerProcessor('CMajDryHarp', CMajDryHarpProcessor);
```
,
Nov 29 2017
In short, the problem here is the async waiting on promise. 1. The parse/eval starts in AudioWorkletGlobalScope. 2. The eval process ended when the parse reaches the end of script code. 3. Technically the code eval is finished, so it invokes "AudioWorkletObjectProxy::DidEvaluateModuleScript" without waiting on the promise. 4. AudioWorklet.addModule().then() fired. 5. Later the promise is resolved on WebAssembly.initiate().then(). Thus at the step 4, the class definition is not ready for sure because registerProcessor() has not even called yet.
,
Nov 29 2017
The proposed solution fix the problem. Thanks.
,
Feb 14 2018
Closing this per #16. I do not believe this will be addressed by Worklet infrastructure either. |
|||
►
Sign in to add a comment |
|||
Comment 1 by hongchan@chromium.org
, Nov 28 2017