Tested on 64.0.3282.24 (Build officiel) beta (64 bits)
(My Chrome Canary version does now work anymore for whatever reason)

This can even be heard when playing a single note for example with Midi Keys (http://www.manyetas.com/creed/midikeys.html). Opens the clarinetMIDI page https://faust.grame.fr/modules-worklet/clarinetMIDI, hit a single note with Midi Keys  and let it playing ==> audio glitches

Reporter@ Thanks for the issue.

Tested this issue on Mac OS 10.12.6 and Windows 10 on the latest Canary 65.0.3309.0, Stable 63.0.3239.108 and Beta 64.0.3282.39 by following the steps mentioned in the original comment.

On clicking on Polyphonic MIDI aware pages links, we are seeing this message "WebAssembly or AudioWorklet is not supported in this browser" and unable to load the page.

Also on navigating to the link mentioned in comment #2, can see 404 - page not found error.
Attached is the screen cast for reference.

Request you to please check and confirm if anything is missed from our end in testing this issue. 
Also request you to please provide us another URL where this issue is reproducible, which will help in further triaging of this issue.

Thanks...

Deleted: 796330.webm 7.8 MB

	796330.webm 7.8 MB View Download

1) As explained in the page, Audio Worklet mode has to be explicitly activated in Chrome Canary, see : https://googlechromelabs.github.io/web-audio-samples/audio-worklet/

2) the second link is : https://faust.grame.fr/modules-worklet/clarinetMIDI.html 

Thank you for providing more feedback. Adding requester "sc00335628@techmahindra.com" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Hi Stéphen,

Here's my finding from a brief look:
- In most cases the renderer finishes its task before the render quantum.
- When GC happens (when I pressed 7~8 keys at the same time), it blocks the rendering thus the buffer underflow follows.

Questions to you because I can't dig deeper the generated JS code.
- How many AudioWorkletNode/Processor are you using? Frequent creation/destruction of processors causes GC more often.
- Are you absolutely sure that WASM code does not generate garbage at all? If you believe so, how can we verify that?

What I can do:
- Change how AudioWorkletGlobalScope uses AudioBuffer so we don't have to create buffers every render quantum. No allocation, no GC.

> We hear audio glitches, much more that the equivalent code written with the old ScriptProcessorNode model.

I don't think this is true. SPN has a variable buffer size and is double-buffered, so it might look like it can handle more stress at the expense of large latency. AudioWorkletProcessor is required to be finish the task in 128 samples.

Fine tuning the sweet spot for the AudioWorklet implementation is actually quite crucial, I would say.

Deleted: Screen Shot 2018-01-03 at 1.19.28 PM.png 44.6 KB

	Screen Shot 2018-01-03 at 1.19.28 PM.png 44.6 KB View Download

- even for polyphonic aware pages, there is a single AudioWorkletNode/Processor allocation, done at init time. The polyphonic AudioWorkletProcessor allocates a set a voices, each one being Faust generated WASM instance. When receiving keyOn, the MIDI handler selects a free voice and play the received pitch/velocity using it. Note that the glitch even occurs when playing a long single note (to test it, you can hit a key and just let the note playing several seconds)

- I am sure that WASM code does not allocate dynamic memory and should never cause GC by itself. Faust model assure that the generated DSP instance memory is allocated once at init time.

- so basically I don't see how the Faust related code can cause memory allocation and so GC by itself. 

- "Change how AudioWorkletGlobalScope uses AudioBuffer so we don't have to create buffers every render quantum. No allocation, no GC." So you mean the current code does create buffers every render quantum ? Hum... then this should be the first change to test yes.

Thanks for confirmation. Yes, that was my understanding on Faust examples.

> So you mean the current code does create buffers every render quantum ? Hum... then this should be the first change to test yes.

This fix might improve the performance, but I've seen other complex AudioWorklet+WASM examples that don't cause GC like this. Not sure what the differences are. With that said, the fix will be the first optimization on AudioWorklet.

Is there any way to get a finer understanding of which JS code causes GC ? 

At least not in Chrome. Emscripten does not have a tool for that?

Or you can easily compare the tracing between an AudioWorklet page doesn't produce glitches and the page in this issue. By comparing frequency of GC in two cases, we might learn something useful.

In the meantime, I am trying to introduce the "buffer pool" approach here:
https://chromium-review.googlesource.com/#/c/chromium/src/+/861167

Hope we can mitigate this soon.

Emscripten code is not supposed to cause any GC : memory has to be explicitly managed like in C/C++. 

"buffer pool" approach : thanks for that, waiting for the result to be tested!

Well, this is disappointing. I ran the Clarinet demo above with the patched build (reusing DOMFloat32Array for every RQ) but it didn't make any difference.

FYI, I pressed 4 simultaneous keys for the testing until the tracer buffer reaches to 10%.

Deleted: Screen Shot 2018-01-11 at 11.08.22 AM.png 154 KB

	Screen Shot 2018-01-11 at 11.08.22 AM.png 154 KB View Download

So even the patched build causes frequent GC and we need to find out what causes it. Apparently creating DOMFloat32Array was not the problem.

OK. Can you event simplify by just hitting one key, let the note play, and see what happens ? (trace GC...etc..)

re #11:
I've tried "Performance" tab in the dev tool and it actually shows you what triggers the memory allocation and the type of GC (DOM, Minor, Major). This might be useful to you.

re #16:
The tracing result is almost identical to the 4-voice case. (for both the patched build and Canary) We can confirm that the number of voice is not an issue.

Here is a even simpler example: a monophonic clarinetMIDI, with completely deactivated GUI (the note is started when loading the HTML page and the page stays black), and simplified clarinetMIDIProcessor.process function in the clarinetMIDI-processor.js file. But sill audio glitches...

Deleted: clarinetMIDI-processor.js 28.7 KB

clarinetMIDI-processor.js 28.7 KB View Download

Deleted: clarinetMIDI.html 294 KB

clarinetMIDI.html 294 KB View Download

I can reproduce audio glitches ("crackling") by simply running the AudioWorklet bypass demo:

https://googlechromelabs.github.io/web-audio-samples/audio-worklet/basic/hello-audio-worklet.html

just let it run, I am testing currently in 65.0.3318.0 (Official Build) canary (64-bit on a 2,5 GHz Intel Core i7 MacBook Pro (10.12.6), and after a while (ca. 1 min) there will be small crackles in the sound - they seem to be worse (happening more often) when you put the Chrome application in the background (e.g. having the Apple Mail client in the foreground), but I also have them running Chrome in the foreground.

When profiling the code they seem to be loosely connected with major garbage collection, but this could also be a total red herring.

One of optimization CL has landed:
https://chromium-review.googlesource.com/c/chromium/src/+/876183

letz@ & stefan.brunner@

I have done some profiling and confirmed the new optimization removes the glitch from the example in #18. The optimization CL will be available in Canary tomorrow, so let me know how it works for you.

Just played a couple of minutes of https://faust.grame.fr/modules-worklet/clarinetMIDI.html with ToT build (which has the optimization patch), and have not heard glitches so far.

Testing https://faust.grame.fr/modules-worklet/clarinetMIDI.html

Version 66.0.3330.0 (Build officiel) canary (64 bits) indeed cause much less glitches, but still some from time to time.

Re #23:

Glad to hear that. This is how we make progress - could you provide the trace of your example with glitch? (be sure to check audio, blink, blink_gc, webaudio in the trace option)

We developed a "plugin host" that accepts so far some heavy synths and some FAUST plugins that run in audio worklets. I made a vidéo 10 days ago that showed that when the instruments are playing and if you're dragging GUI elements, for example, we could here small glitches (video: https://youtu.be/dGJVxSYaA_A?t=2m8s), but since Canary version 66.0.3330.0 they seemed to have disappeared. I haven't run the examples for more than 10 minutes, and we did not generate a lot of notes. But with a midi keyboard, a dx7, an Oberheim OBX (webaudiomodules plugins), a zita_verb (FAUST), two guitar tube amp sims, a delay pedal, chorus, an an analyser plugin, it ran without glitches.
Keep doing the good work :-)
We'll try to make some stress tests during the next days...

Ah, you can try yourself: https://wasabi.i3s.unice.fr/pedalboard/ 
Right now it's asking for credentials, we'll fix that within 12h. In between you watch this video: https://www.youtube.com/watch?v=elbjh6tBK6U

It is indeed difficult to cause glitches in "normal" usage.  To facilitate testing on the page https://faust.grame.fr/modules-worklet/clarinetMIDI.html, note that you can use the sustain knob (top-right to be set to 1) to sustain playing notes. This way you can start 1, 2, 3... notes and let them playing continuously.

I could get a glitch by opening/closing the Javascript console. Here is the trace (attached file). 

Deleted: trace_clarinetMIDI.json.gz 456 KB

trace_clarinetMIDI.json.gz 456 KB Download

Another test that is interesting to do:

- go to the following Faust test page https://faust.grame.fr/dynamic/faustlive-wasm.html

- select AudioWorlet as the audio rendering mode in the "ScriptProcessor/AudioWorklet" menu

- select ON in the "Polyphonic instrument" menu

- drop some DSP examples from here : https://github.com/grame-cncm/faust/tree/master-dev/examples/physicalModeling (you may have to clone the git repository to easily access the .dsp files)

- the clarinetMIDI.dsp of violinMIDI.dsp are good examples.

- set the sustain knob to 1, so that played nots stay on

- then you can load the CPU more and more by adding playing notes, and see when is start glitching. 

- you can change the maximum number of available voices in the "Polyphonic instrument" menu. This way you can raise the number of playable voices, and so the CPU limits all this mixed voices will reach, and trace glitching.

Following the previously exposed procedure:

- I do add voices until the CPU of "Google Chrome  Helper" reach 50%. I don't hear glitches

- I start a compilation in a terminal. Then I hear glitches. 

- I would say this could be a thread scheduling issue ? That is since the AW thread is actually not a "real" real-time thread on OS X, we hit another limit here?

- my point is : GC is a first part of the problem that has to be solved and it is much better now. But them it seems we now hit the thread scheduling issue, and progressively loading the CPU with this polyphonic instrument is a good way to test.

cool - i can confirm that my demo use case is fixed (no more glitches in the AudioWorklet bypass demo), and also no glitches in most of my other use cases

i can still reproduce some in heavier applications (with only ca. 35% CPU load), but i will have to dig deeper to actually have anything report-able

thanks !

Re #30:

> I start a compilation in a terminal. Then I hear glitches.

Please take the trace when you do the performance testing. It really helps us to figure out why it's happening.

> I would say this could be a thread scheduling issue ? That is since the AW thread is actually not a "real" real-time thread on OS X, we hit another limit here?

We don't know. It might be underflow from the rendering thread, or the callback from the device thread is getting descheduled. The trace and the time profiling must be done.

> it seems we now hit the thread scheduling issue

If we can confirm on why, then we can try to fix it. :)

- Can you reproduce the problem on your machine?

- Here is a trace of the clarinetMIDI.dsp instrument, adding voices (with the sustain knob at 1), loading the "Google Chrome Helper" process up to 60%, then starting a compilation. Glitches start to appear.

Deleted: trace_clarinetGlitch2.json.gz 2.8 MB

trace_clarinetGlitch2.json.gz 2.8 MB Download

letz@

I have not reproduced the case in #29 yet. By looking at the trace in #33, some render quanta are taking longer than others. (See attached pic) My guess is GC. The recent optimization removed a big portion of it, but there still are small amounts of GC in the thread.

Two important things:

1. As we already have discussed, using the realtime priority thread for JS execution is not our option for the security reason. If we want to go down the path, we might have to convince a LOT of people along the way and I don't think it's possible in Web Audio API V1.

2. I am not sure if AudioWorklet system should be responsible for glitches from the interaction between the browser process and some arbitrary compilation task. Figuring out what OS kernel does to accommodate these two cpu-intensive processes seems to be outside of the scope of AudioWorklet project.

My question again:
Did your example produce no glitch when it used ScriptProcessorNode?

Deleted: Screen Shot 2018-02-13 at 2.05.18 PM.png 28.5 KB

	Screen Shot 2018-02-13 at 2.05.18 PM.png 28.5 KB View Download

And what is the buffer size used in the ScriptProcessorNode version?

- I still don't understand why you guys don't try to reproduce the glitches on your machines, do tracing, understand what happens and so on. Is the testing procedure not explained enough? 

- with the same clarinetMIDI example, I do also hear the glitches with ScriptProcessorNode, running at 256 frames, with the "Google Chrome Helper" process loaded about 50%/60% of the CPU, and doing compilation in a terminal

- what we have to compare against is native audio applications: they would certainly *not* glitch in a same testing context (128 frames, 50/60% DSP CPU load). Why ? because they execute audio in a RT thread, period. If AudioWorklet implementation cannot achieve the same result, then I guess professional level audio applications will certainly not be doable on the Web platform, that would be a pity.

- are you able to build a test version of Chrome, with the realtime priority thread for JS execution activated ? This would at least help to characterize the glitch issue. If a RT thread for JS execution does not glitch with the same testing conditions, then we would know (almost for sure...) that the glitches come from thread scheduling priority.
And this RT version could possibly be activated with a special preference? Or could a CPU load guard could be installed to avoid to completely lock the Chrome application in case of a misbehaving application? I'm sure different strategies could be found.

I can understand your frustration, but please keep it mind that this is just the beginning of AudioWorklet project. Let us not conclude this discussion with "It can't do anything serious". We have been making process over last 8 years, and I don't see us stopping it anytime soon.

Few more technical points:

> Why ? because they execute audio in a RT thread, period.

The definition of RT thread can be different between different OS or application. In Chrome, we have several ENUMS of thread priority for each platform and they have slightly different meaning. (AFAIK, you can't acquire RT thread on Linux without root privilege which Chrome does not have)

> are you able to build a test version of Chrome, with the realtime priority thread for JS execution activated

I already built it a while ago to fix the deshceduling issue. I actually did not make much of difference. I have not followed up on it since. I will try to revisit in a couple of days.

> And this RT version could possibly be activated with a special preference? Or could a CPU load guard could be installed to avoid to completely lock the Chrome application in case of a misbehaving application? I'm sure different strategies could be found.

This certainly can be considered and discussed. However, these ideas do no fit it Web Audio API V1's time frame. It needs a considerable amount of spec work and a new set of security review. When the time comes, we can talk about strategies and designs.

- well 8 years is already huge..., what can I say more? ((-;

- it would be interesting so see what happens at least on OS X where RT threads are a completely separated class (mach time-constraints threads). This is used for CoreAudio and CoreMIDI threads since the beginning of OS X (like 2001). On Linux AFAICS you don't need root privilege: the system have to be configured once in some configuration files, then RT threads (SCHED_FIFO class) can be used. This can be asked on linux-audio-dev list again to check is this is still the case, but I don't think the overall model has changed.

- if this is not fixed for V1, then I guess "glitch bug reports" will come again and again, if real (= more demanding) audio applications start to appear on the Web platform 

Glitches can even be produced more easily by simply scrolling a page in a editor (like the Xcode one, or scrolling a "man page" in the terminal), with the  "Google Chrome Helper" process even less CPU loaded. 

Re #38:

> well 8 years is already huge..., what can I say more? ((-;
I really don't think 8 years of work in WebAudio is in vain just because the arbitrary user code cannot use the RT thread.

> it would be interesting so see what happens at least on OS X where RT threads...
Technicality is not a problem here. As I mentioned, the actual problem is that we need to convince other stakeholders. There will be overhead and security concern no matter what. The history of Web Audio API is the evidence of that painstaking process.

> if this is not fixed for V1, then I guess "glitch bug reports" will come again and again...
I see this as a "chicken and egg" problem. More bug reports can justify the use case, but the lack of RT thread might be a roadblock for many serious audio developers. We understand the fact and are willing to go through the process to push it forward. I am certain that we will discuss this when the V2 discussion starts.


Re #39:
I can explain why that's the case. The browser's main thread runs on "DISPLAY" priority and a thread in the web platform cannot go higher than that. Even getting the "DISPLAY" priority to WebAudio thread needed valid justification with many Chromium engineers.

Worklets have only been around for a few months, not 8 years, so I find that comment disingenuous at best.

If RT threads require some custom setup of your linux system, then, as far as Chrome is considered, they don't exist.  It's a huge burden to maintain this across all varieties of linux.

Real-time priority will not happen in V1.  We will certainly look into it for V2, but the nature of the web is that if something can be abused, it will be.  This is why we can't easily have nice things. :-(

My point is not to be "disingenuous" at all ((-;  but to try to give testing material and feedback as precise as I can.

If RT threads cannot be happen in V1, then OK. But if will help developers to properly describe what they can expect, and what are the current limitations.

Reproducing the case in #29 on MacOS 10.13.3.

* Setup
- Goto: https://faust.grame.fr/dynamic/faustlive-wasm.html
- Drop clarinetMIDI.dsp file to the page.
- Polyphonic instrument "on" with 128 voices.
- Press my MIDI keyboard rapidly until Chromium Helper goes to ~70%.
- Starts building Chromium debug build.
- All CPU goes up to 100%.

* On ToT (66.0.3347.0)
- Audio drops out.

* On ToT with local RT thread patch
- Audio doesn't drop out, but glitches.

* Notes
- So glitch is better than drop out? They are equally bad from my perspective.
- Even with the RT thread the audio quality suffers when all CPU goes up to 100%. Like I mentioned, the RT thread in Chromium might behave different from the RT thread in various operating systems.
- I am quite impressed by the fact that AudioWorklet + Faust Clarinet can handle ~100 voices without glitch until the compilation starts.

Interesting: 

- what do you mean by "drop out" versus "glitches"?

- RT thread in Chromium: how is this thread configured ? (at lest on OS X) Any code to look at?

Another question: how does it feels with a heavily loaded AudioWorklet thread in the local RT thread patched version ? I mean does the "tab" containing the clarinetMIDI is unusable? For a user point of view: does the Chrome application behave so differently compared to the normal 66.0.3347 version?

Glad you asked. :)

> what do you mean by "drop out" versus "glitches"?

With drop out, you get "short burst of silence" in the stream. Glitch means the audio stream does not have silent portion but you get distortion or noise. (primarily due to the sustained samples). My usage on these terms might be different from yours, but that's how I draw the line.

Drop out means the rendering thread could not fill the buffer by the deadline, but the device thread was able to play the data. I believe glitch means the device thread could not deliver the audio data to the OS or the device driver's buffer by the deadline. This might be the limitation of Chromium thread mechanism.

> RT thread in Chromium: how is this thread configured ? (at lest on OS X) Any code to look at?

The ENUMS for thread priority:
https://cs.chromium.org/chromium/src/base/threading/platform_thread.h?type=cs&sq=package:chromium&l=108

The place where AudioWorklet thread is instantiated:
https://cs.chromium.org/chromium/src/content/child/blink_platform_impl.cc?sq=package:chromium&l=384

I replaced DISPLAY with REALTIME_AUDIO.

I hope you can build Chromium locally and try things out. :)

> how does it feels with a heavily loaded AudioWorklet thread in the local RT thread patched version ? I mean does the "tab" containing the clarinetMIDI is unusable?

When CPU goes up to 100%, the tab will be unresponsive because it uses the main browser thread which is "DISPLAY" priority. Does this answer you question?

So I understand they are 2 Audio threads, so on OS X:

- the  device thread  (I guess the real CoreAudio thread? so with RT setting given by CoreAudio right ?)

- the rendering thread with is a Chrome created thread with DISPLAY value (normally) that you tried to raise to REALTIME_AUDIO ?

- then both threads have to run properly interleaved so that audio drop out/glitches are minimized

> the rendering thread with is a Chrome created thread with DISPLAY value (normally) that you tried to raise to REALTIME_AUDIO ?

https://cs.chromium.org/chromium/src/media/audio/audio_device_thread.cc?sq=package:chromium&dr=CSs&l=54

> then both threads have to run properly interleaved so that audio drop out/glitches are minimized

Yes. I think that explains no drop out.

Here's POC CL: https://chromium-review.googlesource.com/c/chromium/src/+/919873

Some automated test cases would be great to justify this change.

"Here's POC CL: https://chromium-review.googlesource.com/c/chromium/src/+/919873" => Have you tried this change on your local machine ? Does is help to minimize audio drop or glitches?

The point is that is this period/computation/constraints values where taken by CoreAudio developers, then we could assume they where carefully chosen. Note also that on OS X several audio applications can possibly run at the same time with different buffer-sizes, so different sets of period/computation/constraints values, so we can expect those values to optimize correct RT audio thread interleaving.

For #51, I opened the new issue but it is not blocking this one:
https://bugs.chromium.org/p/chromium/issues/detail?id=813825

I am closing this one - feel free to open a new bug if needed.