New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 795666 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[audioworklet] Adjustable buffer size for AudioWorkletNode/Processor

Reported by andre.mi...@gmail.com, Dec 18 2017

Issue description

We included AudioWorklet for a test in https://next.audiotool.com (account required). The playback was unfortunately very glitchy, even with simpler setups. We also noticed that the fps was going down from 60 to 40 frames-per-second.

My questions:

Is it possible that the processor-script is talking too often for any reason to its main-thread counterpart (AudioWorkletNode)? That would explain the FPS dropping. We made sure in our testing that no additional messages were exchanged after all necessary resources were loaded into the worklet.
Is it possible to adjust the buffer-size yet from 128samples to something more reliable? I tried to test the latency-hint, but I could not make it work. Will it actually change the buffer-size or just hold more buffers to be filled?
Same question as #2 goes for the sample-rate.
new AudioContext({sampleRate:44100}).sampleRate // will always output 48000 on my machine. Lowering the sample-rate to 44100 would save performance on bigger graphs even it needs to be resampled in the end.
All tests done on Version 65.0.3295.0 (Official Build) canary (64-bit)

Thanks in advance.
 

Comment 1 by rtoy@chromium.org, Dec 18 2017

A few comments.

The latencyHint value doesn't currently change the actual size in some cases.  There's CL up to fix that and it should be landing soon.

Chrome has not yet implemented the sampleRate option for the AudioContext.

Not exactly sure what you mean by the "buffer-size".  WebAudio is required to render the graph in chunks of 128 samples.  If that is not suitable for the work  you do in an AudioWorklet, you'll have to do your own buffering (and possibly introduce latency).
Labels: Needs-Triage-M65

Comment 3 by rtoy@chromium.org, Dec 18 2017

Labels: Needs-Feedback
I created an account, but don't know what to do to show the problem.  Please provide more explicit details on what to do to reproduce the issue.
"I created an account, but don't know what to do to show the problem."
I am sorry if I confused you by thinking that the AW test is actually online available. It was an internal test and the outcome was not worth uploading.

"Not exactly sure what you mean by the "buffer-size"
The number of frames in a chunk. Is that not the usual wording? Block-size, buffer-size?
Project Member

Comment 5 by sheriffbot@chromium.org, Dec 19 2017

Cc: rtoy@chromium.org
Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "rtoy@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Cc: sc00335...@techmahindra.com
Labels: Triaged-ET
rtoy@ Request you to please look into the issue and provide an update.

Thanks...
Labels: -Triaged-ET -Needs-Triage-M65
Owner: hongchan@chromium.org
> It was an internal test and the outcome was not worth uploading.

Still, if you can provide any specifics on the test I would appreciate!

> The playback was unfortunately very glitchy, even with simpler setups. We also noticed that the fps was going down from 60 to 40 frames-per-second.

Without any detail on the setup, this does not mean anything to us.

> Is it possible that the processor-script is talking too often for any reason to its main-thread counterpart (AudioWorkletNode)? That would explain the FPS dropping.

That's not how AudioWorklet works. It does not do any message exchange between two components unless you call "postMessage" on them. The project landing page has some resources you might find useful: https://googlechromelabs.github.io/web-audio-samples/audio-worklet/

> We made sure in our testing that no additional messages were exchanged after all necessary resources were loaded into the worklet.

We don't know what's loaded and what kind of messages are being sent, so I can't really be of help here.

> Is it possible to adjust the buffer-size yet from 128samples to something more reliable?

No. That violates the fundamental of WebAudio. Alternatively you can implement FIFO inside of AudioWorklet so you can buffer the data for yourself.

> I tried to test the latency-hint, but I could not make it work. Will it actually change the buffer-size or just hold more buffers to be filled?

WebAudio's buffer size (we call it Render Quantum) is always 128 samples. With this latency hint, it changes the platform-dependent batch render size. So technically yes, it should work like DAW's audio engine buffer size.

> Same question as #2 goes for the sample-rate. new AudioContext({sampleRate:44100}).sampleRate // will always output 48000 on my machine. Lowering the sample-rate to 44100 would save performance on bigger graphs even it needs to be resampled in the end.

The variable sample rate on AudioContext has not been implemented yet.
Labels: Needs-Feedback
rtoy: "you'll have to do your own buffering (and possibly introduce latency)."
hongchan: "Alternatively you can implement FIFO inside of AudioWorklet so you can buffer the data for yourself."

me:
Introducing latency by increasing the number of filling blocks (FIFO) in the rendering queue is used to stabilise playback. The current web-audio-api assumes that the rendering time is almost constant. However that is rarely the case in the real world.

A proper FIFO can only be implemented with at least two threads. One or more rendering threads and one playback-thread. The problem is that AudioWorkletProcessor is single-threaded and will ask for new audio data in the process method but might just be busy rendering audio for a later to play audio-block. So even if you have audio-data prepared there is no way to ship it.

Hopefully this ascii can make it clear:
-----|-----|-----|==========|
                    ^ process
The first three blocks in our custom FIFO are already rendered and could immediately be sent to the output in the process method. However when the process method is called, it cannot provide the prepared blocks because it is busy preparing the block that currently takes slightly longer to render than the previous ones. So we probably get a playback-glitch even though enough audio-data has been buffered.

That is why I think we should have an option to increase the number of FIFO buffers that are used internally in the web-audio-api.
Project Member

Comment 10 by sheriffbot@chromium.org, Jan 28 2018

Cc: hongchan@chromium.org
Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "hongchan@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Summary: [audioworklet] Adjustable buffer size for AudioWorkletNode/Processor (was: [audioworklet] Performance)
andre.michelle@

FIFO with AudioWorklet has been discussed in many occasions and the "proper" implementation is based on SharedArrayBuffer + WebWorker. Perhaps I missed the detail when I wrote the comment above.

Also a proper FIFO does not necessarily require the multi-thread situation. (Well, what's the definition of a "proper" FIFO anyway?) Oversampling can be an example for such operation.

The user-implemented buffering or FIFO is just a speculative solution we laid out here because we do not have any actionable information here. If you can provide a reproducible test case, we can certainly be more helpful.

More importantly:

The adjustable buffer size requires a spec change. That's not something we should discuss here. If you like to propose your idea, please take it up to AudioWG's discussion: https://github.com/WebAudio/web-audio-api/issues

FYI, AudioWorkletNode is AudioNode. The premise of WebAudio is a fixed render quantum size for all AudioNodes. Making AWN special by violating that principle does not seem to be feasible at this point.
Labels: Needs-Feedback
The discussion in the other thread might be helpful:
https://bugs.chromium.org/p/chromium/issues/detail?id=796330
"FIFO with AudioWorklet has been discussed in many occasions and the "proper" implementation is based on SharedArrayBuffer + WebWorker. Perhaps I missed the detail when I wrote the comment above."
I was not aware that we can talk through a SharedArrayBuffer. I have implemented such and it works. We will test this on Audiotool soon.
Project Member

Comment 15 by sheriffbot@chromium.org, Jan 30 2018

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "hongchan@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: WontFix (was: Unconfirmed)
Re #14:

SharedArrayBuffer is actually disabled for the recent security problems (Spectre/Meltdown): https://www.chromium.org/Home/chromium-security/ssca

I hope this is only a temporary measure until we can solve this issue in the near future - because I believe SAB + AudioWorklet can unlock the potential of the apps like Audiotool.

Closing as wontfix.
We have implemented an option to run Audiotool with AudioWorklet.
https://www.audiotool.com/board/news/chrome_audioworklet
You can find credentials in this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=708154#c10
andre.michelle@

Thanks for sharing the progress! Also Why don't you register the site under Origin Trial? Then your users don't have to go through the flags to enable the app.

https://developers.google.com/web/updates/2017/12/audio-worklet#experimental
Sounds interesting but we need to have SharedArrayBuffer enabled which is not supported by Origin Trial. But we will check this idea sooner or later. Thank you!

Sign in to add a comment