New issue
Advanced search Search tips

Issue 736561 link

Starred by 3 users

Issue metadata

Status: Verified
Owner:
Closed: Aug 2017
Components:
EstimatedDays: ----
NextAction: 2017-08-17
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

OfflineAudioContext creation is blocking the main thread

Reported by chrisgut...@gmail.com, Jun 24 2017

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.109 Safari/537.36

Steps to reproduce the problem:
1. Run `now = performance.now(); new OfflineAudioContext(2, A_VERY_LARGE_NUMBER, 44100); console.log(performance.now() - now); // Should be close to nothing`

What is the expected behavior?
The little snippet should not take a significant amount of time to run.

What went wrong?
When replacing A_VERY_LARGE_NUMBER with an actual number. The snippet takes longer to run if the number gets larger. This can take up to several hundred milliseconds.

Did this work before? N/A 

Does this work in other browsers? Yes

Chrome version: 59.0.3071.109  Channel: stable
OS Version: OS X 10.12.5
Flash Version: 

In Firefox for example creating an OfflineAudioContext does not take any measurable time no matter how large the length of the OfflineAudioContext gets.
 

Comment 1 Deleted

Comment 2 Deleted

Comment 3 by rtoy@chromium.org, Jun 26 2017

How large is A_VERY_LARGE_NUMBER?  Can you give an example?

Comment 4 by rtoy@chromium.org, Jun 26 2017

I tried with a size of 100000000.  This takes 289 ms on my Linux machine.  The only thing I can think of is that creating the context also creates a 2-channel AudioBuffer of length 100000000 frames.  Perhaps allocating this memory is slow?
Thanks, for having a look at it. I noticed this when I was creating OfflineAudioContexts with a length of about 5 minutes at 44100 Hz. It took also about 200ms on my MacBook and caused the application to freeze noticeably.

I filed this bug, because I thought an OfflineAudioContext should not allocate an AudioBuffer until startRendering() gets called and therefore should be created immediately.

Comment 6 by rtoy@chromium.org, Jun 26 2017

Status: Available (was: Unconfirmed)
We allocate the buffer at creation, but it could also be done at startRendering() which also makes a lot of sense.  But this would also delay startRendering() as well.

Comment 7 by rtoy@chromium.org, Jun 26 2017

A profile of chromium says memset is taking 98.6% of the time clearing out the array that is being created for the AudioBuffer.
Ah, okay. So there is no way to have an OfflineAudioContext of 5 minutes without blocking the main thread on audio data allocation. It either happens when creating the OfflineAudioContext or when calling startRendering(), right?

I naively assumed there might be a way to do the buffer allocation with something like requestIdleCallback() and maybe in chunks if necessary after startRendering() was called to not interrupt the main thread.

This would of course not be a problem, if the OfflineAudioContext could be used inside a WebWorker.

Comment 9 by rtoy@chromium.org, Jun 26 2017

For the case of an offline context, there's really absolutely no need to initialize the buffer to anything. I'll have to poke around to see if there's a way to do that.

We could do the allocation in the separate thread that is used for the offline context.  That would fix the issue you're seeing.  Have to look to see how that would work.

It has been proposed to enable WebAudio in a worker.  That would require a change in the spec, of course, so that's not going to happen for a while.
Many thanks in advance for trying to find a solution. Allocating the buffer in a separate thread sounds very good to me, but I have of course no idea how complicated and difficult it would be to implement that.

Comment 11 by rtoy@chromium.org, Jul 14 2017

A quick test says new Float32Array(100000000) blocks for about 100 ms too.

I have some prototype code that allocates space with startRendering().  This makes creation of the offline context quite fast now.  However, this means startRendering() will now block the main thread for 1-200 ms too.

The ArrayBuffer can't be allocated in the rendering thread; it has to be done in the main thread.  I thought of just using some internal arrays that aren't initialized to zero at allocation.  This would make startRendering() fast by allocating the memory in the other thread without initializing it to 0.

However, this would double memory usage because rendered data would have to be copied to an ArrayBuffer eventually.  That's not a great approach either.

I don't know why Firefox is so fast, but based on the description in https://webaudio.github.io/web-audio-api/#widl-OfflineAudioContext-startRendering-Promise-AudioBuffer, it's possible that Firefox is doing the allocation in startRendering and not construction.
First of all many many thanks for evaluating possible solutions.

I know that calling "new Float32Array(100000000)" blocks the main thread for some time. But it is possible to create a typed array in a worker and transfer it afterwards to the main thread without blocking the main thread at all. Couldn't the Web Audio API use the same technique to initialize the AudioBuffer for an OfflineAudioContext? That would not speed up the process but it would allow the main thread to be responsive in mean time.

Since I'm not aware of the implementation details of the Web Audio API in Chrome that might of course be a stupid idea.

Comment 13 by rtoy@chromium.org, Jul 18 2017

The rendering thread being used by the offline context doesn't have any Javascript instance backing it so we can't actually create the AudioBuffer there today.

And it seems really heavy-weight to create a full worker instance just to create an AudioBuffer for the offline context.

The best we can currently do is create the buffer when startRendering is called. The spec seems to imply that that is when the buffer should be created.

The other option would be to figure out if it's possible to create an AudioBuffer without initializing the the memory to zero.  Not sure if that's possible either.
Not initializing the memory of an AudioBuffer used by an OfflineAudioContext sounds great, although I know that it generally can cause security problems. At least to my knowledge the AudioBuffer is only accessible after the rendering promise resolved and therefore will have each byte initialized or overwritten by the rendering process anyway. Does that also hold true for an OfflineAudioContext without anything connected to its destination?

Comment 15 by rtoy@chromium.org, Jul 24 2017

Yeah, the resulting AudioBuffer is only available after rendering is complete. I'd have to check to make sure the AudioBuffer is filled with zeroes even if nothing is connected.  I think it is though.

Comment 16 by rtoy@chromium.org, Jul 31 2017

Owner: rtoy@chromium.org
Status: Started (was: Available)
Project Member

Comment 17 by bugdroid1@chromium.org, Aug 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/43f4d76865609abf431fc26a64b2ba776daf2bbb

commit 43f4d76865609abf431fc26a64b2ba776daf2bbb
Author: Raymond Toy <rtoy@chromium.org>
Date: Tue Aug 01 16:22:11 2017

Lazily allocate AudioBuffer at startRendering()

When constructing an OfflineAudioContext, defer construction of the
AudioBuffer for the result until startRendering is called.  The
construction of the AudioBuffer can block the main thread for
significant amounts of time because the AudioBuffer needs to be zeroed
out.

startRendering() will now block the main thread to allocate the
AudioBuffer, but this is a better place for that and a first step in
speeding up AudioBuffer allocation.

Bug:  736561 
Test: none
Change-Id: I72beedac6b8a0445f51f0827a4f46e36c557115c
Reviewed-on: https://chromium-review.googlesource.com/572043
Commit-Queue: Raymond Toy <rtoy@chromium.org>
Reviewed-by: Hongchan Choi <hongchan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#491006}
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/LayoutTests/webaudio/dom-exceptions-expected.txt
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/LayoutTests/webaudio/dom-exceptions.html
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/Source/modules/webaudio/OfflineAudioContext.cpp
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/Source/modules/webaudio/OfflineAudioContext.h
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/Source/modules/webaudio/OfflineAudioDestinationNode.cpp
[modify] https://crrev.com/43f4d76865609abf431fc26a64b2ba776daf2bbb/third_party/WebKit/Source/modules/webaudio/OfflineAudioDestinationNode.h

Project Member

Comment 18 by bugdroid1@chromium.org, Aug 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/28b810692b54bb562b51b36de2c4701a49954498

commit 28b810692b54bb562b51b36de2c4701a49954498
Author: Raymond Toy <rtoy@chromium.org>
Date: Mon Aug 07 18:08:28 2017

Use uninitialized AudioBuffer for startRendering()

startRendering() will fill up the resulting AudioBuffer with the
rendered data so it's a waste to zero-initialize the buffer when it's
going to be filled up anyway.  Therefore, allow creation of an
AudioBuffer that is not zero-initialized.

Also, when creating and AudioBuffer from an AudioBus, there's no need
to initialize the AudioBuffer either because we're going to immediately
copy the bus contents to the buffer.

Bug:  736561 
Test: none
Change-Id: Ibfe1c0d7da0fba62901445a5530026e6ff297b28
Reviewed-on: https://chromium-review.googlesource.com/586858
Commit-Queue: Raymond Toy <rtoy@chromium.org>
Reviewed-by: Kentaro Hara <haraken@chromium.org>
Reviewed-by: Hongchan Choi <hongchan@chromium.org>
Reviewed-by: Kent Tamura <tkent@chromium.org>
Cr-Commit-Position: refs/heads/master@{#492361}
[modify] https://crrev.com/28b810692b54bb562b51b36de2c4701a49954498/third_party/WebKit/Source/modules/webaudio/AudioBuffer.cpp
[modify] https://crrev.com/28b810692b54bb562b51b36de2c4701a49954498/third_party/WebKit/Source/modules/webaudio/AudioBuffer.h
[modify] https://crrev.com/28b810692b54bb562b51b36de2c4701a49954498/third_party/WebKit/Source/modules/webaudio/OfflineAudioContext.cpp
[modify] https://crrev.com/28b810692b54bb562b51b36de2c4701a49954498/third_party/WebKit/Source/platform/wtf/typed_arrays/Float32Array.h
[modify] https://crrev.com/28b810692b54bb562b51b36de2c4701a49954498/third_party/WebKit/Source/platform/wtf/typed_arrays/TypedArrayBase.h

Comment 19 by rtoy@chromium.org, Aug 8 2017

chrisguttandin@ Please test out the latest canary.  With these two CLs, we now create the required AudioBuffer when startRendering is called, so creation of the offline context should be fast now.  startRendering is a bit slower because we need to create the large buffer to hold the result, but at least we don't waste hundreds of milliseconds zero-filling an array that is going to be replaced with the actual result.

Comment 20 by rtoy@chromium.org, Aug 10 2017

NextAction: 2017-08-17
rtoy@ Many thanks, I can confirm that the creation of the OfflineAudioContext is super fast now.

Comment 22 by rtoy@chromium.org, Aug 14 2017

Status: Verified (was: Started)
Thanks for testing!
The NextAction date has arrived: 2017-08-17

Sign in to add a comment