Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 23 users
Status: Duplicate
Merged: issue 577431
Owner:
Closed: Jan 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment
Web Audio API stops firing callbacks unexpectedly on Linux
Reported by jussi.ka...@gmail.com, May 16 2011 Back to list
Chrome Version       : 11.0.696.65
OS Version: 
URLs (if applicable) :
Other browsers tested:
  Add OK or FAIL after other browsers where you have tested this issue:
     Safari 5:
  Firefox 4.x:
     IE 7/8/9:

What steps will reproduce the problem?
1. Create a JS callback to generate sound using the Web Audio API

What is the expected result?
You should hear a sine(ish) sound, the callback should keep getting fired.


What happens instead?
The callback gets fired 1-11 times or so, and then the audio output starts looping the last buffer as if a broken record, and the callback is no more fired. (See console logs)


Please provide any additional information below. Attach a screenshot if
possible.

I've just managed to isolate a test case, which actually fails every time you try to open it:

http://niiden.com/bugz/chrome-waa-lx-bug.html

In case there's though of it being driver related, here's the output for $ aplay -l

**** List of PLAYBACK Hardware Devices ****
card 0: SB [HDA ATI SB], device 0: VT1708S Analog [VT1708S Analog]
  Subdevices: 1/2
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
card 0: SB [HDA ATI SB], device 1: VT1708S Digital [VT1708S Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 3: ATI HDMI [ATI HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Of which the first one being the primary one.

Here's the stdout from the browser, this is actually kinda revealing:

[3:4:2495534277:ERROR:audio_device.cc(179)] Not implemented reached in virtual void AudioDevice::OnStateChanged(AudioStreamState)
[2346:2392:2495543679:ERROR:alsa_output.cc(744)] Failed querying delay: Input/output error

UserAgentString: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/11.0.696.65 Chrome/11.0.696.65 Safari/534.24



 
Comment 1 by crogers@google.com, May 16 2011
Owner: crogers@google.com
Status: Assigned
Comment 2 by crogers@google.com, May 17 2011
Helpful info reported by Jussi:


I went through the Firefox source code to see how they tackled the problem, and I found these:
http://mxr.mozilla.org/mozilla-central/source/media/libsydneyaudio/src/sydney_audio_alsa.c#322
https://bugzilla.mozilla.org/show_bug.cgi?id=591719

It seems that what they did was to store the last valid position reported by ALSA and measure the delay against that.

Well, another surprising turn of events: this problem is also reproducible on my laptop (Lenovo Thinkpad R500) Windows 7 Managed Enterprise Edition, on Chrome Canary 11.0.696.68. This is unexpected.
And it also seems to be that if you defer the creation of the AudioContext to be significantly after window.onload, the problem doesn't appear. Seems like underruns are not handled as well as they could be on either OS.
Comment 5 by crogers@google.com, May 19 2011
Interesting, let me have a look on my Windows machine.  It would be great if the cause (and fix) are the same.
Comment 6 by crogers@google.com, Jul 14 2011
Jussi, I've looked at this a bit, and have a work-around for you while I investigate how to fix the actual bug.  Please see attached file "test-jussi.html".  The work-around is either to keep a reference to the function returned by SineSweep() (which is getting garbage collected early apparently).  Or, you can keep a reference to "node" which is the JavaScriptAudioNode.  In the attached file, I've created a global called "ss" to reference the function, but keeping a reference to it anywhere should be OK.

I'm not sure yet all of the details of why this is happening, but I'm looking deep down in the V8 generated bindings and the EventListener / EventTarget code.  Something about .onaudioprocess is not properly keeping a reference to the function returned by SineSweep().

By the way, please also note that I've removed the unnecessary call to createBufferSource(), and its connection, since it's OK to have an unconnected (no source) JavaScriptAudioNode

test-jussi.html
1.2 KB View Download
Yes, I actually started thinking it might be a garbage collection bug as well. But that's quite weird, I thought it was in the spec that you aren't allowed to GC event listeners attached to the DOM tree, so I'm hoping the bug doesn't go too deep. Maybe the bindings to DOM aren't ready yet on window.load, allowing the listener to be GCed?
Cc: dominicc@chromium.org
I think it is more likely that your page doesn’t allocate much memory after it has started, so the GC isn’t triggered. (I forget the logic for triggering V8 GC on idle.)

In terms of being in the DOM tree, JavaScriptAudioNodes are not nodes, right? So although they are DOM objects, they’re not in the DOM tree.

There is a concept in WebKit for keeping alive objects that have visible side effects even if they are not reachable, like XHR and Image. See ActiveDOMObject. Perhaps something in the audio graph should be an ActiveDOMObject.

FWIW this looks like a WebKit bug. So it will ultimately be more productive to file a WebKit bug and carry on discussion on that bug.
I think it is more likely that your page doesn’t allocate much memory after it has started, so the GC isn’t triggered. (I forget the logic for triggering V8 GC on idle.)

In terms of being in the DOM tree, JavaScriptAudioNodes are not nodes, right? So although they are DOM objects, they’re not in the DOM tree.

There is a concept in WebKit for keeping alive objects that have visible side effects even if they are not reachable, like XHR and Image. See ActiveDOMObject. Perhaps something in the audio graph should be an ActiveDOMObject.

FWIW this looks like a WebKit bug. So it will ultimately be more productive to file a WebKit bug and carry on discussion on that bug.
Comment 10 by rtoy@google.com, May 24 2012
 Issue 129306  has been merged into this issue.
Comment 11 by colin...@gmail.com, May 24 2012
 Issue 129306  notes that this bug is reproducible on OS X.
Using Chrome 20.0.1132.47 on Ubuntu Linux. It seems JavaScriptAudioNode is not working all for me. The callback never gets called. This problem seems to have suddenly appeared.
That's probably because in the current implementation none of the arguments for createJavaScriptNode are optional, even the inputChannels argument has to be non-zero (at least for now), so please check if that fixes your problem.
I had indeed set inputChannels to 0. Setting it to 2 fixes the problem.
I'm passing 2 as both output and input to createJavaScriptNode, but it does not solve the issue for me.

Can easily reproduce with latest chrome canary on OS X with this script:
http://debian.as/~nano/chromebug/

It takes a couple of reloads before the onaudioprocess callback is called at all, but once it starts it will play for a 1-60 seconds until it's never called again and the last buffer is being repeated forever.
I took a look at this recently and I believe this bug still exists. I believe you could run Chrome with --js-flags="--expose-gc" and call gc() explicitly to make a more reliable repro.
Worth noting is that it works a lot better in Chrome Canary on Linux, as long as the following doesn't happen:

* Switch to another tab while playing
* Open the inspector while playing

When inspecting the memory usage, which works if I open the inspector and reload the page, the GC activity isn't that big, just a couple of MB's a second. Comming from a Java world to me that doesn't look very much... but I don't know.
Also, sometimes the onaudioprocess isn't called at all, without any errors in the log.
I can confirm I have seen this happening since at least May 2012 (I don't have data from before), both using Chrome on Mac OS and Chromium on Linux.

Most of the times I had to close the tab and reopen it, as reloading didn't help--the callback function wasn't called, or was just called for a very short amount of time, or just kept playing the latest buffer data in a sort of glitchy way.

I recently tried an approach where I delay the connection of the jsAudioNode to the context destination a bit after window.onload, and I can confirm that the audio output has been solid as a rock since then. I can switch to other tabs, come back to the one with audio output and it keeps working, or I can load it with the confidence that it won't fail after a while. I don't know why, but this seems to be working for me. The pseudocode is something as simple as this:

window.onload = function() {
// get audio context, do all node initialisation
// but do not connect to the destination yet

// set timeout to connect 1 second later:
setTimeout(function() {
jsAudioNode.connect( audioContext.destination );
}, 1000);
}


Tried suggestion from #19 to the code in #15 without luck. Not exactly applicable, but in short, delayed setting up the WebAudio stuff until 1 second after window.onload. Problem still easily reproducible.

supers...@gmail.com: Could you see if my example in #15 works for you? (it should play for at least 300 seconds if it works, possibly forever, don't recall the details)
dsvens...: It gets stuck in the last buffer data after a while, probably after less than one minute.

It would help if you displayed the played time as well :-)

Tried it on Mac OS, Chrome 23.0.1271.91 ("it's up to date")
Yep, that's what I get as well. Still broken. The callback executes, I produce the data, the callback exits, and then it doesn't get called again and instead just repeat the last rendered chunk forever. This is what I expect this bug report describes.
I believe as a workaround you can store a reference to the JavaScriptAudioNode and AudioContext on the window object, for example:

// code which sets up AudioContext and JavaScriptAudioNode
window.savedReferences = [audioContext, javaScriptAudioNode];
dominicc: Excellent, that works fine! Rock solid playback now. I must have skipped reading the comments when I found the issue back in september :( .. could have gotten this to work months ago... and yeah, now in hindsight, a GC collection sounds like a very plausible cause.
Just noting that this bug also happens in Safari 6.0.2. So the bug is in the common WebKit implementation and is not specific to Chromium - i.e. the *same* GC issue (if it is one), occurs in both V8 and Nitro. 
Project Member Comment 26 by bugdroid1@chromium.org, Mar 10 2013
Labels: -Area-Undefined
I reported this in webkit and added a patch that fixes the issue.

https://bugs.webkit.org/show_bug.cgi?id=112521
Comment 28 by e...@chromium.org, Jul 14 2013
Is there something we can do about this now that we control the memory manager details in blink?
I don't think there are technical issues blocking fixing this. There are analogous situations (<button>:event listener :: audio context : JavaScript node) and solutions:

Weak reference from C++ thing that needs to fire the callback to JavaScript--see ScopedPersistent::makeWeak) and strong reference from some object in the JavaScript heap that already has the right lifetime (probably the audio context wrapper) to the callback. (Use set SetHiddenValue for that purpose.)

This case slightly complicated because:

1. The audio context is an active object. But if it is a properly implemented ActiveDOMObject the above should just work because the wrapper system keeps the wrappers of ActiveDOMObjects alive when the objects are active.

2. I guess you can have multiple JavaScript nodes per context? So you'll need to store multiple references on the JavaScript side. I guess you could store them in an array.

This just needs engineer time. I'm not a bindings/v8 reviewer but I'm happy to do informal reviews.

FWIW I just tried the repro on CrOS and although the audio playback became choppy it did not halt. But maybe Linux != CrOS. I think with --js-flags=--expose_gc and a well timed gc() call you should be able to make something that reproduces on all platforms.
Project Member Comment 30 by bugdroid1@chromium.org, Jul 31 2013
The following revision refers to this bug:
    http://src.chromium.org/viewvc/blink?view=rev&rev=155194

------------------------------------------------------------------------
r155194 | dominicc@chromium.org | 2013-07-31T00:46:55.726751Z

Changed paths:
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/webaudio/scriptprocessornode-premature-death-expected.txt?r1=155194&r2=155193&pathrev=155194
   A http://src.chromium.org/viewvc/blink/trunk/LayoutTests/webaudio/scriptprocessornode-premature-death.html?r1=155194&r2=155193&pathrev=155194

Demonstrate that Web Audio does not keep callbacks alive long enough.

This is discussed on  Bug 82975 . Unfortunately, not keeping the
callback alive long enough encourages authors to implement a
workaround which is prone to leaks by retaining (and typically
leaking) a reference to the callback themselves.

BUG= 82795 
R=crogers@google.com

Review URL: https://codereview.chromium.org/21216002
------------------------------------------------------------------------
Comment 31 by rtoy@google.com, Sep 20 2013
Cc: ranjitkan@chromium.org rtoy@chromium.org cwilso@chromium.org
 Issue 266303  has been merged into this issue.
Comment 32 by rtoy@chromium.org, Sep 20 2013
Cc: -rtoy@chromium.org
Owner: rtoy@chromium.org
Labels: WebKit-ID-112521
Project Member Comment 34 by bugdroid1@chromium.org, Sep 23 2013
Labels: -WebKit-ID-112521 WebKit-ID-112521-NEW WebKit-Rev-146033
https://bugs.webkit.org/show_bug.cgi?id=112521
http://trac.webkit.org/changeset/146033

Cc: kouhei@chromium.org
Kouhei, this might interest you. It's not a leak but a premature free (which doesn't result in a crash. It just stops generating audio data.) However the craft solution web developers employ to work around it is to reference the audio context and hence leak it, so we should fix this one. (It's old, but I don't think it's fixed.)
Comment 36 by lj1...@gmail.com, Jan 27 2015
I just experienced this issue on:

Chrome Version 39.0.2171.99 (64-bit)
OS: Ubuntu 14.04 LTS

I think the expedient solution here is for AudioContext to get Custom=VisitDOMWrapper and have it create an object group for at least its ScriptProcessorNodes. It's not immediately clear which ones those are... just a subset of AudioContext::m_liveNodes?

Alternatively the object grouping mechanism for Nodes could be generalized to have AudioContexts be the group leader. That's appealing because it handles event handlers, which ScriptProcessorNode has.

Is the Web Audio graph navigable? It looks like from a given AudioNode you can get to the context, but from there only to its AudioDestinationNode and not some arbitrary node?
Comment 38 by rtoy@chromium.org, Jan 29 2015
Labels: Cr-Blink-WebAudio
I would assume anything that has an Event like ScriptProcessorNodes, OfflineAudioContexts, AudioContexts, and so on.

I think the only way to reach all nodes is via m_liveNodes. From the destination, you can walk backwards and find everything connected to the destination, but some nodes may not (yet) be connected.  Similarly, from a given AudioNode, you can walk forwards (and backwards) to find other nodes, but not all nodes can be reachable this way.  There can be subgraphs disconnected from any other part of the audio node graph.

Blocking: chromium:577431
Owner: hongchan@chromium.org
Status: Started
This happens because AudioNode is not ActiveDOMObject. The fix is coming.
Mergedinto: 577431
Status: Duplicate
Blocking: -chromium:577431
Project Member Comment 43 by bugdroid1@chromium.org, Mar 7 2016
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/246a91f12971f2f6e4ba0b68fdbf5efa8afb344e

commit 246a91f12971f2f6e4ba0b68fdbf5efa8afb344e
Author: hongchan <hongchan@chromium.org>
Date: Mon Mar 07 18:02:15 2016

Add ScriptWrappable::hasPendingActivity to ScriptProcessorNode

|onaudioprocess| event in ScriptProcessor node is not fired once the JS
reference of the node goes out of local scope.

This CL is to keep the 'out-of-scope' reference from being collected when
it has an active |onaudioprocess| handler.

With this patch, scriptprocessornode-premature-death.html is now correctly passing. Thus the expected text was fixed accordingly.

BUG= 577431 , 500335 , 360378 , 121654 , 82795 
TEST=LayoutTests/webaudio/scriptprocessornode-premature-death.html (revised expected file)

Review URL: https://codereview.chromium.org/1762103002

Cr-Commit-Position: refs/heads/master@{#379586}

[modify] https://crrev.com/246a91f12971f2f6e4ba0b68fdbf5efa8afb344e/third_party/WebKit/LayoutTests/webaudio/scriptprocessornode-premature-death-expected.txt
[modify] https://crrev.com/246a91f12971f2f6e4ba0b68fdbf5efa8afb344e/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.cpp
[modify] https://crrev.com/246a91f12971f2f6e4ba0b68fdbf5efa8afb344e/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.h
[modify] https://crrev.com/246a91f12971f2f6e4ba0b68fdbf5efa8afb344e/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.idl

Sign in to add a comment