It seems that with buffers less than 4096 bytes the "onaudioprocess" method does not get called with enough cadence, causing the audio to stutter and break.

It works with buffers >= 4096 bytes, but then the delay is unacceptable for realtime audio apps like games.

Please take a look at this. This makes my project unusable in Android devices. It works great on Chrome for iOS though.

I already tested in several devices, even the fastest tablets and phones on the market show the same problem. Its not a problem with the device being too slow.

Isn't anyone else using WebAudio to generate realtime audio on Android?
It seems impossible right now.

Even on my fairly powerful Z620, I hear glitches.  The main problem is that the selected buffer size for the ScriptProcessorNode is too small.  On linux, audio is processed as 4 batches of 128 samples, and your size is 512.  This makes any slight delays in processing really sensitive.  If I change the buffer size to 1024, I don't hear any glitches.

This is why ScriptProcessor allows a buffer size of 0, which is a hint to the browser to select an "optimum" size.  However, Chrome hasn't really done a great job of selecting an optimum and tends to be pretty conservative in selecting a fairly large size.  This is fine for sources, but if you want low latency processing, you lose.

Also, I see you're reporting this on an Android device.  This is a whole different animal.  The internal processing size can be as large as 3840 samples (or more!!!!) so you need to set the buffer size at least that large for those devices.  Unfortunately, there's no easy way to determine the reported size from Android other than writing a bit of Java code to query it.

We do plan to optimize the buffer size on Android a bit more for those devices that support Android's low latency audio path. Currently, we always set the buffer size to 1024 or more.  We know that most Nexus devices (like a Nexus 4 or Nexus 9, probably also the new 5x, 6, 6p, and Pixels?) could support a size of 256 without any trouble.

Thanks! Let me clarify a bit:

The problem is specific for Chrome on Android. I have no problems on the desktop versions, including Windows, OSX and Linux. I have successfully used buffers of 512 and even 256 bytes, together with very intensive javascript processing (tied to requestAnimationFrame).
All works beautifully.

It also works great on Chrome for iOS.

Also, it is not a problem with Android either, at least not on the devices I tested, because I was able to make the same code work nicely with buffers like 512 and 1024 in Firefox, on the same device!

But in Chrome for Android it always glitches unless the buffer is >= 8192. It seems like Chrome simply does not *accept* buffers less than that. Maybe there is something on the code, some too conservative assumption, limitation or default values that does not allow it to work correctly with smaller buffers, even if the device is capable. Anything I tested with 2048 or even 4096 bytes glitches...

But I'm sure it is possible since Firefox does it nicely on the same device, you see?

Raymond, did you get my test case to run correctly with 1024 bytes on an ANDROID device? If I could make it work with 1024 or even 2048 on Android I would be very happy!

Please, run it on Chrome for Android with 1024 bytes, then run exactly the same on Firefox on the same device, you will see (or hear) what I'm talking about!

This is a very limiting issue, and very important, because as of now this is the only way to generate realtime audio (not predefined samples) on games in Javascript, designed to run on Android. Supporting Chrome for Android is crucial!

We don't really need 256 or 512 buffers on the ScriptProcessorNode. If 1024 worked it would already be good enough.

But as of now Chrome needs 8192 bytes of buffer, according to my tests. With this kind of latency, is not really usable!

Thank you very much!

Yes, you are right.  Something is really broken.  My Nexus 9 should be able to run with a buffer size of 2048, but it doesn't; 4096 is ok. A Nexus 5x works fine with 2048.

This will take some time, unfortunately.

> This will take some time, unfortunately.

Really? That's too bad... I wish I could support Chrome on my project. For now it will only work on iOS or in Firefox on Android. But Firefox has other stability issues. Chrome runs everything else nicely, but this. 

I'm really surprised no one else is struggling with this issue!

I was looking into the code, and it really appeared to me that this problem was due to some fixed buffer value somewhere, like in here:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/audio/AudioDestination.cpp?rcl=0&l=44

I wish I could help you more, but I don't really understand the code yet. I hope it's not too hard, since other browsers don't seem to have any trouble with this.

How long you think it would take to fix?

Thank you very much

Just another important bit of information:

I have run some other tests that *do not* use the ScriptProcessorNode and there is no problems on Android. There is no noticeable delays or any high latency. Its ok!

So it seems to me that this issue is not related to the AudioContext in general, nor with hardware buffers or other underlying rendering processing on the stream.

It just happens in ScriptProcessorNodes, because they need to call javascript, and the only problem is that the "onaudioprocess" callback function is not called with the correct frequency for smaller buffers.

So we are having "buffer underflows" on the ScriptProcessorNode. The buffer was already consumed but there is no javascript callback to refill it... If we can make the callback function to get called at the right times, it should work.

Paulo

The fifo size you link to probably isn't the cause of the problems. That's just for the internal buffering between webaudio and Chrome's audio backend.

It's great Chrome is otherwise working fine for you.

I'm guessing the threading on Android is somehow different between desktop and Android.  But I don't yet know what's going on.

Yes, it's probably not that buffer.

I really think the problem is not related to underlying buffers.
It's just the callback to the "onaudioprocess" that needs to happen more frequently for smaller buffers.

Typically, for 44100 KHz audio on the hardware and a ScriptProcessorNode with 1024 bytes of buffer, we should get approx. 43 calls to "onaudioprocess" per second.

It's not too much, right? It's less than one call per "frame" in a game using requestAnimationFrame, for instance.

It does not appear to be related to complex platform-dependent threading or hardware buffers.

It sounds like some hard-coded limitation or faulty calculation on how frequently the "onaudioprocess" should get called, as opposed to requestAnimationFrame, for instance, which is called correctly.

What is used to determine when, or what actually triggers the call to "onaudioprocess"?

The handling of the ScriptProcessorNode is all done here:  https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.cpp?rcl=0&l=112 in case you want to look.

Yes, I looked at that code for quite some time. Please take look at the code that begins here:

https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.cpp?rcl=1480681004&l=188

Its where the callback to "onaudioprocess" happens.

Please keep in mind that I don't understand the code completely, but I see 2 things that may spark some ideas:

1) There is a condition when some index wraps back to 0 then the above code will fire a callback. But what index is that? Are we sure that this index refers to the correct ScriptProcessor buffer (that may have any size and can be as small as 512 or 1024 bytes), and not to some other internal or fixed-size buffer like that one which is always 8192 bytes? If that index does not refer to the correct ScriptProcessor buffer, we will get callbacks at the wrong times!

2) There is a condition that may decide NOT to make the callback if the main thread is busy, right? Maybe that is too much conservative and are completely skipping callbacks when it should instead just queue it to be processed as soon as possible? Maybe it should stop queuing callbacks only after a certain number of callbacks are still waiting... You see, imagine a scenario where the callbacks should be fired each 20ms. Then on the exact moment the above code will fire the callback it looks at the main thread and its busy so it decides not to make the callback, BUT just 0.1ms later the main Thread becomes available, but its already too late since the ENTIRE callback was skipped. And that can happen for several callbacks successively. Even if the main thread is busy only, lets say, 5% of the time and is completely capable of providing the needed buffers we still would skip statistically 5% of the callbacks!

Of course, I don't fully understand the locking mechanics and how quickly the main thread is able to respond to queued requests.

Hope that helps!

Raymond, did you get a chance to look at the 2 points I mentioned above?
Any insights?

I found another important bit of information.

I run a test logging the times when "onaudioprocess" is called.

With a buffer size of 1024 and 44100kHz sampling, the callback should get called approx. each 23ms, uniformly.

But, my test showed a very odd and NOT uniform pattern.

The callback method gets called about 5 or 6 times in a row almost on the same millisecond, or with very small intervals of 1ms or 2ms between calls, then we have a waiting period like 120ms with no calls, and then the overall pattern repeats almost the same way.

With larger buffers the pattern is similar, only with larger times and LESS calls in sequence, followed by proportionally decreasing waits between call sequences.

So with larger buffers the problem is less noticeable.

Please notice that in the 1024 bytes buffer test, the "onaudioprocess" was able to run and complete in less than 1ms on the calls executed in sequence, so it appears the problem is not caused by a slow callback method that cannot cope with the needed cadence.

Its the calls to the callback method that are not on the correct cadence.

Hope that helps.

Thanks for doing these tests!  What phone are you using for the testing?

I am using the device described in this issue:

Device name: Samsung Galaxy S6
Chrome version: 53.0.2785.124
OS: Android 5.1.1; SM-G920I Build/LMY47X

Looking at the code you pointed above, I think I have found a bug!

There is a Mutex "m_processEventLock" being used to synchronize process() with fireProcessEvent()

The code tries to use this Mutex to detect if the main javascript thread still has a pending "onaudioprocess" callback that has not yet processed/finished. If it detects so, it does not fire another one. It skips this one and produces silence instead. This makes sense, so it does not queue up several callbacks on the main thread while there is still one waiting to be processed.

BUT, the way it uses the Mutex does not detect if a pending callback is yet to be processed/finished!!! It detects if a callback is BEING PROCESSED at that exact moment, right?

That leaves the code free to build up requests on the main thread as much as it tries to, unless there is one being processed at the exact moment. That defeats the purpose of the scheme!

I don't know if this bug alone is causing the issue reported, but its definitely a synchronization problem. The code clearly does not do what it should, nor what the comments say it does.

That may explain my test results, that show several callbacks piled in sequence with no interval.

I found this just by reading the code, so I'm not 100% sure.
I still have to set up my machine so I can get the code and try to build it, which is a little complicated to me as I have little experience with C++.

One question: Is this the same code running on Windows, Android and iOS?
Why I have this problem only in Chrome on Android?

Raymond, I am investing quite some time trying to figure this bug out, since my project sounds crappy on Android right now.

I will continue to study the code and will try to modify and test it.
If someone could have a look at this, it would be very nice. If you like me to try or test anything, please let me know.

Thanks

I really appreciate our investigations in this area. I will take a closer look at the beginning of next week at this problem.

The code is the same for all platforms, but Mutex (and other library functions) could be different on Android from desktop.

Building Chrome in a reasonable amount of time requires a pretty powerful machine (https://chromium.googlesource.com/chromium/src/+/master/docs/linux_build_instructions.md#System-requirements), so I wouldn't recommend it unless you have such a machine.

Yes, I'm trying to get such a machine so I can build and test Chrome on Android. But that may take a couple of days, if I'm ever able to get it.

In the meantime, I have some conclusions/observations about the code I was reading and how we could improve it. If you want I can try to explain my findings even before testing.

Yes, if you have ideas, I'd love to hear it.  While ScriptProcessorNode is deprecated and while we prefer not to do anything with it, we do want to make it work reasonably.  It's not going away any time soon.

Is there any way of achieving what ScriptProcessorNode does by any other means TODAY? I would love to make my project more future proof, but I did not find any other realistic path. AudioWorker/Worklet is not yet really supported, is it?
 
So, today, the best I can do to make my project work is to drive all my efforts to fix this only issue in Chrome.

Depends on what you're trying to do in the ScriptProcessorNode.  In your current example, you could just create a long buffer of random samples and play that through an AudioBufferSourceNode.  (There was a request for a NoiseGeneratorNode to generate noise, but that got pushed off to the next version of the spec.)

But in general, if you need a ScriptProcessor, you need it and there is no alternative today.

Yes, I don't really generate noise on my real project. :-) That was just
the test case do show the bug.
I really need to generate real time samples, with low latency, for a
game-like scenario.
That's why I think I need a ScriptProcessorNode, with a reasonably low
buffer. With a big buffer it is unusable for my scenario, since it becomes
not "real-time" anymore.

My project is working VERY well on desktops and on modern iOS devices, also
in Firefox for Android, which has other problems, but the audio works as
expected.

I am sure that its possible to get one "onaudioprocess" callback each 23ms
or so, that is required for a 1024 bytes buffer.
If we can get the sync to work correctly, and if we can post a task to be
executed on the main javascript thread with a reasonable waiting time (like
the one from requestAnimationFrame, for instance), it will work nicely,
just like it does on iOS.

Tell me, are we using the same code also for iOS?

Yeah, I was expecting you were using the node for more complicated scenarios that can't be done any other way.

Chrome on iOS uses Webkit, the same engine as used by Safari.  But the webaudio code in Webkit hasn't really changed much over the years. (Well, I haven't looked lately, but Safari still only supports webkitAudioContext and not AudioContext.)

Its very surprising to me that not many people are complaining about this issue.
Isn't anyone else using the ScriptProcessorNode???
It's a great concept, and the only solution for games that generate realtime samples in JS.

Regarding my analysis of the issue itself, I found another possibility:

The AudioNode has a method "process(framesToProcess)" that is responsible for providing the samples for each rendering quantum right?

1) Does this method get called with the right cadence (I mean evenly distributed in time as the AudioContext rendering takes place quantum by quantum)
or
2) Does it get called several times in bursts in order to fill a bigger internal rendering buffer?

If (2) is true, that will completely break the strategy of the ScriptProcessorNode, if that internal buffer is bigger than the ScriptProcessorNode itself, you see? The "process()" method will get called several times in sequence, requesting several ScriptProcessorNode buffers to fulfill its needs, and that will try to schedule several callbacks to "onaudioprocess" almost at the same moment. Those will get piled to be executed on the main JS thread, and then a big interval with NO callbacks will follow.

We will get random silenced buffers sometimes, and as the pilled callbacks get indeed to be executed later, they will fill the wrong buffers (one of the two buffers on the double buffering scheme)...

I believe the locking mechanics on the ScriptProcessorNode is broken and will not prevent nor solve this case, and know why.

That explains exactly the odd behavior I got, which is several callbacks at the "same" moment, followed by a big pause, resulting in a stuttering and scrambled audio stream.

That is my main hypothesis at the moment. It would be great if anyone could validate it, or rule it out.

Many thanks

2) is exactly how it works.  In all cases, process() is called to process 128 frames at a time.  Depending on the system, there may be several calls to process back-to-back because the audio backend requires that we process larger blocks. Mostly because the audio backend doesn't have regular enough callbacks.  Mac can maintain 128 frame callbacks easily, but linux can only do 512 frames.  Anything smaller causes glitching on simple graphs because the audio hw callbacks are regular enough.

ScriptProcessorNode is supposed to know this and internally buffer up enough samples before firing the event. But if you set the ScriptProcessorNode size to be less than the block size, you will have problems.  This is why ScriptProcessorNode allows 0 to allow the system to choose an appropriate size.  (This has not been optimized in any way.)

It would be really nice to know what kind of HW block size your phone uses, but there's no current way to know short of building a custom Chrome or creating a small Java program to query the appropriate API for the values.

But I think most Samsung devices have good hardware and supports the Android low-latency path so the block sizes are small.  Currently the lower limit is 1024 frames for any Android device.  (This needs tuning.)

Couldn't I just create a ScriptProcessorNode with 0 and then get the default buffer size chosen?

Gotcha... Exactly what I was expeting.
On my Samsung Galaxy S6, the ScriptProcessorNode selects a 8192 byte buffer by default.
Anything lower than this does not work.

So I suppose there is something else that is hard-coded to that size, and it appears to affect only the ScriptProcessorNode, since on my tests with other simpler Nodes, they don't show such a big latency. I mean, it appears the internally the AudioContext is working with such a big buffer, as it is able to render simple nodes with no big latency.

The good news: I am already able to edit the code and build a custom Chromium to test on my device. Can you point me to the place on the code where it thinks it must use a 8192 buffer?

*** it appears the internally the AudioContext is NOT working with such a big buffer, as it is able to render simple nodes with no big latency.

The AudioContext forces all nodes to process 128 frames at a time.  Depending on the platform several calls to process will happen back to back.  In addition, the generated audio is double-buffered so the latency can be fairly large.

This is where ScriptProcessorNode chooses the buffer size: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/webaudio/ScriptProcessorNode.cpp?rcl=0&l=345

The buffer size for the AudioContext is computed here: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/audio/AudioDestination.cpp?rcl=0&l=84

>>> The good news: I am already able to edit the code and build a custom Chromium to test on my device. Can you point me to the place on the code where it thinks it must use a 8192 buffer?

I found this place. It chooses a value that is 4 times bigger than what it thinks the hardware buffer is.

Actually, in my Galaxy S6 (one of the best Androids around) it chooses 16384, so it thinks my hardware buffer is 4096???
Can this be right? It just seems too big to me.

On my Windows it chooses 2048, so it thinks the hardware buffer is 512.

Something is wrong with the value for Android. It can't really be a hardware limitation, since Firefox does not show this, remember?

Without having that device, it's hard to know what is actually being returned.

Our internal stats indicate that some 50% of all Android devices have a HW block size of 3840 frames or so.  That would result in script buffer size of 16384. I do not know what devices have 3840 as the size.

On my Nexus 9. the script processor uses a buffer size of 512.  The context buffer size is 1024.

>>> But I think most Samsung devices have good hardware and supports the Android low-latency path so the block sizes are small.  Currently the lower limit is 1024 frames for any Android device.  (This needs tuning.)

Its not, its actually 2048!
If the hardware buffer is anything less or EQUAL to 1024, it forces to 2048.
See here:
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/audio/AudioDestination.cpp?rcl=0&l=101

That seems too much to me. Definitely it need tuning.
I will build a Chromium that allows 1024 and see what happens!

>> On my Nexus 9. the script processor uses a buffer size of 512.  The context buffer size is 1024.

How is that possible? How can the ScriptProcessor choose a size smaller than the Context if it takes the HW size and multiply by 4?

What regulates the size of the Context buffer? The code you and I pointed above is for the AudioDestination, are talking about the same thing?

If so, how can your Context use 1024 if that code, on Android, forces it to 2048 for any HW size <= 1024?

Oops.  My memory was wrong.  But it is quite complicated.  When support was added for android, my test device (Galaxy Nexus S) reported a buffer size of 144.  This was really too low for the webaudio, and the best result was something much bigger (1024? 2048?).  Otherwise, demos would glitch.

Selecting this is really hard because the number of devices I can personally test is fairly limited.

I built a custom Chromium for testing. My logging capabilities are very poor, but I changed the code and removed the 2048 minimum size for the AudioDestination, and changed that 4x multiplier for the default size of the ScriptProcessorNode to just 1x.

So I was able to see what got chosen for my device, and it was 4096.

It's not possible that on a Galaxy S6 the audio buffer needs to be that big... Remember, Firefox seems not to use such big buffers.

Also, my device does not show such a big latency on other audio sources, games, etc.

Maybe the code that chooses the audio path or best hardware buffer size to use has some wrong or outdated assumptions?

Can you point me to the code that actually gets and chooses the HW buffer size to use?

Please, if you can answer quickly, it would help me alot!

I am allocating a powerful Amazon EC2 machine just so I can build Chromium, and I pay that by the hour! :-)

Thanks

I rebuilt chromium and set a scriptprocessor buffer size of 2048 on my Nexus 9.  It works fine.

The code that gets the HW buffer size seems to have changed. It's going to take me a while to find it....

The HW buffer size is queried here:  https://cs.chromium.org/chromium/src/media/base/android/java/src/org/chromium/media/AudioManagerAndroid.java?rcl=0&l=516

And if you can build chromium already, you might want to put

LOG(ERROR) << "HW Buffer size = " << recommendedHardwareBufferSize

near line 86 in AudioDestination.cpp.  Then run "adb logcat" to see the size for your device.

>> I rebuilt chromium and set a scriptprocessor buffer size of 2048 on my Nexus 9.  It works fine.

That's great!

The more I look into this, the more I believe there is something wrong or outdated in the code that deals with the HW and chooses the buffer size.

I tried to trace the calls on the code to find the place where it happens, but without good tools its very hard for me to find it.

You said last week that you maybe could spend some time this week to look deeper into this issue. Is that still a possibility?  :-)

Thanks

Yes, I'm looking into it.  But it works (so far with simple tests) on the devices I have available, so progress is slow.  (And debugging on Android is really slow since I don't do that very often.)

Knowing the HW buffer size on your device would certainly help me to simulate something that is important to you.

>>>  Yes, I'm looking into it.

Great! Thank you very much!

I will run the test you suggested tomorrow.
But I don't know exactly how to debug and see the logs.
And I'm using a machine on Amazon, editing the sources on Notepad, transferring files between machines, and I cannot "plug" my device... Very hard and slow.

But, what I could see in my tests here, is that the HW buffer that WebAudio on Chrome chooses in my device is 5760.

That can't be the best choice! The Galaxy S6 is one of the best devices around today.

Thanks!

Are we using low latency audio on Android through OpenSL ES?

https://source.android.com/devices/audio/latency_app.html

If it's possible, yes.  I think this is all done in media/audio/android/audio_manager_android.cc.

Raymond, I got to build Chromium and run the test logging the buffer size exactly as you suggested 6 posts above. The result:

12-14 14:53:49.794 14679-14694/? E/chromium: [ERROR:AudioDestination.cpp(89)] HW Buffer size = 5760

That is a gigantic value! Something must be wrong...

Ugh.  That would certainly explain why you're getting horrible behavior.  Can you run your demo with a ScriptProcessor bufferSize of 16384?  At least it should sound better.

Now, as to why it should return this, I don't know.  I'm guessing that for some reason your device says it doesn't support the low latency mode and we end up using the large buffer size.

To investigate this further, it would probably be good to write a small Java app to query these values. I'll probably have to talk to the Android audio team about this.

Thanks for all of your help!

Yes, with 16384 it sounds great. But 2 seconds later...  :-)

I can write an Android app to query things, if you guide me on what you need queried. That should be easy.

So, what do I query?

Ok, so at least we know that ScriptProcessor works ok if the buffer size is adjusted correctly.

For the app, I'd start here:  https://developer.android.com/ndk/guides/audio/audio-latency.html.  Scroll down to "Validate audio performance".  There's a bunch of snippets of code to provide info on what's supported. 

Yes, it works with a large buffer, but its also not very useful...

I will try to create the App and see what else I can find!

But... I DO think there is a bug in the synchronization strategy on the ScriptProcessorNode. With such large buffers, it kinda goes unnoticed, but when/if smaller buffers start to work, it will mess things up.... This must get fixed.

Maybe that part of Chrome is not very used or tested, unfortunately.

I already mentioned this on some answer above, but I can show you where the problem is in greater detail if you want.

I have spent the last 2 days investigating this.
I made a small Android app to query some properties on my device. The results:

PackageManager.FEATURE_AUDIO_LOW_LATENCY: false
AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE: 48000
AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER: 1440

Well... A Galaxy S6 does not support the LowLatency feature? Strange...
(could you tell me what your test device reports for those?)

But it is worse. Chrome is using a 5760 buffer, which is 4x the size reported by Android! Why?

On further investigation, I found Chrome uses some other Android query:

AudioTrack.getMinBufferSize(): 5760

That explains it, but I still don't understand why there are 2 queries that return different values, and why Chrome chooses to use the bigger one.

On top of that, I believe Chrome adds several double buffers for WebAudio, at least 2 but maybe more.

Also, there is a limitation that forces the ScriptProcessorNode buffer to be a power of 2. I also don't understand why, it seems unnecessary, but that forces me to use a minimum buffer of 8192 instead of the already big 5760.

Chrome then adds double buffers to that... and it becomes enormously big. Really unusable high latency.

So I built a special version of Chromium that uses the lower reported 1440 buffer, and also removed that power of 2 limitation on the ScriptProcessorNode, which allowed me to use 1536 (to meet the multiple of 128 requirement).

And it worked!!! Very small latency, but a little too sensitive so I could hear some artifacts, probably buffer underflows. Bumping the buffer to 2048 practically got rid of the artifacts.

Well... I don't know what can be done here.

There are some issues on the WebAudio spec itself, like the power of 2 rule for the ScriptProcessorNode. We need to know in JS what is the size of the internal Context buffer! And also be able to use the exact same value for the ScriptProcessorNode!

But there's also some things in Chrome that hurt the performance, I think. Maybe there are too much double buffers, or the buffers chosen are too conservative. Maybe other things are preventing Chrome from choosing the best path, at least on my device.

I can now understand what is making Chrome perform the way it does on my device, but I also remember that Firefox performs a lot better in this regard. 

Maybe Firefox uses the lower 1440 value instead of the 4x higher 5760, or maybe it does not double buffer as much.

I don't think I can go much deeper than this with the tools I have and my limited understanding of the code.

I would very much like to hear your thoughts on all of this!
Do you think there is some hope of improvement in the short term?

Thank you.

First, thanks so much for doing all this work.  It's really helpful!

I am surprised that your phone doesn't support the low-latency path. And the buffer size of 1440 is rather large (but not crazy huge).

My Nexus 9 device reports 128 for OUTPUT_FRAMES_PER_BUFFER, but still uses 1024 as the callback buffer size.  The spn node uses 512 as the bufferSize. I get roughly similar results on a Nexus 4.  My Nexus 7 (original) doesn't support low-latency and uses a 3K buffer.  But I did test it using a callback size of 256 or 512 and it worked fine.

When you mention 5760 as the buffer size, which buffer size are we talking about.  There's the HW audio buffer size (OUTPUT_FRAMES_PER_BUFFER), the callback buffer size, and the ScriptProcessor bufferSize.

We can clearly do a better job of choosing the ScriptpProcessor buffer size, which is just the power of two that is 4 times larger than the (HW) buffer size. (That's wrong.  See  issue 673854 ). I need to think about some heuristics for this.  If the callback size is already large, it might be fine just to round up to the next power of two.

We can also optimize the callback buffer size better. I think for devices that have HW buffer sizes that are 128 or 256, we can use 2-4 times that value.  For larger values, like your phone, we could probably keep it as is or maybe the next power of two.  Will need to experiment a bit with this.  We do have a collection of random Android devices that I might be able to use for this.

I'll let you know (here) when I come with something.

BTW, since you asked about the usage of ScriptProcessorNode: https://www.chromestatus.com/metrics/feature/timeline/popularity/646.  Usage is very low, except for some strange unknown spike in Nov or so.

>>>>  BTW, since you asked about the usage of ScriptProcessorNode: https://www.chromestatus.com/metrics/feature/timeline/popularity/646.  Usage is very low, except for some strange unknown spike in Nov or so.

Maybe it was me! Do you know the website doing this?

If you're interested, I can send you a link to my project.
It's an Online MSX emulator (vintage computer/game console), running completely on vanilla JS. Its a nice showcase of several Web technologies being used together. It pushes the browser a little!
I also have an Atari 2600 emulator using the same technologies.

I'm am now releasing the mobile-ready version, with a specialized GUI for touch devices.
Then I stumbled on the audio latency problem on Android, and have spent some time trying to make it work better before releasing.

We just count how many times createScriptProcessorNode is called.  AFAIK, we don't keep track of where that came from.

If your project is publicly available, please send a link. We're always looking for projects making interesting uses of WebAudio.  Because they're interesting and because it helps us figure out what's important and what's not.

Probably won't get any fixes in until next year, but I will start laying some of the ground work to optimize WebAudio better for Android.  It's long overdue.

>>>> When you mention 5760 as the buffer size, which buffer size are we talking about.  There's the HW audio buffer size (OUTPUT_FRAMES_PER_BUFFER), the callback buffer size, and the ScriptProcessor bufferSize.

My phone reports this:

PackageManager.FEATURE_AUDIO_LOW_LATENCY: false
AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE: 48000
AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER: 1440
AudioTrack.getMinBufferSize(): 5760

As I understand, Chrome first asks if Low Latency is supported. If YES, then it uses OUTPUT_FRAMES_PER_BUFFER, if NOT it uses AudioTrack.getMinBufferSize(), right? Starting here: https://cs.chromium.org/chromium/src/media/audio/android/audio_manager_android.cc?rcl=0&l=403

In this scenario, what would be Callback buffer size you mention?
As I could understand from the code, it will be 5760 for my device. The value from OUTPUT_FRAMES_PER_BUFFER is never used when LowLatency is false!!! I don't see the value 1440 being used anywhere in my device, am I wrong?

So in my device, the SPN by default will use at least 4x 5760! (BTW, there is also a bug in this calculation)

>>> My Nexus 9 device reports 128 for OUTPUT_FRAMES_PER_BUFFER, but still uses 1024 as the callback buffer size.  The spn node uses 512 as the bufferSize.

I can't see how this is possible, from my understanding of the code. Being ANDROID your device would use 2048 for the callback, right?
See: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/platform/audio/AudioDestination.cpp?rcl=0&l=88

How is the 1024 callback size calculated for your device? From the code above, it WOULD try to use 128, but then would bump it to 2048.

The SPN will in fact use 4x 128 = 512. But you see, there is also a problem here. In your case you got lucky, but in most cases the SPN buffer will NOT be a multiple of the callback buffer, which results in uneven calls to the SPN onaudioprocess. My device included.

Even if I specify a custom value for the SPN buffer, in my device it will perform poorly. Some choices for the SPN for me are:
1) 4096, which is close to the callback size of 5760, but is LOWER than it and will produce uneven calls.
2) 8192, which is bigger but not a multiple, will also produce uneven calls.
3) Any value < 5760 produces terribly uneven calls, which SPN is simply not prepared to handle and will scramble audio buffers.

That without mentioning the very high latency...

I have tested some devices, and all of them report FALSE for the LowLatency feature. Maybe only NEXUS devices support this yet?
Maybe the other Android vendors are not going to support the LowLatency feature...

Any plans to support Samsung Professional Audio SDK? http://developer.samsung.com/galaxy/professional-audio
Its not the standard Android OpenSL path, but works great...

About my project.
Its a one-file responsive WebApp that runs in both Desktops and Mobiles. Can be installed as an App on iOS/Android and also runs offline. It can even be sent as en email attachment! 

Its an emulator, so you would need ROMS to try it.

Here is a link that will automatically open a game ready to play on the latest alpha:
http://webmsx.org/alpha/?auto=/files/Hype.zip

The alpha release with support for mobile touch devices: http://webmsx.org/alpha
The current stable release, good only for desktops: http://webmsx.org

Hope you enjoy it! ;-)

Oh god... I updated my phone from Lollipop to Marshmallow and updated Chrome to the latest version to see of that would bring any improvements.

And now I have more problems. :-(
The audio issue remains, but now I have terrible slowdowns and lots of frames dropped each time the screen is touched! Its a lot worse than it was before. No changes in my side.

I removed ALL my touch events. No difference.
Even with no events attached, if you just touch the screen, there is a slowdown and "requestAnimationFrame" is delayed for some frames.

That makes Chrome unusable for any game-like apps that use requestAnimationFrame, and touches...

I will try to debug and find more info.
Any hints or ideais on how I could run and see some profiling on Chrome for Android?
I know how to do that on the Desktop, but not on Android.

Thanks.

This is really terrible.  Sorry about that!

Can't help much on this new issue because I don't know how that part works. You may want to file a new bug on requestAnimationFrame behaving badly on Android Marshmallow.

I don't have any experience with profiling Android at this level, but perhaps https://developers.google.com/web/tools/chrome-devtools/rendering-tools/ can help.  My profiling has been mostly limited to using perf record to capture runtime CPU usage.

It appears to be a speed problem...
With the new version my app can barely compute each frame in time.
Somehow when you touch the screen Chrome is adding some substantial extra processing, and that makes my app not fast enough to render the frame in time...

I have rolled back to version 50.0.2661.89 and it works fast again.
The new (current) version is definitely slower, at least for my app.

BTW, did you get a chance to look at my app?  :-)

Doing more test, I found that the poor performance also happens in my Windows Desktop since I updated to version 55. Its not an Android issue!

The performance is like 5 TIMES SLOWER than it was before. No changes in my part, and now the emulation core runs 5x slower. Nothing related to graphics, sound or input, just the core!

Typed Arrays performance is VERY VERY POOR in this release. 

I have a method that makes writes to TypedArrays, and before this release it almost was not even shown in profiles. Now it shows in the first place taking up to 60% of all the processing time!

Something is really wrong, maybe in V8. If that is intentionally part of the "memory use optimizations" of release 55, we are doomed! :-(

Can you please direct this to someone involved with this part?

Thank you!

First, I have not yet looked at your project, but will soon.

Second, I'd file another bug about your TypedArray experience.  Cc me so I see it and I'll try to route it to the appropriate people.

How can I CC you? I don't see such feature in the interface!

Down at the bottom of crbug.com/new, there should be a Cc line.  Add (or enter) rtoy@chromium.org to that line.

Raymond, I have created the new issue regarding the JS performance problem, but I forgot to CC you.
https://bugs.chromium.org/p/chromium/issues/detail?id=676005

I am pretty sure this is a very bad optimization problem introduced in v55.
This is now my priority, even over the WebAudio problem, since this affects also all Desktop Chromes.

Can you please CC yourself and maybe direct this to the right people?
I believe this one is pretty urgent!

Thank you again!

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

With the imminent availability of AudioWorklets enabled by default for Chrome 66, we would rather spend our time making AudioWorklets work better instead of working on the deprecated ScriptProcessorNode.

Thus, closing this as won't fix (obsolete).

If this issue is really important and can't be solved with an AudioWorklet, please reopen or file a new issue (referencing this one).