New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 80 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2013
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Launch-OWP
Launch-Accessibility: ----
Launch-Exp-Leadership: ----
Launch-Leadership: ----
Launch-Legal: ----
Launch-M-Approved: ----
Launch-M-Target: ----
Launch-Privacy: ----
Launch-Security: ----
Launch-Test: ----
Launch-UI: ----
Rollout-Type: ----

Blocked on:
issue 239498



Sign in to add a comment

Support for Web Audio API on Chrome for Android

Reported by meh@chromium.org, Dec 13 2012

Issue description

(See go/owp-launch-guide for an overview)

Change description:
Web Audio is a high-level JavaScript API for processing and synthesizing audio. The goal of this API is to include capabilities found in modern game audio engines and some of the mixing, processing, and filtering tasks that are found in modern audio production applications (and now bring it to mobile!).

Changes to API surface:
Extensive new functionality will be available with 'new AudioContext()' (prefixed form: webkitAudioContext();).

Links
Cr bug feature request: https://code.google.com/p/chromium/issues/detail?id=112930
Public standards discussion: https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html
HTML5Rocks: http://www.html5rocks.com/en/tutorials/webaudio/intro/
Fun drum machine: http://chromium.googlecode.com/svn/trunk/samples/audio/shiny-drum-machine.html

Support in other browsers: current (expected)
Internet Explorer: no (unknown)
Firefox: using deprecated version (planned, specifics unknown)
Safari: no (expected prefixed in 6.0)
Safari on iOS: no (expected prefixed in 6.0)
Opera: no
Chrome desktop: prefixed support since 10
Chrome on iOS: yes
http://caniuse.com/audio-api
 
Are there any plans to implement this soon on Android?  I work for a large e-learning company and it's a huge competitive advantage to have the Web Audio API implemented.  Since Safari on iOS 6+ supports the Web Audio API, we direct most users towards Apple devices.

Comment 2 by meh@chromium.org, Feb 11 2013

Cc: rtoy@chromium.org
Labels: -Mstone-X Mstone-27
Thanks for your comment. Work is in progress, and we plan to launch something in the first half of this year.

Comment 3 by monte...@gmail.com, Feb 25 2013

Any chance this will make it to the next stable chrome android release?

Without this we're kinda dead in the water for html5 gaming because we have no way of preloading sounds with audio tags in android:  https://code.google.com/p/chromium/issues/detail?id=138132

Comment 4 by meh@chromium.org, Feb 25 2013

This is still scheduled for M27, due to hit stable in the first half of this year.
Project Member

Comment 5 by bugdroid1@chromium.org, Mar 9 2013

Labels: -Mstone-27 M-27
Project Member

Comment 6 by bugdroid1@chromium.org, Mar 9 2013

Labels: -OWP-DesignReview-No OWP-Design-No

Comment 7 by kdj.ti...@gmail.com, Mar 10 2013

Hi,
In Chrome-Desktop WebAudio and <audio> tag are using pipeline+FFMPEG decoders.
Here for webaudio/<audio>tag support in chrome for android. i have one query:

1. As per my understanding, currently FFMPEG is disabled for <audio> tag as well as WebAudio, so is there any plan to use FFMPEG or any other libraries for WebAudio decoding purpose ?

Also correct me if i am wrong.

Comment 8 by kdj.ti...@gmail.com, Mar 11 2013

In case of FFMPEG usage already audio_decoder.cc is available with decoder API implementation. Whereas In case of normal <audio> tag the decoding is happening using JNI_MediaPlayer; are you going to modify the audio_decoder_android.cc for calling decoder in WebAudio case ? 

Comment 9 by meh@chromium.org, Mar 18 2013

Labels: -M-27 M-X

Comment 10 by meh@chromium.org, Mar 21 2013

Labels: -OWP-LaunchBug-Needed OWP-LaunchBug-NA
Goal is to land this behind a flag in M-27.
Any comments on below lines:
1. In case of Desktop Chrome, FFMPEG is used with audio_decoder.cc and has decoder API implementation. Whereas In case of normal <audio> tag the decoding is happening using JNI_MediaPlayer; so in future modifications would be done for FFMPEG usage or JNI MediaPlayer ?
nice! so today is M27... any news?

Comment 13 by meh@chromium.org, Mar 28 2013

We're now targeting M-28.
Hello, my name is Alex and I'm working for Qualcomm on behalf of Code Aurora Forum. I've looked at enabling WebAudio on content shell for Android. We'd like to contribute on the topic.
Using those following patches, we've enabled WebAudio on content shell for Android
https://codereview.chromium.org/13084005
Also the following patches under https://www.codeaurora.org/patches/quic/webtech:
1602-kissfft-add-neon-optimized-routines.patch
1603-webkit-add-kissfft.patch
1624-chromium.src-implement-webaudio.patch
1625-webkit-enable-webaudio.patch
I'm currently working uploading 1624 as my first tentative failed because of illegal include.
I'd really appreciate any feedback as it's a first contribution
hi, any idea how is going to affect, Blink (and the abandon of webkit), to the web audio api?
Alfonso: it won't.  Support for the Web Audio API is still being worked on, and it should be in good enough shape to at least launch behind a flag in Chrome 28.

Alex: I cc'ed Raymond on your CL, he may be able to provide more insight.  In general, please always file an issue so there's a more convenient place to discuss your contribution :-).  Thanks!
Hi Peter and team,

I would like to add more here.

1. As per the approach suggested in#14, we can see the overload of memcpy() and "wait-time till decode complete", which is not favourable. Just imagine if the encoded file is huge (5MB); after decoding huge pcm data( 5 * CompressionRate MB) copy from host to browser and API has to wait till decoding get finished.

2. Also  w3c-WebAudio specs has recommended Web Audio API for fast quick responsive audio effects.

3. So here I have modified the code for FFMPEG usage and tested it; WebAudio API is up. This approach is quite simlilar to Desktop solution keeping decoding on renderer side and avoiding shared-memory memcpy() . Also keeping JNI player usage for <audio> tag is intact.

4. Referring: https://code.google.com/p/chromium/issues/detail?id=145255
FFMPEG is not being used for Android, so bit worried about the performance of Web Audio API.


5. Please let me know where to discuss further more on this. I am ready to join chat-forum, chat-room  ? or New bug to be raised for discussion ? 

6. If it is ok to submit my patch also let me know.

Please comment on this or suggest for corrections.

br,
kdj


Comment 18 by rtoy@google.com, Apr 26 2013

Let me address some of your questions.

1. Yes, there can be a large delay in copying large decoded pcm samples from the browser.  However, typical WebAudio usage is to preload such large samples before starting processing, so there is a startup delay, but that should not affect audio.

3. I also added support for FFMPEG some time ago and it worked quite well.  However, on Android, licensing issues prevent us from using FFMPEG.  Also, it seems a quite wasteful to duplicate all of the decoders when Android already has decoders available.

4. I believe WebAudio uses FFMPEG for two things:  the audio decoders and the FFT.  The audio decoders, as mentioned in 1, are used to load up assets before using them.  The FFT, however, is very important since it represents a significant amount of processing.  Some tests that I have done shows that the FFMPEG FFT is about 25% slower than what we are using now.

Also, you didn't mention it directly, but looking over your proposed patches, the Android team highly recommended using the Java MediaCodec class to do decoding instead of OpenSL ES.  MediaCodec is, I think, the only fully supported scheme for decoding audio.
Hi,

Thanks for providing valuable reply.

I really wonder about FFMPEG usage, as per my understanding it is under LGPL, so if we are loading and using in shared object form, it is fine. Please let me know if i am wrong. Also,

1. Regarding copying delay, with this approach starting time would be too high. It would be really bottle-neck in complex game applications, where multiple files and AudioContext are doing processing.

2. I want to just have a look for your approach, so if you landing all your patches together in android code, please tell me the patch numbers.

3. which code repository are you publishing the patches ?

4. For FFT solution, is there any separate library being used ? or it has been integrated with Java MediaCodec class.

br,
kdj
one more thing, wrt 25% better performance of your fft solution,  
5. are you comparing with FFT C implementation of FFMPEG or any platform based optimised code ? like x86, ia32, arm, arm-neon etc.. 
6. Similar to Apple webkit.org are you using any HW accelerators for FFT ?

Comment 21 by rtoy@google.com, Apr 29 2013

0. I do not really understand the licensing issues, but I have been told that FFMPEG is not allowed.

1. I've done some measurements and the bottleneck is not the copying.  It is the cost of MediaExtractor.setDataSource which seems to take a minimum of 20 ms to sniff the audio format.

2,3. The code is now checked into the chromium and blink repositories.  The main patch is https://codereview.chromium.org/12457043/ for chromium.  The blink side is http://trac.webkit.org/changeset/147491.

4. We are using a modified version of OpenMAX DL from ARM.  You can find it at third_party/openmax_dl.

5. I tried to compare implementations using NEON optimizations.  It is certainly possible I made a mistake.  Our OpenMAX DL implementation requires NEON.  

6. The only HW "acceleration" is the use of the NEON instructions for the FFT.
hi, 20ms is like one frame time + memcpy(it is system specific), as overall App.The delay should not cause web audio go down :-)

Regarding FFMPEG FFT, yes it is one of the best and having vectorised NEON instructions in assembly too.

i would like to test once with all your patches, Could you please tell me are those patches available with latest nightly build chromium.r197479.tgz ?
or do i have get from any other code repo ?

thanks for information.

Comment 23 by rtoy@chromium.org, May 3 2013

20 ms doesn't seem like a lot, but if you have 200 files that you want to load, that means no output for 4 sec or more.  (See, for example, https://code.google.com/p/chromium/issues/detail?id=232973.)

The openmax dl FFT uses vectorized NEON instructions in assembly too.

All of the patches should be available in the chromium repo.  r197479 should have it, but I didn't check.

Comment 24 Deleted

Comment 25 by rald...@gmail.com, May 12 2013

Hi Ryan, if you have a little read up the thread. The chromium team are hoping to have this available for M28 behind a flag.

Rgds
Rob

Comment 26 by Deleted ...@, May 12 2013

Hi Rob

Thank you, though I'm not sure what "M28 behind a flag," refers and or means?

Best,
Ryan
Hi Ryan,

"M28 behind a flag," refers to web audio API  being in release M28 for Google Chrome for Android ( we are at M27.xxx ATM so M28 is the next major release )
e.g. : http://www.chromium.org/developers/calendar & http://googlechromereleases.blogspot.co.uk/ 


and then you will need to enable the Web Audio API flag ( http://blogs.adobe.com/cantrell/archives/2012/07/all-about-chrome-flags.html )

cheers
Justin

Justin Heyworth
Head of Production
www.atticmedia.com

 
Sign up to our e-bulletins
 
Atticmedia, Unit A502, The Biscuit Factory, 100 Clements Road, London SE16 4DG.
Tel: +44 (0)20 7237 0778   Fax: +44 (0)20 3318 0271
This email is intended only for the addressee named above. If you are not the named addressee or the person responsible for delivering the message, contact the sender and delete the material from your computer.This message reflects the sender's views and opinions not necessarily those of Atticmedia.
Atticmedia Ltd, Registered Address: 145/157 St John Street, London EC1V 4PY. Company registered in England and Wales with company number: 7323829. VAT registration number: GB 994 7473 54
I've been thinking M28 is Version 28, so we'll have this in next release,
that's great!
Hi support@toremote.com

To be clear web audio API  would first appear in the next MAJOR release (as aposed to minor ) of  Chrome BETA for Android ( for testing by developers ) and NOT  the next Stable release of Chrome for Android which is what the majority of the general public would have on their contemporary Android device.

The current Stable release of Chrome for Android is 26,  Chrome BETA for Android is 27, and this thread proposes that Web Audio API would first appear in Chrome BETA for Android 28 ( behind a flag ).

Thus, web audio API would not appear in a Stable release of Chrome for Android until two MAJOR releases from now at the earliest. 
WEB AUDIO API NOW WORKS ON CHROME for ANDROID BETA M28 ( just released TODAY):

https://play.google.com/store/apps/details?id=com.chrome.beta&hl=en

Comment 31 by rald...@gmail.com, May 23 2013

This is good progress. With chrome 27 now live Google released the racer experiment which has very good audio, without web audio api (unless of course they fudged something in 27) I'm interested to know how they got audio to play without user interaction and without delay.
Even though the newly released M28 on Android Chrome Beta (specifically 28.0.1500.21) is supposed to support the Web Audio API (I do see the option in Chrome flags now), it doesn't appear to actually do anything when you enable the flag. Using the remote Chrome debugger for Android there is still no webkitAudioContext object like exists on the desktop version of Chrome. What additional functionality is the Web Audio flag supposed to enable?
Works for me.  When I run http://dungeonfury.playcanvas.com/ all the sounds play correctly on Android now.  Did you click Relaunch Now when you toggled the flag in chrome://flags?
i had to reboot device to get web audio api audio to play. I can confirm it does play web audio api audio.

Justin Heyworth
Head of Production
www.atticmedia.com

 
Sign up to our e-bulletins
 
Atticmedia, Unit A502, The Biscuit Factory, 100 Clements Road, London SE16 4DG.
Tel: +44 (0)20 7237 0778   Fax: +44 (0)20 3318 0271
This email is intended only for the addressee named above. If you are not the named addressee or the person responsible for delivering the message, contact the sender and delete the material from your computer.This message reflects the sender's views and opinions not necessarily those of Atticmedia.
Atticmedia Ltd, Registered Address: 145/157 St John Street, London EC1V 4PY. Company registered in England and Wales with company number: 7323829. VAT registration number: GB 994 7473 54
You are correct - dumb user error on my part. I can confirm it works as well. Thanks!

Comment 36 by rtoy@chromium.org, May 24 2013

It's a bug if you have to reboot to enable webaudio.  Justin, can you just disable and enable webaudio using about:flags and relaunching?
I can. I had run out of RAM I think ...

Justin Heyworth
Head of Production
www.atticmedia.com

 
Sign up to our e-bulletins
 
Atticmedia, Unit A502, The Biscuit Factory, 100 Clements Road, London SE16 4DG.
Tel: +44 (0)20 7237 0778   Fax: +44 (0)20 3318 0271
This email is intended only for the addressee named above. If you are not the named addressee or the person responsible for delivering the message, contact the sender and delete the material from your computer.This message reflects the sender's views and opinions not necessarily those of Atticmedia.
Atticmedia Ltd, Registered Address: 145/157 St John Street, London EC1V 4PY. Company registered in England and Wales with company number: 7323829. VAT registration number: GB 994 7473 54
No problem here to relaunch and play on Nexus 7, but compared with iPad2 Safari6, it's little lag on Nexus 7 (using same application)

Comment 39 by rtoy@chromium.org, May 24 2013

I would appreciate it you could file a bug on this (including a link to the app) so we can track this.  We know, though, that the audio latency is rather high right now.
My app is an html5 RDP client which support audio playing. You can try it here (need connect to an rdp enabled computer);

http://www.remotespark.com:8080

Web Audio on Android is still in beta and I believe it'll be better finally.

Comment 41 by murat...@gmail.com, May 24 2013

It keeps crashing on Nook HD+ (Android 4.0). It installed ok. Enabled the flags(s). But no demo worked. It crashes the whole chrome. Restarted the whole tablet but still the same...I hope it works  ok for other devices...

Comment 42 by rtoy@chromium.org, May 24 2013

Please file a bug report on the crash including a link to the demo you tried to run.
Cc: punyabrata@chromium.org
Blockedon: chromium:239498
Many crash reports coming in with:

[Android Java Exception] java.lang.NoClassDefFoundError: android.media.MediaExtractorat org.chromium.media.WebAudioMediaCodecBridge.decodeAudioFile(Unknown Source)

https://crash.corp.google.com/reportdetail?reportid=27995a08ef3fc7bb
https://crash.corp.google.com/reportdetail?reportid=c17c8191ad4473d5
https://crash.corp.google.com/reportdetail?reportid=4fbdd0f129d60264

Comment 46 by rtoy@chromium.org, Aug 14 2013

These crashes are caused by a wave decoder issue in Android.  See https://code.google.com/p/chromium/issues/detail?id=259048.  They should be fixed now.
When will the Web Audio API feature be pushed to the stable release version?

Comment 48 by rald...@gmail.com, Aug 16 2013

I think if everything goes to plan, M30 according to: http://www.chromium.org/developers/calendar
So late September.

Comment 49 by Deleted ...@, Aug 29 2013

When trying to load a web audio enabled source on any of the following phones with the below written android version and the Chrome Browser version: 29.0.1547.59 it results in the browser crashing. Trying to do the same with Android version 4.1.X and the same phone and it works fine.

One example that I tried to run was: labs.dinahmoe.com/plink
All are European devices:
HTC one X		4.0.4
HTC ONE V 		4.0.3
HTC ONE S 		4.0.3
HTC Sensation XL	4.0.3
HTC Sensation		4.0.3
Samsung Galaxy S2	4.0.4
Samsung Note 		4.0.3
Incredible S		4.0.4
Xperia S 		4.0.4

Comment 50 by rtoy@chromium.org, Aug 29 2013

WebAudio will not run on Ice Cream Sandwich (4.0.x); it requires Jelly Bean or later.  I think this particular crash is fixed in the latest M29, which disables webaudio on ICS.

Comment 51 by Deleted ...@, Sep 25 2013

Using my Nexus 7 tablet I notice in chrome that audio clips at times.

A big bummer for our play audio in sync across devices app (SpeakerBlast).  The clipping causes the audio to go out of sync with the other devices (iPhones & iPads) playing in sync.

Any others experience clipping issues with Chrome for Android's web audio API implementation?

Comment 52 by rtoy@google.com, Sep 25 2013

Do you mean speakerblast.com or something else?

And what do you mean by audio clips? Gaps in the audio output or that the audio is clipped because it's too loud?

Comment 53 by Deleted ...@, Sep 28 2013

Hi

Thanks for the response.

Indeed, http://SpeakerBlast.com is our app.  

In July we presented at a tech demo with an audience of 100; had 40 to 50 Apple devices blasting audio in sync.  You can see this demo video on our blog (http://www.speakerblast.com/blog.html), if interested.

Well we are presenting in DC to audience of 700 in October and we'd love for half to more then half to blast audio in sync on their devices.  From Apple to Android devices, though it seems Web Audio API acts slightly differently either on different OS(es) or devices?  We'd love to understand and possibly see if there's a fix so indeed the majority of the audience can join the SpeakerBlast in DC in October.

I'll post a video in a few days of my various devices playing in sync to detail what I mean by my Android device clipping & falling out of sync.  Maybe their is a solution?

thanks,
Ryan

Comment 54 by rtoy@google.com, Sep 28 2013

Ok. A video would be fine.

I tried this out using a Nexus 7. It won't even try to play any audio. There's never a button or anything to click to connect and after the start time has elapsed my tablet says it missed the start time and can't play.

I'm missing something on my tablet.

Comment 55 by meh@chromium.org, Oct 28 2013

Cc: -rtoy@chromium.org peter@chromium.org
Labels: -OWP-Format-Prefix -OWP-LaunchBug-NA -OWP-Documentation-Needed -OWP-TargetChannel-Stable -M-X M-30
Owner: rtoy@chromium.org
Status: Fixed

Comment 56 by Deleted ...@, Jan 2 2014

Hi 

We wanted to thank the Chromium team!

Our web app that plays, pauses & stops audio in sync across multiple devices (http://SpeakerBlast.com) now works in Chrome for Android.

Previous Chrome for Android versions that supported the web audio API had a few bugs where audio clipped when the user changed the device's orientation (landscape to portrait). 

We are excited that issue has been solved and to note our web app now functions across all popular platforms.

Happy New Year!

Sign in to add a comment