New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 59 users

Issue metadata

Status: Fixed
Closed: Nov 2016
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Feature

Sign in to add a comment

decodeAudioData can't decode audio, but Chrome itself can

Project Member Reported by, Apr 30 2015

Issue description

I have attached a file that's been generated with libopus (via Recorderjs -

If you drag the file onto Chrome, or use it with an <audio> tag it works just fine. If you pass it to decodeAudioData it fails (meta issue: the error is just 'null'). The same file works in Firefox, and you get the decoded buffer.
23.1 KB Download
Wondering if this is because it uses Opus as the Ogg codec rather than Vorbis. Does the Web Audio API use the same decoding machinery as Audio and <audio>? If so it should've supported Opus since ~M33 (

Comment 4 by, Apr 30 2015

WebAudio doesn't support Opus.  WebAudio depends on ffmpeg (on desktop) for decoding files and opus support isn't enabled in ffmpeg.  I think Opus support on desktop uses a different opus decoder, and that hasn't been ported to webaudio yet.

Comment 5 by, Apr 30 2015

I really think we need Web Audio to have broader decoding support.  It's pretty goofy when you can decode via <audio> and pipe it to Web Audio, but can't decodeAudioData directly.

Comment 6 by, Apr 30 2015

I agree.  WebAudio should support whatever <audio> can.  Eventually.

Comment 8 by, May 5 2015

 Issue 409402  has been merged into this issue.

Comment 9 by, May 5 2015

Labels: -Type-Bug Type-Feature

Comment 10 by, Oct 20 2015

Status: Available

Comment 11 by, Oct 31 2015

I'm trying to workaround this issue (by implementing a decoder myself), but it is difficult to detect support.  According to html5 specs (as well as references on forums by devs), canPlayType is supposed to be the method to detect decodeAudioData's supported codecs.  Unfortunately this is not the case.
Is there an alternative (reliable) way to detect, other than UA sniffing?
just to let you know that I was able to re-compile Chrome on my mac with support for Opus via web audio API.
As described in this thread this can be easily achievable by compiling ffmpeg with support for Opus decoding via libopus:

From src/third_party/ffmpeg:
git diff
--- a/chromium/config/Chromium/mac/x64/config.h
+++ b/chromium/config/Chromium/mac/x64/config.h
@@ -1027,7 +1027,7 @@

and by adding compilation of avcodec/libopus.c and avcodec/libopusdec.c 
I'm new to Chrome. I'll try to figure out if I can push these changes.
My guess is that the change will not be accepted, because it would mean that Chrome would compile two libraries that do the same thing (decoding Opus). This is a waste of space.
#13: My counterargument here would be: the disparity between <audio> and WebAudio indicates that there are already two libraries decoding media. Until the larger issue of unifying these can be resolved, adding Opus support to WebAudio would at least abate the FF<->Chrome issues brought up in
While this may seem logical given this bug and its root cause, do you know that there are multiple decoders for the same codec for sure (excluding hardware and software decoders, of course)? :)
My guess would be that this does not happen, but I may be wrong, of course.

Comment 16 by, Nov 6 2015

WebAudio and <audio> share the same ffmpeg interface.  Opus support is a separate codec interface that has not been added for webaudio.

We certainly do want webaudio to support exactly (and no more) the same set as <audio>.  It just takes time and manpower.
hi, thank you for your comments.
I'm not an active chromium committer so I cannot push my changes in any case. Of course I'm available in sharing the patch if needed.
I agree with you that chrome incorporates at least two different paths in decoding Opus, but at least both seem to end up in the libopus implementation (already linked to chrome). Indeed I selected the libopus support in ffmpeg and not the native support (for native support see #define CONFIG OPUS DECODER in ffmpeg config.h).
Since Chrome isn't supporting Opus from some time I just wanted to produce a simple fix.

Comment 18 by, Nov 7 2015

Can we get a quick fix solution, even if it is a little inefficient?  Does it really add a significant footprint (don't they both just endpoint at libopus anyways)?  If it doesn't, you can just optimize it later.  Let's please just not have a broken implementation (it is stated by standards body that decodeAudioData should support codecs indicated by canPlayType).
This makes it impossible to use Opus, even with a fallback.  It's effectively handicapping Opus.

Because of this, we are stuck with using mp3.  Or aac.  But if we can have Opus, wouldn't it be more beneficial to everyone?
Instead, the open better choice (Opus) is handicapped: fragmented, buggy, unusable, encouraging the use of proprietary or inferior codecs, at no fault of its own.

I can understand with Vorbis, which is just mostly inferior, but Opus is really just a better choice even ignoring licensing issues.

Comment 19 by, Nov 9 2015

For a quick fix, use createMediaElementAudioSource to send the Opus data to webaudio.

I suspect no reviewer would allow adding Opus support to ffmpeg.
I wouldn't support adding libopus to ffmpeg, but we should probably look at changing AudioFileReader to reuse AudioDecoder implementions like FFmpegAudioDecoder and OpusAudioDecoder.  I don't think this would be too hard to do, I'm happy to help with the design if the WebAudio team wants to take it on.
Actually, I'm not sure I wouldn't support libopus usage either. Since we statically link ffmpeg now, this may not be very hard to do -- if it allows us to avoid maintaining OpusAudioDecoder and allows WebAudio to use opus w/o rearchitecting -- it may not be a bad idea.

The libopus shim is very small for ffmpeg, ~200 lines, while our own decoder+tests+etc is probably around a 1000 lines or more.  So I guess I would also support exploring this idea :)
+vigneshv in case he wants to weigh in.

Comment 23 by, Nov 9 2015

I will defer to dalecurtis@ and vigneshv@ to decide on the best approach.

Making it part of ffmpeg so that we can use the standard ffmpeg interface certainly appeals to WebAudio.
+tomfinegan, fgalligan

I'm not sure about what the reasons were behind adding OpusAudioDecoder instead of using the FFmpeg wrapper. The first CL [1] or the bug [2] doesn't shed any light on this. But my guess would be that the FFmpeg wrapper for libopus didn't exist or wasn't mature enough when we added OpusAudioDecoder in Chrome.

Here are my thoughts on this issue:
1) We (myself, Dale and Tom) have spent quite some time in handling Codec Delay and Seek Preroll in OpusAudioDecoder and it works correctly right now. I really doubt if the FFmpeg wrapper will handle that as correctly. So there definitely will be some work to be done around that.

2) FFmpeg also has a native opus decoder [3] (one that does not depend on libopus at all). It might be worth considering that as well and finding out which is better.

3) Will this change affect MSE somehow (since we support Opus in MSE)? I can't think of anything off the top of my head, but it's probably a good idea to consider that as well before making the switch.

TL;DR: there is no technical reason that's blocking removal of OpusAudioDecoder and switch to FFmpeg for opus decoding. But there will definitely be more work to be done in handling opus on par with how we handle it in OpusAudioDecoder today.

In the past, ffmpeg was a shared library, so linking in libopus decoder was a pain / added too much binary size. Now we static link ffmpeg, just like how opus is static linked, so we could definitely try it out.

I think ffmpeg might handle pre-skip, codec-delay correctly, we have plenty of tests for this, so if it's not the case they'll fail right away.

The tedious part is rebuilding ffmpeg to include libopus support, but if they get everything right, I don't see the need to maintain our own OpusAudioDecoder as well. We have integration tests which would work as is w/ opus through ffmpeg.
ah alright. i would still like to ensure that libopus performs better than ffmpeg's native opus decoder.

other than that, i don't mind this change.
+1 (again as long as everything works :) )
I think it's worth getting the numbers, but we will need to keep libopus for encoding anyways, so i think it makes sense to keep using it unless its substantially faster (it might be).
> but we will need to keep libopus for encoding anyways

we already have a case with vp8 where we use ffmpeg's native decoder for decoding and libvpx for encoding. so i think we should just pick whichever opus decoder is more stable and performant. :)

Comment 30 by, Nov 14 2015
Understandable.  I do remember how there were a lot of bugs in HTML5 video related to ffmpeg (as well as with pepperflash).
Thank you for the workaround.  Is there a way to detect this other than userAgent sniffing?

Comment 31 by, Nov 14 2015

For feature detection, I guess you can use an ArrayBuffer with the smallest amount of bytes needed for an Opus file to play (a single sample) and try to decode it. If it fails, use the workaround.
The just released MediaRecorder API records audio in...Opus format! Because of this bug, the recording cannot be used in Web Audio. Funny if not sad. Please fix this so not to waste the implementation of the MediaRecorder API.

Comment 33 by, Dec 18 2015

FYI: MediaRecorder isn't supposed to hit production until version 49.  By
then this bug will probably already have been sorted out.
In the meantime you'll probably want to look into asm.js and fallbacks.  Or
use raw.
It's a bit of extra work and not perfect, but the functionality can be
shimmed for the most part.  There's some hints in this thread; the most
hacky part is detecting the need for fallback.

Comment 34 by, Dec 18 2015

>By then this bug will probably already have been sorted out.
Since no one is working on this bug and the Chrome 49 branch point is January 15th, 2016, "probably" is too optimistic.

Comment 35 by, Dec 18 2015

 Issue 570980  has been merged into this issue.

Comment 36 Deleted

Comment 37 by, Dec 20 2015

'audio/wav;codecs=1' should work (although I haven't tested it on chrome).
Works on Firefox.
Guys...this had to be fixed in is almost April and still no one assigned to the issue. A bit sad as this blocks communication between mediarecorder api and webaudio api. The result of the recording cannot be used in web audio api...Please update us regarding this

Comment 41 by, Mar 25 2016

@dalecurtis:  Any decision on using ffmpeg to decode Opus?
I don't think we want to switch to using ffmpeg's opus decoder and trying to build ffmpeg with support for libopus might be pretty hard, but we should be fine to either add support to AudioFileReader for OPUS decoding. See here for an example of how that could be done using OpusAudioDecoder:

Comment 43 by, Mar 25 2016

So no intention to switch to ffmpeg in the foreseeable future?  Just want to be sure before investing time in adding a separate opus decoder.
Correct, you're welcome to experiment with trying to get ffmpeg building with --enable-decoder=libopus within Chrome, if you can that route is fine with me too.  Otherwise I think you'll have to abstract some of the decoder setup in AudioFileReader so that you can use either ffmpeg or libopus.

An alternative path is to rework AudioFileReader such that it can run in an asynchronous manner. You could then use the AudioDecoder classes used by <audio> directly. This has the advantage of allowing you to do a streaming decode of decodeAudioFileData() instead of waiting for the whole clip to download and then decoding it.

I think the latter is the most efficient and convenient route long term since it will improve performance and allow WebAudio to always support every audio codec used by <audio>.

Comment 45 by, Mar 25 2016

Thanks for the suggestions; much food for thought.

Having decodeAudioData being able to stream has been a common request but such a change would require a spec update.
Why does it need a spec update? That seems like something you can hide from the caller via a waitable event for now.

Comment 47 by, Mar 25 2016

I meant WebAudio decodeAudioData() method that only takes an ArrayBuffer.  Internally, we can whatever we want.  Making the internals use the same methods as used by <audio> is good.
Ah, dang, I thought it took a URL.
Hi everybody...any update on this?

Comment 50 by, May 19 2016

I discussed this a while ago with dalecurtis@ and decided to modify Chrome to use ffmpeg to use libopus to do the decoding. I have it mostly working on linux, android, and mac, but there are some issues that need to be resolved.

And ideally, what ever changes we make, we want the upstream ffmpeg project to take these changes.  This could be a showstopper.

Other tasks with higher priority have prevented me from continuing this work for now.
Given that the ffmpeg internal opus decoder is the default, I wouldn't be surprised if there's some minor issues with the libopus wrapper in ffmpeg that could use improvement - they keep it around mainly for use as a comparison and to check for regressions against the internal decoder.

That said, if you have some generally applicable ffmpeg changes to the libopus wrapper, I don't see why they wouldn't be accepted… Do you have a list of the issues that you're hitting available somewhere?

Comment 52 by, May 19 2016

We wanted to use libopus from chrome; ffmpeg supports this.  That way, exactly the same decoder would be used with webaudio as with the rest of Chrome.

The main issue I ran into is that ffmpeg wants libopus.dylib to exist when configuring on mac.  This means a burden for chrome's ffmpeg maintainer to have to have libopus built. (But IIRC, it only wanted to make sure certain functions existed; it didn't actually test anything else.)

Just to clarify, in the chromium build, I assume you're doing a static build of libopus (using e.g. ./configure --disable-shared) to link internally in the chromium binary, and you're also doing a static build of ffmpeg which should use the same static libopus?

The ffmpeg build is going to require a built libopus. There have been people trying to change the pkg-config checks in ffmpeg to not also do a build&link test, but they've hit considerable resistance in the past (people want a 'fail fast' on incorrectly set up systems). But it shouldn't require a system-installed libopus or a .dynlib file; a static archive from the chromium build tree will do fine.

I don't see where the maintainer burden comes in, shouldn't the build system handle building libopus before ffmpeg, since it's a dependency?

The usual way of building a static ffmpeg that links to a static libopus is to do a "make install" of libopus into a temporary staging directory, then set PKG_CONFIG_PATH to the location of the 'opus.pc' file in that staging directory. This should work fine on all setups.

There's another alternative available, which is to set PKG_CONFIG_PATH to the root of a build libopus source tree, in which case it should pick up the 'opus-uninstalled.pc' file...

But I just spotted an issue there where the opus-uninstalled.pc file uses a direct reference to the .a file rather than setting a linker path and doing -lopus, so the ffmpeg pkg-config test puts the library in the wrong link order and the link test fails. Not sure what the best fix for that is, it could be changed on either the libopus or ffmpeg side and I think in either case it would be a generically useful improvement.

Aside: to fix the opus-uninstalled.pc problem:
the libopus fix is to change "Libs: ${libdir}/libopus.a @LIBM@" to "Libs: -L${libdir} -lopus @LIBM@" in
the ffmpeg fix is to change the check_ld function's lib vs flags filter from '-L*|*.so' to '-L*|*.so|*.a' in configure

(I'd love to take a look at the chromium source tree... but it's kind of big and scary to check out! Perhaps this weekend...)

Comment 54 by, May 19 2016

Right now, I just build chromium, and use the libopus there for ffmpeg.

In third_party/ffmpeg/chromium there are a bunch of directories that ultimately contain a config.h that was generated specifically for that platform configuration.  Some one needs to update them all. And update them whenever chromium pulls a new version of ffmpeg.  I think; I'm not the ffmpeg maintainer for chromium.

I tried changing PKG_CONFIG_PATH to include third_party/opus, but there's no *.pc file.  I didn't want to touch opus unless necessary.

Anyway, that's all the further I got before I had to move on.

You can look at the code without checking it out at

Comment 55 by, May 27 2016

rtoy - is this a feature that will be implemented at some point? (has a decision been made?)
This needs to be fixed, it breaks the whole flow between MediaRecorder API and WebAudio API. 

Comment 57 by, May 31 2016

tommi: Can't say about a decision, but it is definitely something we want; it is silly that webaudio can't decode files the chrome can.

Comment 58 by, Aug 11 2016

Status: Assigned (was: Available)
rtoy - can you take a look at this?  As is I don't see that any progress is being made and I think someone needs to own this and start working on it.

Comment 59 by, Aug 22 2016

This blocks us supporting Opus in Construct 2.
I think this will be useful for people working in webvr with MediaRecorder API and WebAudio API... because of positional audio. It would be great for it to work. 
i am looking forward to a resolution for this issue.
Unless I'm missing something, this blocks audio recordings from MediaRecorder being usable in OfflineAudioContext, because you need to use BufferSource nodes.

Mixing down and/or merging recordings can't be done until this is fixed, the createMediaElementAudioSource workaround won't cut it.
What is the status on this? I am really looking forward to a resolution on this issue too. I spent quite a lot of time into a project only to find that WebAudio can't decode what MediaRecorder does.
Components: Internals>Media>FFmpeg
Labels: M-55
On a whim picked this up, have this working locally by switching to ffmpeg's opus decoder. It's as good as our built-in one these days, so no need to maintain ours anymore.

Comment 65 by, Oct 21 2016

Oh, wow. I also had it working locally, but not on all platforms and all slightly different.  

From WebAudio's viewpoint, having everything go through ffmpeg is a very nice.
by ffmpeg's opus decoder I actually just mean it's libopus shim, so we're still using libopus.

Comment 67 by, Oct 21 2016

Yeah, that's what I did too.
i am fine with switching over to ffmpeg's shim instead of having to maintain ours.

There is some documentation work for Windows, Mac I need to do so that includes work across platform, but is otherwise the gist of the idea.
The things which make using libopus reasonable is we can just use the system installed packages for Cygwin, Linux variants. Will test out OSX tomorrow, but it seems we don't need to maintain more build instructions for libopus for this to work so it's not so bad after all.
Actually, with some include path work for build_ffmpeg, libopus can be used directly from Chromium source tree, so even better.
Project Member

Comment 72 by, Oct 21 2016

The following revision refers to this bug:

commit b4d337e66827f3cced2a5762c428035703412aed
Author: Dale Curtis <>
Date: Fri Oct 21 22:49:44 2016

Enable libopus builds within

This prepares the path for using ffmpeg's opus decoder instead of
our own. Allowing opus playback in WebAudio and reducing some of
our code complexity.

BUG= 482934 
TEST=Opus media_unittests pass w/o modification.

Change-Id: I8867b015ac9e8bd8ec3b0ae04b926f87cb2aae09
Reviewed-by: Matthew Wolenetz <>


Project Member

Comment 73 by, Nov 20 2016

The following revision refers to this bug:

commit 655b0cf987eca28d8035cb3ceb1af6c4b916b179
Author: dalecurtis <>
Date: Sun Nov 20 00:00:13 2016

Use ffmpeg for opus decoding, no need to maintain our decoder.

As a bonus this allows WebAudio to start using opus.

BUG= 482934 
TEST=all opus tests pass with the previous expectations.

Cr-Commit-Position: refs/heads/master@{#433433}


Comment 74 by, Nov 28 2016

This is totally awesome!  I played very briefly with ToT chromium (on linux) with opus decoding and it works great.
Labels: -M-55 M-57
Should have flac now too :) It looks like this may not make it until M57 though, we've got a few issues with the ffmpeg roll this depends on that are needing some soak time.
Labels: -M-57 M-56
Status: Fixed (was: Assigned)
Actually ffmpeg roll was merged to M56, so this should work there.
Labels: -M-56 M-57 Merge-Request-56
@#76, though the roll is merged to M56 (as of last night - see  issue 591845 ), #73 hasn't been merged to M56. I believe such needs to be requested to make your comment "so this should work there" in #76 true.

Requesting merge of #73 to M56. (dalecurtis@, please confirm.)

Comment 78 by, Dec 1 2016

Labels: -Merge-Request-56 Merge-Approved-56 Hotlist-Merge-Approved
Your change meets the bar and is auto-approved for M56 (branch: 2924)
Project Member

Comment 79 by, Dec 1 2016

Labels: -merge-approved-56 merge-merged-2924
The following revision refers to this bug:

commit a76943e15311ba010af0dec0df9b6c79e5532989
Author: Matt Wolenetz <>
Date: Thu Dec 01 20:08:40 2016

To M56: Use ffmpeg for opus decoding, no need to maintain our decoder.

As a bonus this allows WebAudio to start using opus.

BUG= 482934 
TEST=all opus tests pass with the previous expectations.

Cr-Commit-Position: refs/heads/master@{#433433}
(cherry picked from commit 655b0cf987eca28d8035cb3ceb1af6c4b916b179),,

Review URL: .

Cr-Commit-Position: refs/branch-heads/2924@{#259}
Cr-Branched-From: 3a87aecc31cd1ffe751dd72c04e5a96a1fc8108a-refs/heads/master@{#433059}


Labels: M-56
(@#77 - I confirmed with dalecurtis@ offline about the merge of #73 to M56, which #79 shows is done now.)

Sign in to add a comment