New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 673782 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug

Blocking:
issue 409402



Sign in to add a comment

decodeAudioData is unable to decode audio of a dataUrl generated from a recording with MediaRecorder

Reported by sambel...@gmail.com, Dec 13 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36

Steps to reproduce the problem:
1. Get the user's microphone input with getUserMedia
2. Make a recording of this stream using a MediaRecorder
3. Generate a data-url from the recording
4. Fetch that data-url as an arrayBuffer
5. Decode that audio using audioContext.decodeAudioData

What is the expected behavior?
It should create an AudioBuffer

What went wrong?
It throws an error: error DOMException: Unable to decode audio data

Did this work before? No 

Does this work in other browsers? Yes

Chrome version: 54.0.2840.98  Channel: stable
OS Version: OS X 10.11.6
Flash Version: Shockwave Flash 24.0 r0

I created a jsfiddle demonstrating the problem: https://jsfiddle.net/Sambego/op79c5xb/7/

Running this on Chrome it throws an error, but running this in firefox just returns the AudioBuffer as expected.
 

Comment 1 by ajha@chromium.org, Dec 15 2016

Labels: Needs-Milestone
Labels: NeedsTriage-M56
Cc: krajshree@chromium.org
Labels: Needs-Feedback
Unable to reproduce the issue in Mac 10.12.2 by using chrome reported version #54.0.2840.98 and latest canary #57.0.2952.0.

Steps followed to reproduce the issue are as follows:
-----------
1. Navigated to the URL: https://jsfiddle.net/Sambego/op79c5xb/7/
2. Observed that an AudioBuffer was created without any issues.

Attaching screen cast for reference.

sambellen@ - Could you please check this issue on latest chrome stable #55.0.2883.87 by creating a new profile without any apps and extensions and please let us know if the issue still persist or not.
The latest stable can be downloaded from the following link:
https://www.chromium.org/getting-involved/dev-channel

Thanks...!!
673782.mp4
1.4 MB View Download

Comment 4 by sambel...@gmail.com, Dec 16 2016

I tested it in both the latest stable chrome (55.0.2883.95 (64-bit)) and the latest canary (57.0.2953.0 canary (64-bit)), but the error seems to be persist in both of them.

I've attached a screenshot of the console output of the jsfiddle.

It might be worth mentioning that this error is also present when changing to other audio codec's, but when trying to load an external .ogg file, it works as expected.
I tested with this file 'https://upload.wikimedia.org/wikipedia/en/4/45/ACDC_-_Back_In_Black-sample.ogg`.
Screen Shot 2016-12-16 at 09.50.08.png
134 KB View Download
Screen Shot 2016-12-16 at 09.48.13.png
81.5 KB View Download
Cc: mcasas@chromium.org rtoy@chromium.org
Components: -Blink>Media Blink>MediaStream>Recording Blink>WebAudio

Comment 6 by rtoy@chromium.org, Dec 16 2016

Status: Untriaged (was: Unconfirmed)
I can confirm the issue using ToT chromium.  I thought MediaRecorder only supported Opus files, and webaudio couldn't decode Opus-encoded files until just a few weeks ago.  Nevertheless, ToT chromium can support Opus-encoded files.

Comment 7 by hdodda@chromium.org, Dec 19 2016

Labels: -NeedsTriage-M56
Tested on Mac OS 10.12.1 using latest chrome M57 #57.0.2956.0 and able to create the audio buffer.

Attached screencast for reference.

Thanks!
673782.mp4
320 KB View Download

Comment 8 by sambel...@gmail.com, Dec 19 2016

Maybe the audio component in the jsfiddle is a bit confusing, but it is to demonstrate that chrome can handle the opus codec, at least when using it with an audio element.

According to this issue, which got merged earlier this month the decodeAudioData method should be able to handle opus: https://bugs.chromium.org/p/chromium/issues/detail?id=482934

The audio recorded by the MediaRecorder is playable by the audio component when converted to a blob data url.

However, when you want to convert that blob to an audio-buffer using audioContext.decodeAudioData(buffer) it fails.

This part of the code fetches the blob data url and converts it into an array-buffer. That works fine, but the next step is to convert that array-buffer into an audio-buffer, and that is where chrome fails.

The audioContext.decodeAudioData method accepts an array-buffer, but for some reason can not handle the array-buffer created from the blog of the recording created earlier.

// Fetch the blob's data-url
fetch(blobDataUrl).then(response => {
    // Works well, the data-url gets fetched
    return response.arrayBuffer();
}).then(buffer => {
    // Converting the data-url to an array-buffer works as expected
    return audioContext.decodeAudioData(buffer);
}).then(data => {
    // we expect an audio-buffer here, but chrome fails to convert the array-buffer
    // to an audio buffer, so we end op in the error callback of the promise.
}, error => {
    // Chrome throws this error:
    // error DOMException: Unable to decode audio data
});

You can see the output of each step when opening the console in the developer tools.

I hope this explains the issue more in detail.

Comment 9 by rtoy@chromium.org, Dec 19 2016

Yes, I was seeing exactly that. I don't understand why the array buffer doesn't work.  Will need to examine the contents of the buffer to see why ffmpeg doesn't like it.
rtoy@, are you the right owner for this issue?

Comment 11 by rtoy@chromium.org, Jan 3 2017

Status: Available (was: Untriaged)
Someone on the WebAudio team will look into this. Possibly me, but not necessarily.
Could be the same root cause as  https://crbug.com/606000 .

sambellen@, just to be sure, downloading the whole
recording can be correctly played in either Chrome or an
external player, e.g. vlc?
mcasas@ Yep the recording can be played in both Chrome and VLC.

You can find a sample of a recording here: https://www.dropbox.com/s/4gp2b8cdxyrjzdy/myrecording.mp4?dl=0
Components: Blink>MediaRecording
Components: -Blink>MediaStream>Recording
Bulk move
Blink>MediaStream>Recording ---> Blink>MediaRecording
ping rtoy@, sambellen@ any updates here?

There's been a few actions and a major update to libwebm,
so perhaps anyone could try again?
mcasas@, does not seem to be fixed.
Tested on both the latest stable Chrome version and canary.

Comment 18 by rtoy@chromium.org, Jan 20 2017

Owner: rtoy@chromium.org
Status: Assigned (was: Available)

Comment 20 by rtoy@chromium.org, Jan 24 2017

Cc: dalecur...@chromium.org
dalecurtis:  Since MediaRecorder saves files in webm format, this means WebAudio needs ffmpeg to support webm, right?  As best as I can tell, Chrome's ffmpeg doesn't.
AudioFileReader definitely supports webm; you'll have to add some logging to figure out why it's not working. I'd guess there's a parse error or we're not handling data:// URLs properly in AudioFileReader.

Comment 22 by rtoy@chromium.org, Jan 24 2017

Thanks Dale.  It's failing because the duration is not specified (at least for the test file I'm using).  Then AudioFileReader::Open() returns false indicating that we can't read the file.  

I guess that means we'll need to modify DecodeAudioFileData to be able to read the data into an internal bus that grows as needed and copy it to the destination bus.

Does that sound right?
Correct, you'd need to setup a method for streaming decode.
Blocking: 409402
Project Member

Comment 25 by bugdroid1@chromium.org, Feb 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3489b2812bf4e670cc27627ee9a7a1fd1919f399

commit 3489b2812bf4e670cc27627ee9a7a1fd1919f399
Author: rtoy <rtoy@chromium.org>
Date: Fri Feb 10 18:15:55 2017

Decode entire in-memory file for WebAudio

DecodeAudioFileData assumed that any in-memory file to be decoded had
a known duration that could be estimated without reading the entire
file.  For webm files produced by MediaRecorder, this can be false.

Read and decode the entire file, ignoring any estimated duration.  This
will result in increased memory consumption because the final output
is built up in pieces before finally copying it to the final destination.

Add tests for decoding webm files and WAV files using float32 samples.
Vorbis tests need to be updated because the resulting outputs were
previously truncated.

BUG= 673782 
TEST=codec-tests/webm/webm-decode.html, codec-tests/wav/f32-44khz.html

Review-Url: https://codereview.chromium.org/2655783004
Cr-Commit-Position: refs/heads/master@{#449662}

[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/content/renderer/media/audio_decoder.cc
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/media/filters/audio_file_reader.cc
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/media/filters/audio_file_reader.h
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/media/filters/audio_file_reader_unittest.cc
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/vorbis/vbr-128kbps-44khz-expected.wav
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/vorbis/vbr-70kbps-44khz-expected.wav
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/vorbis/vbr-96kbps-44khz-expected.wav
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/wav/f32-44khz-expected.wav
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/wav/f32-44khz.html
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/webm/test-webm.webm
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/webm/webm-decode-expected.wav
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/codec-tests/webm/webm-decode.html
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/decodeAudioData/decode-audio-data-basic-expected.txt
[modify] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/decodeAudioData/decode-audio-data-basic.html
[add] https://crrev.com/3489b2812bf4e670cc27627ee9a7a1fd1919f399/third_party/WebKit/LayoutTests/webaudio/resources/media/f32-44khz.wav

rtoy@/sambellen@ did #25 solve the problem? Can we mark this as Fixed?

I.e. is https://jsfiddle.net/Sambego/op79c5xb/7/ working fine now
with the latest Canary? -- seems to work fine to me.

Comment 27 by rtoy@chromium.org, Feb 13 2017

Yes, #25 was intended to solve this problem (and some others).
Status: Fixed (was: Assigned)
Hey all,

Has the fix hit version 57? I'm still encountering this issue on Windows and Ubuntu when fetching the object URL of a Blob, captured via the MediaRecorder API, as an ArrayBuffer.

https://github.com/learnable-content/jamesseanwright/blob/web-audio/webrtc/complete/index.js

Thanks,
James
Labels: M-58
#29 - Chrome 58 has the fix.

Sign in to add a comment