High memory/CPU usage when trying to load FLAC into MSE
Reported by
vinz...@gmail.com,
Apr 21 2017
|
||||||||
Issue description
UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36
Steps to reproduce the problem:
1. Try to load chunked flac into MSE
const mediaSource = new MediaSource();
mediaSource.addEventListener('sourceopen', () => {
const sourceBuffer = mediaSource.addSourceBuffer('audio/mpeg');
const onAudioLoaded = (data, index) => {
console.log('chunjreceived', index);
this.buffered = this.buffered + 10.0;
const gaplessMetadata = {
audioDuration: 10.0,
frontPaddingDuration: 0.025057
};
const buffered = sourceBuffer.buffered;
const appendTime = index > 0 ? (buffered.length ? buffered.end(0) : 10) : 0;
sourceBuffer.appendWindowEnd = appendTime + gaplessMetadata.audioDuration - 0.02;
sourceBuffer.appendWindowStart = appendTime;
sourceBuffer.timestampOffset = appendTime - gaplessMetadata.frontPaddingDuration;
if (index === 0) {
sourceBuffer.addEventListener('updateend', () => {
console.log('updateend');
if (++index < this.props.source.chunks) {
console.log('addchunk', index);
this.getChunk(index).then(function(data) { onAudioLoaded(data, index); } );
} else {
// We've loaded all available segments, so tell MediaSource there are no
// more buffers which will be appended.
mediaSource.endOfStream();
URL.revokeObjectURL(this.audio.src);
}
});
}
sourceBuffer.appendBuffer(data);
}
this.getChunk(0).then(function(data) { onAudioLoaded(data, 0); } );
What is the expected behavior?
Load quickly each flac and then make audio playable
What went wrong?
When trying to create the MediaSource, the resources get fetched at a lower rate than with mp3, tab uses 50% of CPU (4 cores in a VM) and memory increases relatively quickly (I got 500MB for the tab, 170MB before). Audio never plays (or I never got there ^)
Did this work before? N/A
Does this work in other browsers? N/A
Chrome version: 57.0.2987.98 Channel: stable
OS Version: ArchLinux
Flash Version:
The audio is just a flac file splitted in 10s chunks.
,
Apr 24 2017
,
Apr 25 2017
vinz243@ could you please provide a sample Html test file to triage the issue from TE end. Thanks.
,
Apr 25 2017
Hi, This needs a backend as well, I'll try to provide a minimal working example as soon as possible. In the meantime feel free to check the full source https://github.com/vinz243/cassette/blob/6801f4cb202fbf2c85da593f45e57870018771a8/src/client/assets/javascripts/features/toolbar/components/AudioPlayer/AudioPlayer.js
,
Apr 25 2017
Thank you for providing more feedback. Adding requester "sureshkumari@chromium.org" to the cc list and removing "Needs-Feedback" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 26 2017
Sorry for the delay, here you can find the requeste MWE https://gist.github.com/vinz243/cfac087a7ddbc3debd9f87ff9b213730 Commit 3b0fce0 shows a working example with mp3 playback. I edited in 2a3df98 to show the issue
,
Apr 26 2017
Please note that you will need a 1m20 valid audio.flac (not necessarily flac, just the name) along with the files, ffmpeg/ffprobe installed and node_modules installed
,
Apr 27 2017
,
Feb 15 2018
,
Feb 15 2018
,
Feb 15 2018
I'll take a look today.
,
Feb 15 2018
FWIW before I investigate: FLAC is lossless, so is likely to consume more memory for the encoded/compressed audio samples than lossy codecs like mp3.
,
Feb 15 2018
Ah -0 from looking at #6 (https://gist.github.com/vinz243/cfac087a7ddbc3debd9f87ff9b213730), it appears (without digging deeper) that you might be providing a raw FLAC stream to the web client to feed to MSE. Raw audio/flac is not supported with MSE (though it is with non-MSE <audio src='audiofile.flac'>). Rather, you'll need to mux FLAC into ISOBMFF for it to be usable in Chrome via MSE. Please reactivate if that's not the problem you're encountering and provide the contents of the player's log from chrome://media-internals
,
Feb 15 2018
Also, this line of the original report is incorrect if you want to try FLAC-in-ISOBMFF in Chrome MSE:
const sourceBuffer = mediaSource.addSourceBuffer('audio/mpeg');
Instead, use 'audio/mp4; codecs="flac"' and mux the stream accordingly before appending it to the SourceBuffer.
,
Feb 15 2018
And on even further thought, it's also likely the increased CPU was due to the parser trying to find recognizable audio/mpeg container parts in the incoming audio/flac stream, before eventually failing. |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by vinz...@gmail.com
, Apr 21 2017