New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 713963 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

High memory/CPU usage when trying to load FLAC into MSE

Reported by vinz...@gmail.com, Apr 21 2017

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36

Steps to reproduce the problem:
1. Try to load chunked flac into MSE

const mediaSource = new MediaSource();
mediaSource.addEventListener('sourceopen', () => {
  const sourceBuffer = mediaSource.addSourceBuffer('audio/mpeg');

  const onAudioLoaded = (data, index) => {
    console.log('chunjreceived', index);
    this.buffered = this.buffered + 10.0;
    const gaplessMetadata = {
      audioDuration: 10.0,
      frontPaddingDuration: 0.025057
    };
    const buffered = sourceBuffer.buffered;
    const appendTime = index > 0 ? (buffered.length ? buffered.end(0) : 10) : 0;

    sourceBuffer.appendWindowEnd = appendTime + gaplessMetadata.audioDuration - 0.02;
    sourceBuffer.appendWindowStart = appendTime;
    sourceBuffer.timestampOffset = appendTime - gaplessMetadata.frontPaddingDuration;

    if (index === 0) {
      sourceBuffer.addEventListener('updateend', () => {
        console.log('updateend');
        if (++index < this.props.source.chunks) {
          console.log('addchunk', index);
          this.getChunk(index).then(function(data) { onAudioLoaded(data, index); } );
        } else {
          // We've loaded all available segments, so tell MediaSource there are no
          // more buffers which will be appended.
          mediaSource.endOfStream();
          URL.revokeObjectURL(this.audio.src);
        }
      });
    }

    sourceBuffer.appendBuffer(data);
  }

  this.getChunk(0).then(function(data) { onAudioLoaded(data, 0); } );

What is the expected behavior?
Load quickly each flac and then make audio playable

What went wrong?
When trying to create the MediaSource, the resources get fetched at a lower rate than with mp3, tab uses 50% of CPU (4 cores in a VM) and memory increases relatively quickly (I got 500MB for the tab, 170MB before). Audio never plays (or I never got there ^)

Did this work before? N/A 

Does this work in other browsers? N/A

Chrome version: 57.0.2987.98  Channel: stable
OS Version: ArchLinux
Flash Version: 

The audio is just a flac file splitted in 10s chunks.
 

Comment 1 by vinz...@gmail.com, Apr 21 2017

OK, I tried with a shorter file and once it loaded everything, it said that the audio has no supported source, even though I can open each chunk in a new window
Labels: Needs-Milestone
Cc: sureshkumari@chromium.org
Labels: Needs-Feedback
vinz243@ could you please provide a sample Html test file to triage the issue from TE end.

Thanks.

Comment 4 by vinz...@gmail.com, Apr 25 2017

Hi,

This needs a backend as well, I'll try to provide a minimal working example as soon as possible. 
In the meantime feel free to check the full source https://github.com/vinz243/cassette/blob/6801f4cb202fbf2c85da593f45e57870018771a8/src/client/assets/javascripts/features/toolbar/components/AudioPlayer/AudioPlayer.js
Project Member

Comment 5 by sheriffbot@chromium.org, Apr 25 2017

Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "sureshkumari@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 6 by vinz...@gmail.com, Apr 26 2017

Sorry for the delay,
here you can find the requeste MWE https://gist.github.com/vinz243/cfac087a7ddbc3debd9f87ff9b213730

Commit 3b0fce0 shows a working example with mp3 playback.
I edited in 2a3df98 to show the issue

Comment 7 by vinz...@gmail.com, Apr 26 2017

Please note that you will need a 1m20 valid audio.flac (not necessarily flac, just the name) along with the files, ffmpeg/ffprobe installed and node_modules installed
Labels: -Needs-Milestone TE-NeedsTriageHelp
Components: Blink>MediaStream
Components: -Blink>MediaStream Internals>Media>Source
I'll take a look today.
FWIW before I investigate: FLAC is lossless, so is likely to consume more memory for the encoded/compressed audio samples than lossy codecs like mp3.
Owner: wolenetz@chromium.org
Status: WontFix (was: Unconfirmed)
Ah -0 from looking at #6 (https://gist.github.com/vinz243/cfac087a7ddbc3debd9f87ff9b213730), it appears (without digging deeper) that you might be providing a raw FLAC stream to the web client to feed to MSE. Raw audio/flac is not supported with MSE (though it is with non-MSE <audio src='audiofile.flac'>). Rather, you'll need to mux FLAC into ISOBMFF for it to be usable in Chrome via MSE.

Please reactivate if that's not the problem you're encountering and provide the contents of the player's log from chrome://media-internals
Also, this line of the original report is incorrect if you want to try FLAC-in-ISOBMFF in Chrome MSE:
const sourceBuffer = mediaSource.addSourceBuffer('audio/mpeg');

Instead, use 'audio/mp4; codecs="flac"' and mux the stream accordingly before appending it to the SourceBuffer.
And on even further thought, it's also likely the increased CPU was due to the parser trying to find recognizable audio/mpeg container parts in the incoming audio/flac stream, before eventually failing.

Sign in to add a comment