New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 753707 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

MediaRecorder stream&codec variety exceeds that supported by MSE

Reported by oleg.tlv...@gmail.com, Aug 9 2017

Issue description

UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36

Steps to reproduce the problem:
1. Create Mediarecorder and record Screen video.
2. Create MediaSource, video element,add SourceBuffer, add mediasource to video element source.
3. Read chunks (webm blobs) recorded by Mediarecorder by filereader and append data to sourceBuffer.

What is the expected behavior?
Video element attached to DOM need to play video.

What went wrong?
Uncaught DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.

Did this work before? No 

Does this work in other browsers? Yes

Chrome version: 60.0.3112.90  Channel: stable
OS Version: 10.0
Flash Version: Shockwave Flash 26.0 r0

I attached JS file with example to better understanding of problem. The MediaSource API works well with webm files but always fail on webm recorded by MediaRecorder. This scenario works well in Firefox.
 
example.js
1.3 KB View Download

Comment 1 by hdodda@chromium.org, Aug 11 2017

Cc: hdodda@chromium.org
Labels: TE-NeedsTriageHelp
@TE-NeedsTriageHelp-- Requesting triage help from dev .

Thanks!
Components: Blink>MediaRecording Internals>Media>Source

Comment 3 by phistuck@gmail.com, Aug 11 2017

Too bad you did not provide a full example.
This is my experimental code, including some modification of yours -

var mediaSource = mediaSource || null, sourceBuffers = sourceBuffers || [];
function PlayVideo(chunks) {

  mediaSource = new MediaSource();

  var player = document.createElement('video');
  player.width = 300;
  player.height = 200;
  player.autoPlay = false;
  player.controls = true;

  player.src = URL.createObjectURL(mediaSource);

  var previewContainer = document.body;
  previewContainer.appendChild(player);
  var once = false

  mediaSource.addEventListener('sourceopen', function(e) {
    console.log('Got here!', mediaSource.sourceBuffers);
    if (once) { return; }
    once = true;
    var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vp8"');
    sourceBuffers.push(sourceBuffer);

    function readCunks(index) {
      index = index || 0;
      var fileReader = new FileReader();

      fileReader.onload = function(e) {
        console.log(mediaSource, mediaSource.sourceBuffers);
        sourceBuffer.appendBuffer(e.target.result);
        // This row is throw exception on second chunk

        if (player.paused) {
          player.play();
        }

        if (index < chunks.length) {
          readCunks(++index);
        } else {
          mediaSource.endOfStream();
        }
      };

      if (index < chunks.length) fileReader.readAsArrayBuffer(chunks[index]);
    }

    readCunks(0);
  }, false);
}
var recorder = recorder || null, chunks = chunks || [];
function record(stream) {
  media = new MediaRecorder(stream);
  media.ondataavailable = event=>chunks.length?null:chunks.push(event.data);
  media.start(1000);
  setTimeout(()=>{
    console.log('Done recording');
    media.stop();
    PlayVideo(chunks);
  },
  3000);
}
function getAndRecordMedia() {
  navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true
  }).then(stream=>{
    record(stream)
  });
}

if (chunks.length) PlayVideo(chunks); else getAndRecordMedia();




It fails on the first chunk when calling play() due to no supported sources, but I may have done something wrong.

Comment 4 Deleted

phistuck@gmail.com i made some modifications on your example with mediaRecorder. It works well on Firefox and throw same error as i described in bug. Modifyed code attached to this comment.
example.js
1.8 KB View Download
I haven't looked deeply yet, but I suspect you'll need to provide the audio codec in addSourceBuffer's parameter, too. Mozilla doesn't require that (and the MSE spec is loose on this point); Chrome requires specific mime types in addSourceBuffer().

That is, unless your MediaRecorder doesn't include any audio track. If that's the case, let me know and I'll take a closer look.
Owner: wolenetz@chromium.org
Status: WontFix (was: Unconfirmed)
Yep -- on Chrome linux (60.0.3112.90 (Official Build) (64-bit)) chrome://media-internals shows the following when using the js in #5:

"Audio stream codec opus doesn't match SourceBuffer codecs."

I then achieved successful record+playback by adjusting #5's addsourceBuffer call to instead be:
    var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vp8,opus"');

Please reactivate if this doesn't fix the problem for you.
 wolenetz@chromium.org - i updated code with audio codec in addSourceBuffer: 

var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"'). Same error

Added updated script.
example.js
1.8 KB View Download
'opus' also throw same error...
What does chrome://media-internals show for the player that gets instantiated but then fails to play? Are you certain it's using the 'vorbis' audio codec and not opus or something else?

On linux, I again achieved success with:
var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="opus,vp8"')

(Note, not vorbis).
Added screen for chrome://media-internals result. I used both of them vorbis and opus, same error.

The bug is on Windows. My machine running Windows 10 64.
chrome-media-internals.png
221 KB View Download
Note, I think you can specify the desired mime-type+codecs in the recorder, like so:
new MediaRecorder(stream, {mimeType : RECORDER_MIME_TYPE})

And make RECORDER_MIME_TYPE match what you use in addSourceBuffer, later.

@#11, in non Canary/Dev/Beta Chrome versions, there's a slightly older version of chrome://media-internals. Please open chrome://media-internals on a different tab/window before trying your example. And before closing the tab containing your example, look for the "blob" url corresponding to the player that was instantiated by your example, click on it, and then inspect/share the log that prints out below.
log.png
279 KB View Download
The log from media-internals was from modified code: var media = new MediaRecorder(stream, {mimeType: 'video/webm'});
Cc: wolenetz@chromium.org dalecur...@chromium.org
Owner: ----
Status: Untriaged (was: WontFix)
FYI - alse mediaElement's error attribute, if not null, has a "message" attribute which could help your app/service understand the errors.

MSE doesn't support V_MPEG4/ISO/AVC codec in the webm container. Chrome follows more strictly the set of webm mime-type parameters in the MSE webm bytestream spec (https://w3c.github.io/media-source/webm-byte-stream-format.html).
Options are:
1) advocate for adding such support (file an MSE spec (and perhaps also a WebM spec) issue and a crbug),
2) using a different camera which produces a codec that Chrome MSE does support within webm container (vp8 or vp9 for video, vorbis or opus for audio),
3) advocate for adding video/mp4 MediaRecorder support (I'm less certain if this has already been requested -- MediaRecording folks, please chime in here)
4) or using JS to transmux the webm container into fragmented MP4 for use with Chrome MSE via 'video/mp4; codecs="..."'

Reactivating for discussion. Retitling, too.
Summary: MediaRecorder stream&codec variety exceeds that supported by MSE (was: MediaSource API not working with MediaRecorder blobs)
as per c#16, it seems this is expected behavior. 
wolenetz@, can I resolve it as won't fix?

Please leave this open for discussion, per #16 ("Reactivating for discussion. Retitling, too.")
Owner: mcasas@chromium.org
Status: WontFix (was: Untriaged)
MediaRecorder will choose a video codec that is likely accelerated if
left unconstrained; in Windows this is possibly H264, which forces it
to use a matroska container, hence the peculiar (V_MPEG4/ISO/AVC) codec
specification that throws MSE off.

oleg.tlv.il@, you can force the codec in the options to the MR constructor
e.g. theRecorder = new MediaRecorder(stream, {mimeType : "video/webm;codecs=vp8"}); 
which will in turn allow creation of a WebM container that will probably
be more to the liking of MSE.

Also FTR MR in Chrome only supports Opus as audio codec.
mcasas@chromium.org thanks. Works fine if i force video and audio codecs in sourceBuffer and MediaRecorder. 

Here is working example:
https://talya.tegrity.com/qar/oleg/test.html
buffering_example.png
84.2 KB View Download

Sign in to add a comment