MediaRecorder stream&codec variety exceeds that supported by MSE
Reported by
oleg.tlv...@gmail.com,
Aug 9 2017
|
||||||
Issue descriptionUserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36 Steps to reproduce the problem: 1. Create Mediarecorder and record Screen video. 2. Create MediaSource, video element,add SourceBuffer, add mediasource to video element source. 3. Read chunks (webm blobs) recorded by Mediarecorder by filereader and append data to sourceBuffer. What is the expected behavior? Video element attached to DOM need to play video. What went wrong? Uncaught DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source. Did this work before? No Does this work in other browsers? Yes Chrome version: 60.0.3112.90 Channel: stable OS Version: 10.0 Flash Version: Shockwave Flash 26.0 r0 I attached JS file with example to better understanding of problem. The MediaSource API works well with webm files but always fail on webm recorded by MediaRecorder. This scenario works well in Firefox.
,
Aug 11 2017
,
Aug 11 2017
Too bad you did not provide a full example.
This is my experimental code, including some modification of yours -
var mediaSource = mediaSource || null, sourceBuffers = sourceBuffers || [];
function PlayVideo(chunks) {
mediaSource = new MediaSource();
var player = document.createElement('video');
player.width = 300;
player.height = 200;
player.autoPlay = false;
player.controls = true;
player.src = URL.createObjectURL(mediaSource);
var previewContainer = document.body;
previewContainer.appendChild(player);
var once = false
mediaSource.addEventListener('sourceopen', function(e) {
console.log('Got here!', mediaSource.sourceBuffers);
if (once) { return; }
once = true;
var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vp8"');
sourceBuffers.push(sourceBuffer);
function readCunks(index) {
index = index || 0;
var fileReader = new FileReader();
fileReader.onload = function(e) {
console.log(mediaSource, mediaSource.sourceBuffers);
sourceBuffer.appendBuffer(e.target.result);
// This row is throw exception on second chunk
if (player.paused) {
player.play();
}
if (index < chunks.length) {
readCunks(++index);
} else {
mediaSource.endOfStream();
}
};
if (index < chunks.length) fileReader.readAsArrayBuffer(chunks[index]);
}
readCunks(0);
}, false);
}
var recorder = recorder || null, chunks = chunks || [];
function record(stream) {
media = new MediaRecorder(stream);
media.ondataavailable = event=>chunks.length?null:chunks.push(event.data);
media.start(1000);
setTimeout(()=>{
console.log('Done recording');
media.stop();
PlayVideo(chunks);
},
3000);
}
function getAndRecordMedia() {
navigator.mediaDevices.getUserMedia({
video: true,
audio: true
}).then(stream=>{
record(stream)
});
}
if (chunks.length) PlayVideo(chunks); else getAndRecordMedia();
It fails on the first chunk when calling play() due to no supported sources, but I may have done something wrong.
,
Aug 11 2017
phistuck@gmail.com i made some modifications on your example with mediaRecorder. It works well on Firefox and throw same error as i described in bug. Modifyed code attached to this comment.
,
Aug 11 2017
I haven't looked deeply yet, but I suspect you'll need to provide the audio codec in addSourceBuffer's parameter, too. Mozilla doesn't require that (and the MSE spec is loose on this point); Chrome requires specific mime types in addSourceBuffer(). That is, unless your MediaRecorder doesn't include any audio track. If that's the case, let me know and I'll take a closer look.
,
Aug 11 2017
Yep -- on Chrome linux (60.0.3112.90 (Official Build) (64-bit)) chrome://media-internals shows the following when using the js in #5:
"Audio stream codec opus doesn't match SourceBuffer codecs."
I then achieved successful record+playback by adjusting #5's addsourceBuffer call to instead be:
var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vp8,opus"');
Please reactivate if this doesn't fix the problem for you.
,
Aug 11 2017
wolenetz@chromium.org - i updated code with audio codec in addSourceBuffer: var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"'). Same error Added updated script.
,
Aug 11 2017
'opus' also throw same error...
,
Aug 11 2017
What does chrome://media-internals show for the player that gets instantiated but then fails to play? Are you certain it's using the 'vorbis' audio codec and not opus or something else?
On linux, I again achieved success with:
var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="opus,vp8"')
(Note, not vorbis).
,
Aug 11 2017
Added screen for chrome://media-internals result. I used both of them vorbis and opus, same error. The bug is on Windows. My machine running Windows 10 64.
,
Aug 11 2017
Note, I think you can specify the desired mime-type+codecs in the recorder, like so:
new MediaRecorder(stream, {mimeType : RECORDER_MIME_TYPE})
And make RECORDER_MIME_TYPE match what you use in addSourceBuffer, later.
,
Aug 11 2017
@#11, in non Canary/Dev/Beta Chrome versions, there's a slightly older version of chrome://media-internals. Please open chrome://media-internals on a different tab/window before trying your example. And before closing the tab containing your example, look for the "blob" url corresponding to the player that was instantiated by your example, click on it, and then inspect/share the log that prints out below.
,
Aug 11 2017
,
Aug 11 2017
The log from media-internals was from modified code: var media = new MediaRecorder(stream, {mimeType: 'video/webm'});
,
Aug 11 2017
FYI - alse mediaElement's error attribute, if not null, has a "message" attribute which could help your app/service understand the errors. MSE doesn't support V_MPEG4/ISO/AVC codec in the webm container. Chrome follows more strictly the set of webm mime-type parameters in the MSE webm bytestream spec (https://w3c.github.io/media-source/webm-byte-stream-format.html). Options are: 1) advocate for adding such support (file an MSE spec (and perhaps also a WebM spec) issue and a crbug), 2) using a different camera which produces a codec that Chrome MSE does support within webm container (vp8 or vp9 for video, vorbis or opus for audio), 3) advocate for adding video/mp4 MediaRecorder support (I'm less certain if this has already been requested -- MediaRecording folks, please chime in here) 4) or using JS to transmux the webm container into fragmented MP4 for use with Chrome MSE via 'video/mp4; codecs="..."' Reactivating for discussion. Retitling, too.
,
Aug 11 2017
,
Aug 15 2017
as per c#16, it seems this is expected behavior. wolenetz@, can I resolve it as won't fix?
,
Aug 15 2017
Please leave this open for discussion, per #16 ("Reactivating for discussion. Retitling, too.")
,
Sep 1 2017
MediaRecorder will choose a video codec that is likely accelerated if
left unconstrained; in Windows this is possibly H264, which forces it
to use a matroska container, hence the peculiar (V_MPEG4/ISO/AVC) codec
specification that throws MSE off.
oleg.tlv.il@, you can force the codec in the options to the MR constructor
e.g. theRecorder = new MediaRecorder(stream, {mimeType : "video/webm;codecs=vp8"});
which will in turn allow creation of a WebM container that will probably
be more to the liking of MSE.
Also FTR MR in Chrome only supports Opus as audio codec.
,
Sep 4 2017
mcasas@chromium.org thanks. Works fine if i force video and audio codecs in sourceBuffer and MediaRecorder. Here is working example: https://talya.tegrity.com/qar/oleg/test.html |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by hdodda@chromium.org
, Aug 11 2017Labels: TE-NeedsTriageHelp