Possibly related to this? https://bugs.chromium.org/p/chromium/issues/detail?id=606000&can=2&start=0&num=100&q=mediasource%20playback&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&groupby=&sort=

If you need a test case, I'll try to build one.

Just to be clear, SourceBuffer is in sequence mode. That is supposed to avoid issues regarding timestamps.
I'm also unable to create a test case. I cannot request video chunks hosted on dropbox and/or use MSE from file:/// protocol... getting this error:

Not allowed to load local resource: blob:null/5da1d258-7212-49d4-9753-29702f6beb3e

:(

Well, just uploaded a test case, but surprisinly Firefox plays no sound...

http://teamcapture.herokuapp.com/mse-test.html

Can someone please confirm if video chunks need a specific format or just the first one  appended to the buffer needs the proper headers and the rest can be just simple byte containers?
I'm pretty sure I'm not doing anything wrong. 
I just want to record screen using MediaRecorder and stream it at the same time, so I upload 3-second webm video chunks to S3 and clients play video using MSE based player that requests chunks one at a time, appending them to the source buffer.

Thanks for your help.

Finally, I can confirm this bizarre bug! ... This only happens when recording video on Mac.

What works:
- recording on Linux and playing on Linux using Chrome/Firefox
- recording on Linux and playing on Mac using Chrome/Firefox

What does not work:
- recording on Mac and playing on Mac using Chrome/Firefox
- recording on Mac and playing on Linux using Chrome/Firefox

Must be something related to webm chunk structure that MediaRecorder generates.

Please update issue title accordingly.

Thanks.

Able to reproduce this issue on Mac OS 10.12 using chrome latest stable M55-55.0.2883.87. By opening the  test case link http://teamcapture.herokuapp.com/mse-test.html, Observed the video is not playing smooth as expected.

Tested this issue on chrome #35.0.1849.0, #45.0.2431.0 and #50.0.2628.0 and observed the video is not even playing in earlier versions, so confirming this is a non-regression issue and marking it as untriaged.

Sorry, what does that mean? Do I need to wait until version 57 for the fix?
I appreciate your help, but that's totally unacceptable for my case since
I'm building a desktop app using Electron currently based on Chrome 53.
That means I'll have to wait for at least 6 months for the fix...
Is there anything I can do to speed this up? Any workaround?
Please help.

If you disable accelerated video decode in chrome://flags does the issue go away?

Does not go away. I'm getting this error in chrome:///media-internals while playing the example above:

Failed to reconcile encoded audio times with decoded output.

Sounds like the media has invalid timestamps then.

Right... But why should the source buffer care about chunk timestamp if
it's working in sequence mode? Isn't that the purpose of that property?
How can I fix those timestamps then? Am I doing something wrong or is it a
bug then? Thanks

Investigating...

Can you link me to the original (un-chunked) recording?

This mostly looks like a duplicate of  Issue 606000 , but the audio timestamps are even more messed up here than in that original issue. Lots of same-timestamps on consecutive buffers (none of which have zero duration). For instance, the first couple of audio timestamps are: 0.161, 0.161, 0.354, 
0.354... very wrong.

The video durations are messed up in the way we expect from  Issue 606000  - webm default duration does not match the actual delta between video frame presentation times. 

Sequence mode is sadly not enough to smooth over this issue. Sequence mode is designed to smooth over gaps in content that otherwise has correct timestamps and durations (imagine you have delete a few minutes from the middle of an otherwise correct file). It still relies on the timestamp/duration metadata on the audio/video frames, and in your case these value are pretty unreliable. What happens to cause the freezing is it detects a gap (due to the bad duration) between the first 2 video frames. It then drops all frames until the next key frame. Then it drops everything after that second key frame (again due to bad gap detection) - so you end up with a 2 frame video (there are only 2 key frames in your demo).

The files timestamps/durations are way bad - not sure any workaround is bullet proof. One thing you might try is editing the webm file in javascript to remove the default duration MediaRecorder is setting for the video track. Then play around again with segments vs sequence mode. 

There is no original recording, I'm generating 3-second webm files straight from the 'dataavailable' event the MediaRecorder dispatches, so there is no single file at all.
I want to stream video while it's being recorded and that's the only way I came up with.
Only the first chunk contains initialization headers and the rest is just supposed to be a container for a chunk of bytes. How can I edit those chunks and remove timestamps?

Key points:
- I'm recording inside an Electron app, based on Chromium 53, doesn't play inside the app either, so bug still exists.
- I'm capturing just video from entire screen or window and then attaching the audio source from a different MediaStream using .addTrack(), maybe that's interfering somehow?

Thanks.

Running sample_muxer -i 1483550250645_0.webm -o 1483550250645_0.webm throws...

 Segment::Load() failed

Not sure the purpose of that tool either, just took the idea from this article: http://axel.isouard.fr/blog/2016/05/24/streaming-webm-video-over-html5-with-media-source

Over to mcasas@. This might be a dup of  Issue 606000 , but the duplicate audio timestamps is new to me. 

> I want to stream video while it's being recorded and that's the only way I came up with.

AFAICT your general approach is correct, you're just hitting a bad bug in media recorder.

> I'm capturing just video from entire screen or window and then attaching the audio source from a different MediaStream using .addTrack(), maybe that's interfering somehow?

mcasas@ - is this the best way to do it? I'm not very familiar with MediaRecorder.

> How can I edit those chunks and remove timestamps?

Its non-trivial. This will likely fix your video troubles, but your audio timestamps are still messed up, so I can't guarantee it will fix everything. 

You'll need to partially parse the webm container init segment in search of the video track DefaultDuration, then clear that value before appending to the SourceBuffer. In this MSE demo [0] you can see an example of parsing mp3 container metadata using Uint8Array, ArrayBuffer, slice, etc. You can use similar tools to do the same for webm (the layout is very different though). Here are some links on webm container layout: [1] [2].

[0] http://dalecurtis.github.io/llama-demo/index.html#parsing-gapless-metadata
[1] https://www.matroska.org/technical/diagram/index.html
[2] https://www.matroska.org/technical/specs/index.html (search for DefaultDuration)

Before marking it as a duplicate of  Issue 606000 , a couple of comments/
questions.

Recording a video stream from the screen/a window/a tab and then an audio
source is in itself correct; the generated timestamps of the webm stream
(file) will be generated after compression [1], so they should generate a
monotonously increasing sequence.

Do I understand correctly that you're generating a different recording
_file_ every 3 seconds? I.e. creating a new MediaRecorder and start()ing
it again? That might very well confuse the MediaSource since every new
"file" will have a webm header and a timestamp train starting from 0.

As usual, if you get a recording a download as file, can you still 
play it back in Chrome/Electron/VLC? That would give a clue as to how
messed up the container is, if at all.

Finally mkclean is a matroska-specific tool that, well, cleans up the
produced stream, and it might be of help now or in the future [2].


[1] https://cs.chromium.org/chromium/src/media/muxers/webm_muxer.cc?dr&q=WebmMuxer::Addframe&sq=package:chromium&l=314
[2] https://www.matroska.org/downloads/mkclean.html

Aside:

I expected that I would be able to cat his 4 separate chunks into a single valid webm file. I'm finding that this is not the case (and also that no chunk seems to be valid by itself either). By valid, I mean something I can pass to ffprobe without getting errors. 

I'm obviously not creating a MediaRecorder instance for every chunk. 

Attaching test file with recording logic below.

Downloading video using downloadVideo() function plays fine in VLC and even in Chrome.
VideoStreamUploader just uploads chunks directly into AWS S3. Those chunks are then used to feed the MediaSource.

Deleted: mediarecorder-test.js 2.0 KB

mediarecorder-test.js 2.0 KB View Download

 Issue 606000  was Fixed by [1], demian85@ could you please
give it another go in either a Chromium build or a Canary?
Thanks


[1] https://chromium.googlesource.com/chromium/src.git/+/50252de90c71b7d5d12c30121a82d4e790f74252

So the fix involves MediaSource or MediaRecorder API? If the recorder is buggy there isn't much I can do for now since I will need to wait for an Electron release based on this Chrome version... that could take months...
By any chance could you patch current stable version?

Just tested on Chrome Canary with attached demo above and playback has the same issues.

mcasas@, I don't think your change has made it in to canary just yet. I'd try again on Monday/Tuesday (not sure if they update that build over the weekend). 

> So the fix involves MediaSource or MediaRecorder API?

MediaRecorder

> By any chance could you patch current stable version?

Sorry, its very unlikely. The bar for patching stable is very high (e.g. urgent security vulnerabilities).

Have you tried the workaround I suggested - removing DefaultDuration via javascript? It will at least fix issues with video-only recordings. :/

Can someone please explain why the fix is not OSX specific? The bug only happens in OSX.
Are there unit tests for this change? Don't get me wrong, but I don't understand why you push changes without proper testing them. Did you just guess that change will fix the issue?
I'm cloning chromium to apply patch in order to build https://github.com/electron/libchromiumcontent for testing, not sure if that's the fastest way. If not, I would have to create a Chrome extension to capture screen.

Well, good news! 
I just made a simple extension html page using desktopcapture chrome API to record a stream while displaying it on a video tag. It works fine!

Attaching demo.

Would be awesome to patch current stable version. Can someone please tell what it takes to achieve that?

Thanks.

Deleted: chrome-recorder.zip 3.2 KB

chrome-recorder.zip 3.2 KB Download

demian85@, I'm afraid patching a stable Chrome so far backwards, as
chunningham@ said, is not feasible. Also, I can't speak for Electron,
but you could possibly patch [1] either locally or in Electron, since
it's quite straightforward and touches a part of the code that has
seen little action. I'm marking this issue as Fixed, please jump to
 Issue 606000  to continue the conversation topic of MR <-> MSE interop.
Thanks!


[1] https://bugs.chromium.org/p/chromium/issues/detail?id=606000#c34

I've tested generating a media recording with https://webrtc.github.io/samples/src/content/getusermedia/record/ and playing back in MSE on Mac 55.0.2883.95 - no issues. 

I manually verified the audio timestamps from the recording - looks good. 

demian85@, if you suspect a MediaRecorder issue still exists please link to a demo demonstrating the recording + playback combined (e.g. something like the github link above - the original repro at http://teamcapture.herokuapp.com/mse-test.html is only the playback half, so its difficult to verify your MR issue is resolved).

I see it's still unclear for you that the issue involves just screen capturing and not video capturing. Please see attached chrom extension above to test properly. There is no way to publish demo as it needs to be an extension to allow screen capture!

Sorry, I confused that detail. But also, you mention that your extension in comment 27 is working fine. Do you have a demo (including the recording half) that is *not* working?

I'm busy trying to compile Electron with this patch applied to continue working on my project. I will be sure when finished.
You can always test before committing changes... :P

I assume this is the commit you're referring to
https://bugs.chromium.org/p/chromium/issues/detail?id=606000#c34

That commit is associated with a different (though related) bug. I did test it with that bug's repro steps. I do not claim it fixes your bug entirely. I cannot know without a repro.

If you provide me a repro, including the recording half (where I believe the error originates), I will be happy to test it and fix any lingering issue.

Just compiled Electron with the patch applied and I can confirm this issue has been solved.

Well, as I said in the other bug report, I cannot confirm if this fix really works.... sometimes I open the app to create a 20-second screen video that can be successfully played in the same app afterwards, but sometimes it doesn't....

bug is still present from my point of view, OS: Ubuntu 14.04.5 LTS; Chrome version: 64.0.3282.186. Looks like buffer fails while appending certain types of GOP frames

Please provide a way to reproduce and we'll take a look. I don't have ubuntu handy, but I confirm the demo I made a while back works well with Chrome 64.0.3282.186 on Mac. Try the link below and let us know.

https://storage.googleapis.com/chcunningham-chrome-shared/mr_to_mse.html

One possibility for any folks continuing to have issues with MediaRecorder + MSE playback using 'sequence' mode MSE appends is that muxed (multi-track bytestreams) sequence mode SourceBuffer processing by MSE is problematic and likely to be deprecated: see bug 737757 and links within it. At least buffered range gaps can occur if the timestamps and durations of the frames in the tracks are not aligned precisely (they very rarely are). 'sequence' mode interoperability is well-defined in the MSE spec and implementations only for single-track bytestreams.

I came here because my MR to MSE node.js web app is failing in the same way the OP mentions, except I'm not in Electron. Chrome shows maybe the 1st second of the stream, then freezes. Manual seeking sometimes shows other keyframes. Sound appears to be more consistent than video. The error on the media-internals page is still "Failed to reconcile encoded audio times with decoded output.". I am on Chrome-OS version Version 69.0.3497.120 (Official Build) (64-bit) on an Acer CB515-1HT-P6W6

Debugging with this app:
https://storage.googleapis.com/chcunningham-chrome-shared/mr_to_mse.html

Shows the following output in the console at the point where it begins to freeze:
new recorder bytes. array size now at:1
mr_to_mse.html:170 appended bytes! 0 buffers of bytes remain
mr_to_mse.html:188 new recorder bytes. array size now at:1
mr_to_mse.html:170 appended bytes! 0 buffers of bytes remain
mr_to_mse.html:101 MSE buffered has a gap!
(anonymous) @ mr_to_mse.html:101
mr_to_mse.html:188 new recorder bytes. array size now at:1
mr_to_mse.html:170 appended bytes! 0 buffers of bytes remain
mr_to_mse.html:101 MSE buffered has a gap!
(anonymous) @ mr_to_mse.html:101
mr_to_mse.html:188 new recorder bytes. array size now at:1
mr_to_mse.html:170 appended bytes! 0 buffers of bytes remain