Media Recorder webm stream cluster length too large when encoding in h264
Reported by
m...@willem.net,
Jul 31 2017
|
||||||||||||
Issue description
UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko) Version/10.1.2 Safari/603.3.8
Steps to reproduce the problem:
Record any stream with MediaRecorder using new { mimeType: 'video/webm; codecs="h264, opus"'}); Use any of the common mkv inspection tools such as mkvinfo to check the length of the clusters in the file.
What is the expected behavior?
webm cluster length to be similar or at least reasonable compared to a recording made with vp8 and vp9 as codec
What went wrong?
I've written an electron app that captures a recording of the desktop, and saves it as a webm file locally. It doesn't include a duration, SeekHead or Cues, making the file unseekable. This is understandable and correct, as it was generated as a (potentially infinite) stream.
I've written some javascript that inserts a SeekHead at the beginning of the segment, and a Cues element at the before the clusters, and it inserts a CuePoint for every Cluster it encounters. This works perfectly for lengthy VP8 and VP9 recordings, however when using it with h264 recording it would fail to play back cleanly, especially seeking near the end of a cluster.
Upon investigation, it appears that whenever the video codec is set to h264, the duration between new clusters is about 30 seconds, whereas using vp8 or vp9 it is a much smaller 4 to 5 seconds.
A lot of webm / mkv players appear to operate on full clusters, and they don't like clusters of that length. If I remux the file in order to make the clusters smaller, the files play perfectly. Players I've tested with are VLC and Chrome itself
I am pretty sure that the files I produce are correct and valid, they pass all validators without warnings or errors (other than h264 not being officially part of the webm spec)
Is there a hidden setting or configuration flag I can use to affect the length between new clusters being generated? I've had a quick look through the chromium source for the relevant source, but haven't found anything yet.
I'm resigned to writing a remuxer in javascript too now to overcome this, but I'd thought I'd report it just in case there is an easy solution or other struggle with this
Did this work before? N/A
Does this work in other browsers? N/A
Chrome version: Version 60.0.3112.78 (Official Build) (64-bit) Channel: stable
OS Version: OS X 10.12.6
Flash Version:
,
Jul 31 2017
,
Aug 1 2017
mail@ Thanks for the issue. Could you please provide us any sample URL or test file to triage the issue from test team end. Thanks,
,
Aug 9 2017
,
Aug 9 2017
Sorry, this slipped through the net! Will provide a repro URL asap.
,
Aug 9 2017
Thank you for providing more feedback. Adding requester "kavvaru@chromium.org" to the cc list and removing "Needs-Feedback" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Aug 10 2017
If you open either of these links, it will ask for recording permission, and record for 60 seconds. The first one will record using vp8, the second one using h264. Once recorded, the webm blob will be processed to have the correct duration, a SeekHead element pointing to Info, Track and Cues, and a Cues section where each Cue will point to the start of a cluster. If you record in vp8 (or vp9) the clusters will be 4/5 seconds apart. but if you record in h264, it will be 30 seconds. https://willem.net/webm/test.html?type=recorder&duration=60&codec=vp8 https://willem.net/webm/test.html?type=recorder&duration=60&codec=h264
,
Aug 11 2017
,
Sep 27 2017
Ping for triaging.
,
Sep 28 2017
I can look into this when time permits.
,
Sep 28 2017
Based on looking at the code my theory as for what happens is as follows. The Chromium code in class WebmMuxer [1] does not set any size or duration limits for the clusters. Neither does it decide itself when to create new clusters by calling Segment::ForceNewClusterOnNextFrame(). This means that the decision when to create a new cluster happens inside libwebm (in file mkvmuxer.cc), and it looks like it creates a new cluster when it encounters a video key frame, see [2]. Probably, the h264 encoder simply produces video key frames further apart than the vp8 or vp9 encoders. It seems odd to me that players have issues with clusters being too far apart, but if it helps, we could probably make WebmMuxer configure libwebm to have a maximum cluster duration of 5 seconds. [1] https://cs.chromium.org/chromium/src/media/muxers/webm_muxer.cc?dr=CSs&l=116 [2] https://cs.chromium.org/chromium/src/third_party/libwebm/source/mkvmuxer/mkvmuxer.cc?dr=CSs&l=3789
,
Sep 28 2017
Actually, it looks like 30 seconds is the default cluster duration in libwebm, see [1]. If players don't handle that correctly, that seems to me like an issue with the players. mail@willem.net, could you elaborate what happens when trying to play back such files? emircan@: Do you know anyone who could give some insights on the playback parts? [1] https://cs.chromium.org/chromium/src/third_party/libwebm/source/mkvmuxer/mkvmuxer.h?l=1863
,
Sep 28 2017
cc'ed some webm folks for their input.
,
Sep 28 2017
The analysis regarding the cluster size is correct. If there's not a keyframe then a new cluster will only be forced when it hits the max (the default is close to the format's actual max). I don't know about the playback issue though.
,
Oct 17 2017
chfremer@, Friendly ping to get an update on this issue. Thanks..!
,
Oct 18 2017
As per #12, with 30 seconds being the default, I don't think that we should add code to lower that cluster size on the recording side until we understand why and how players have issues with it. Changing the component to Blink>Media>Video and removing ownership to get attention from video playback team.
,
Oct 19 2017
,
Oct 19 2017
Keep in mind h264 in webm is not really a webm, it's an mkv file. webm specifically only supports vp8, vp9. The fact that we stuff h264 in webm was a temporary workaround to not having an mp4 muxer.
,
Oct 19 2017
re #18: Oh, okay, I didn't realize that. Would you say that configuring libwebm to use a shorter maximum cluster length would be an appropriate solution here then? I'd be happy to create a CL for that, just want to make sure it is the right approach.
,
Oct 19 2017
Sorry don't know enough to comment one way or the other. I'd look over the matroska spec and see if it has any guidance though.
,
Nov 1 2017
Dale, this bug has been unconfirmed for a while. can you assign to a real owner?
,
Nov 2 2017
|
||||||||||||
►
Sign in to add a comment |
||||||||||||
Comment 1 by m...@willem.net
, Jul 31 2017