MediaRecorder: consider producing seekable WebM files |
||||||||||
Issue descriptionMediaRecorder produces WebM chunks that, when assembled, produce a WebM file that cannot be seeked and does not have correct duration. The code writing the WebM header is in the WebmMuxer ctor [1] and its vicinity, and uses //third_party/libwebm, in particular: segment_.set_mode(mkvmuxer::Segment::kLive); segment_.OutputCues(false); Let's investigate. [1] https://cs.chromium.org/chromium/src/media/muxers/webm_muxer.cc?type=cs&q=webmmuxer::webmmuxer&sq=package:chromium&l=106
,
Aug 29 2016
,
Aug 29 2016
,
Aug 29 2016
keptavista@gmail.com mentioned in another bug: " It seems newest firefox has solved this problem: https://bugzilla.mozilla.org/show_bug.cgi?id=969290 " which is an extended discussion as to how to best support Cues in Firefox, keptavista@ can you please comment? What is your proposed solution?
,
Sep 1 2016
Sorry I am not competent to propose a solution, I found those tickets when debugging my own app. I accidentally pasted duplicate bugzilla ticket though - here is original: https://bugzilla.mozilla.org/show_bug.cgi?id=657791
,
Oct 5 2016
,
Oct 18 2016
Issue 656426 has been merged into this issue.
,
Oct 18 2016
,
Nov 28 2016
Issue 658149 has been merged into this issue.
,
Jan 17 2017
,
Jan 18 2017
Issue 682223 has been merged into this issue.
,
Jan 18 2017
,
Jan 18 2017
Bulk move Blink>MediaStream>Recording ---> Blink>MediaRecording
,
Mar 25 2017
@mcasas - any update on this? Seems like it may have been de-prioritized. We have our own clever way of working around this but curious where this lies. :-)
,
Mar 27 2017
I'm also waiting for this feature/fix! @useloom - what kind of workaround are you using? Do you recreate the webm headers server side with ffmpeg? Or do you have a client side solution (with javascript). Would be great, if you could share your clever workaround ;)..
,
Mar 27 2017
Could you please explain a bit why does it mean to have non-seekable webm files? I can append webm chunks into a buffer using MSE while being able to seek. Also, the complete file plays fine in any player. Is this somehow related to this issue? https://bugs.chromium.org/p/chromium/issues/detail?id=606000#c30 Thanks.
,
Mar 27 2017
#16: Based on my understanding, "non-seekable" here really means "non-efficiently-seekable". It refers to WebM files that do not contain a "cues" section. Without the cue section, seeking is still possible (assuming the player supports this) but requires reading all of the audio/video data up to the target location in order to find it. In order to find the cues section in a WebM file, it's location has to be specified int the SeekHead section at the beginning of the file. This is problematic when streaming the file data in chunks, because it forces us to deliver the beginning of the file before the recording is complete, i.e. before we could possibly know the number and location of cues we would want to produce. #14/15: Without adding an API for updating the beginning of the file after the recording is complete, the best we could do is the following. As long as the WebMediaRecorder API is used in a way that produces the complete file in a single chunk, we could produce a WebM file including SeekHead and cues. As soon as a chunk of data is requested before the recording is stopped, we would have to fall back to producing a WebM without the SeekHead and cues. Would that be something you would be interested in?
,
Mar 27 2017
#17: I would definitely find that useful.
,
Mar 28 2017
#17: That would be perfect for my use case.
,
Mar 29 2017
#17: It is possible that the recorder runs for more than an hour in our use case. We have chunk sizes of about 25MB which are stored in IndexedDB. So we will not benefit from the mentioned single chunk approach. Are there other methods to correct the webm file after chunks are created? Or do we need an additional API?
,
Mar 29 2017
#20: Agreed that the single chunk approach would not work for large/long recordings, because it would have to keep everything in memory. I currently don't see any other method without changing the API, because once the blob containing the beginning of the WebM file has been handed off from the browser to the JavaScript app, we cannot retroactively change it.
,
Mar 31 2017
I have tried with the single chunk approach and I'm still facing the same issue. Video is not seekable before fully watching it. Is there any workaround js based?
,
Apr 2 2017
#22: It isn't great, but a simple workaround is to jump to the end of WebM file and then jump back to the beginning on page load (using JavaScript). The will likely only work with a single chunk. I found this workaround here, which has implementation details: http://stackoverflow.com/questions/38443084/how-can-i-add-predefined-length-to-audio-recorded-from-mediarecorder-in-chrome/39971175#39971175 I am using it here, if helpful: https://github.com/common-nighthawk/hello-its-me/blob/master/public/js/message-create.js (line 68)
,
Apr 2 2017
#23 I am the author of this stackoverflow answer/workaround, but it will make the recorded media "seekable" only in the webpage that uses this trick, it doesn't change the file whatsoever. Actually, it is just implementing the *reading all of the audio/video data up to the target location in order to find it* from #17. So your webm file still won't have the correct cues sections. A real working workaround is to use `ffmpeg -i noCues.webm output.webm`, but this isn't doable currently on front-side, and I don't know of a js mkv's metadata parser/encoder yet which could allow us to do it from front.
,
Apr 18 2017
This issue was discussed in the Spec [1] and we were of the opinion that a polyfill/complementary solution would be enough. Concretely, ts-ebml [2] can be used to (re)construct the Cues/SeekHeads both live and as a post- processing step. All this, of course, notwithstanding solutions or workarounds applied to the playback (such as e.g. #23/24). In light of this, I'm marking this issue as WontFix. [1] https://github.com/w3c/mediacapture-record/issues/119 [2] https://www.npmjs.com/package/ts-ebml
,
May 10 2017
Beyond just seeking, i'm afraid that the current state of webm recording drives transcoding software like ffmpeg mad: the mkv header apparently contains no framerate information (is this related ? https://bugs.chromium.org/p/chromium/issues/detail?id=589561 ). $ LANG=C mkvinfo original_chrome_58_linux.webm + EBML head |+ EBML version: 1 |+ EBML read version: 1 |+ EBML maximum ID length: 4 |+ EBML maximum size length: 8 |+ Doc type: webm |+ Doc type version: 4 |+ Doc type read version: 2 + Segment, size unknown |+ Segment information | + Timecode scale: 1000000 | + Multiplexing application: Chrome | + Writing application: Chrome |+ Segment tracks | + A track | + Track number: 1 (track ID for mkvmerge & mkvextract: 0) | + Track UID: 37541427745733400 | + Track type: audio | + Codec ID: A_OPUS | + CodecPrivate, length 19 | + Audio track | + Sampling frequency: 48000 | + Channels: 1 | + A track | + Track number: 2 (track ID for mkvmerge & mkvextract: 1) | + Track UID: 563566770589222 | + Track type: video | + Codec ID: V_VP8 | + Video track | + Pixel width: 1280 | + Pixel height: 720 |+ Cluster and ffprobe/ffmpeg just falls back to 1000 fps (!!!), causing a lot of trouble when transcoding. $ ffprobe original_chrome_58_linux.webm ffprobe version 3.3 Copyright (c) 2007-2017 the FFmpeg developers built with gcc 6.3.1 (GCC) 20170306 configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-netcdf --enable-shared --enable-version3 libavutil 55. 58.100 / 55. 58.100 libavcodec 57. 89.100 / 57. 89.100 libavformat 57. 71.100 / 57. 71.100 libavdevice 57. 6.100 / 57. 6.100 libavfilter 6. 82.100 / 6. 82.100 libavresample 3. 5. 0 / 3. 5. 0 libswscale 4. 6.100 / 4. 6.100 libswresample 2. 7.100 / 2. 7.100 libpostproc 54. 5.100 / 54. 5.100 Input #0, matroska,webm, from 'original_chrome_58_linux.webm': Metadata: encoder : Chrome Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default) Stream #0:1(eng): Video: vp8, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn, 1k tbc (default) A current workaround is to fixate the framerate in the matroska container using mkvtoolnix: mkvmerge -o out_mkvtoolnix.mkv --default-duration 1:30/1fps original_chrome_58_linux.webm $ LANG=C mkvinfo out_mkvtoolnix.mkv + EBML head |+ EBML version: 1 |+ EBML read version: 1 |+ EBML maximum ID length: 4 |+ EBML maximum size length: 8 |+ Doc type: matroska |+ Doc type version: 4 |+ Doc type read version: 2 + Segment, size 1656070 |+ Seek head (subentries will be skipped) |+ EbmlVoid (size: 4029) |+ Segment information | + Timecode scale: 1000000 | + Multiplexing application: libebml v1.3.4 + libmatroska v1.4.7 | + Writing application: mkvmerge v11.0.0 ('Alive') 64bit | + Duration: 10.466s (00:00:10.466) | + Date: Wed May 10 16:20:15 2017 UTC | + Segment UID: 0x1b 0x67 0x9b 0x6a 0x2a 0xfd 0x6c 0xec 0xac 0x73 0x69 0x72 0x30 0x2b 0xfd 0xc6 |+ Segment tracks | + A track | + Track number: 1 (track ID for mkvmerge & mkvextract: 0) | + Track UID: 37541427745733400 | + Track type: audio | + Codec ID: A_OPUS | + Seek pre-roll: 80.000ms (80000000ns) | + CodecPrivate, length 19 | + Audio track | + Sampling frequency: 48000 | + A track | + Track number: 2 (track ID for mkvmerge & mkvextract: 1) | + Track UID: 563566770589222 | + Track type: video | + MinCache: 1 | + Codec ID: V_VP8 | + Default duration: 33.333ms (30.000 frames/fields per second for a video track) | + Video track | + Pixel width: 1280 | + Pixel height: 720 | + Display width: 1280 | + Display height: 720 |+ EbmlVoid (size: 1116) |+ Cluster Unfortunately, we still need to blind guess/force the real framerate that MediaRecorder used. In an ideal world, the produced webm file should have proper header information, but at least having the framerate would help. Any thoughts ?
,
May 10 2017
Interestingly, even if the current firefox 53 produced files do not contain any framerate either (nor duration for that matter -- seems to contradict comment #4), but ffprobe does not go nuts and falls back to a saner 60 fps.
fthiery@flowmixer:/tmp$ LANG=C mkvinfo original_firefox_53_linux.webm
+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ EBML maximum ID length: 4
|+ EBML maximum size length: 8
|+ Doc type: webm
|+ Doc type version: 2
|+ Doc type read version: 2
+ Segment, size unknown
|+ Seek head (subentries will be skipped)
|+ Segment information
| + Timecode scale: 1000000
| + Duration: 0.000s (00:00:00.000)
| + Multiplexing application: QTmuxingAppLibWebM-0.0.1
| + Writing application: QTwritingAppLibWebM-0.0.1
|+ Segment tracks
| + A track
| + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
| + Track UID: 1305419017
| + Codec name: VP8
| + Track type: video
| + Codec ID: V_VP8
| + Video track
| + Pixel width: 1280
| + Pixel height: 720
|+ Cluster
fthiery@flowmixer:/tmp$ LANG=C mkvinfo original_firefox_53_linux.webm ^C
fthiery@flowmixer:/tmp$ ffprobe original_firefox_53_linux.webm
ffprobe version 3.3 Copyright (c) 2007-2017 the FFmpeg developers
built with gcc 6.3.1 (GCC) 20170306
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-netcdf --enable-shared --enable-version3
libavutil 55. 58.100 / 55. 58.100
libavcodec 57. 89.100 / 57. 89.100
libavformat 57. 71.100 / 57. 71.100
libavdevice 57. 6.100 / 57. 6.100
libavfilter 6. 82.100 / 6. 82.100
libavresample 3. 5. 0 / 3. 5. 0
libswscale 4. 6.100 / 4. 6.100
libswresample 2. 7.100 / 2. 7.100
libpostproc 54. 5.100 / 54. 5.100
Input #0, matroska,webm, from 'original_firefox_53_linux.webm':
Metadata:
encoder : QTmuxingAppLibWebM-0.0.1
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0(eng): Video: vp8, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
,
May 10 2017
Sorry, here is the proper linux chrome file sample
,
May 11 2017
Note that the ffmpeg remuxing workaround is not viable anymore; i created a ticket in ffmpeg: https://trac.ffmpeg.org/ticket/6386 The attached html test file allows to produce small fragments, which seem to always be detected as 1000 fps after remuxing.
,
May 11 2017
Last, i would like to report that h264 produced webm do not have the same issue, so it is probably more VP8/ffmpeg related. Sorry for the noise
,
Oct 27 2017
,
Jan 9 2018
What is Chrome's position on this officially? Firefox has addressed the issue, WebM files generated by the MediaRecorder API are seekable on Firefox, without the use of a package like https://github.com/legokichi/ts-ebml
,
Oct 1
Why is this point not solved today? I am facing same issue, I also used same EBML library. But after that the video produced is not correctly generated, the tool mkvalidator give me always the following error : ERR201: Invalid 'MaxBlockAdditionID' for profile 'webm' in TrackEntry at 204 It makes the video to freeze on some range of time, not always the same...
,
Dec 19
Issue 916563 has been merged into this issue.
,
Jan 2
Issue 905249 has been merged into this issue. |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by mcasas@chromium.org
, Aug 29 2016