New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 42 users
Status: WontFix
Owner:
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 571632


Show other hotlists

Hotlists containing this issue:
BugReplay
Hotlist-1


Sign in to add a comment
MediaRecorder: consider producing seekable WebM files
Project Member Reported by mcasas@chromium.org, Aug 29 2016 Back to list
MediaRecorder produces WebM chunks that, when assembled, produce a WebM 
file that cannot be seeked and does not have correct duration.

The code writing the WebM header is in the WebmMuxer ctor [1] and
its vicinity, and uses //third_party/libwebm, in particular:

  segment_.set_mode(mkvmuxer::Segment::kLive);
  segment_.OutputCues(false);


Let's investigate.

[1] https://cs.chromium.org/chromium/src/media/muxers/webm_muxer.cc?type=cs&q=webmmuxer::webmmuxer&sq=package:chromium&l=106
 
Comment 1 by mcasas@chromium.org, Aug 29 2016
 Issue 639496  has been merged into this issue.
Comment 2 by mcasas@chromium.org, Aug 29 2016
Blocking: 571632
Comment 3 by mcasas@chromium.org, Aug 29 2016
Cc: emir...@chromium.org
 Issue 636666  has been merged into this issue.
Comment 4 by mcasas@chromium.org, Aug 29 2016
Cc: keptavi...@gmail.com
Status: Untriaged
keptavista@gmail.com mentioned in another bug:

" It seems newest firefox has solved this problem: https://bugzilla.mozilla.org/show_bug.cgi?id=969290 "

which is an extended discussion as to how to best support Cues in Firefox, keptavista@
can you please comment?  What is your proposed solution?
Sorry I am not competent to propose a solution, I found those tickets when debugging my own app. I accidentally pasted duplicate bugzilla ticket though - here is original: https://bugzilla.mozilla.org/show_bug.cgi?id=657791
Status: Available
Comment 7 by mcasas@chromium.org, Oct 18 2016
 Issue 656426  has been merged into this issue.
Comment 8 by mcasas@chromium.org, Oct 18 2016
Summary: MediaRecorder: consider producing seekable WebM files (was: MediaRecorder: support producing seekable WebM files)
Comment 9 by mcasas@chromium.org, Nov 28 2016
 Issue 658149  has been merged into this issue.
Owner: chfremer@chromium.org
Status: Assigned
 Issue 682223  has been merged into this issue.
Components: Blink>MediaRecording
Components: -Blink>MediaStream>Recording
Bulk move
Blink>MediaStream>Recording ---> Blink>MediaRecording
Comment 14 by vi...@useloom.com, Mar 25 2017
@mcasas - any update on this? Seems like it may have been de-prioritized. We have our own clever way of working around this but curious where this lies. :-)
Comment 15 by xima...@gmail.com, Mar 27 2017
I'm also waiting for this feature/fix!

@useloom - what kind of workaround are you using? Do you recreate the webm headers server side with ffmpeg? Or do you have a client side solution (with javascript). Would be great, if you could share your clever workaround ;)..
Comment 16 by demia...@gmail.com, Mar 27 2017
Could you please explain a bit why does it mean to have non-seekable webm
files? I can append webm chunks into a buffer using MSE while being able to
seek. Also, the complete file plays fine in any player.
Is this somehow related to this issue?
https://bugs.chromium.org/p/chromium/issues/detail?id=606000#c30
Thanks.
#16: Based on my understanding, "non-seekable" here really means "non-efficiently-seekable". It refers to WebM files that do not contain a "cues" section. Without the cue section, seeking is still possible (assuming the player supports this) but requires reading all of the audio/video data up to the target location in order to find it.

In order to find the cues section in a WebM file, it's location has to be specified int the SeekHead section at the beginning of the file. This is problematic when streaming the file data in chunks, because it forces us to deliver the beginning of the file before the recording is complete, i.e. before we could possibly know the number and location of cues we would want to produce.

#14/15: Without adding an API for updating the beginning of the file after the recording is complete, the best we could do is the following. As long as the WebMediaRecorder API is used in a way that produces the complete file in a single chunk, we could produce a WebM file including SeekHead and cues. As soon as a chunk of data is requested before the recording is stopped, we would have to fall back to producing a WebM without the SeekHead and cues. 

Would that be something you would be interested in?
#17: I would definitely find that useful.
#17: That would be perfect for my use case.
Comment 20 by xima...@gmail.com, Mar 29 2017
#17: It is possible that the recorder runs for more than an hour in our use case. We have chunk sizes of about 25MB which are stored in IndexedDB. So we will not benefit from the mentioned single chunk approach. Are there other methods to correct the webm file after chunks are created? Or do we need an additional API?
#20: Agreed that the single chunk approach would not work for large/long recordings, because it would have to keep everything in memory. I currently don't see any other method without changing the API, because once the blob containing the beginning of the WebM file has been handed off from the browser to the JavaScript app, we cannot retroactively change it.
I have tried with the single chunk approach and I'm still facing the same issue. Video is not seekable before fully watching it. Is there any workaround js based? 
#22: It isn't great, but a simple workaround is to jump to the end of WebM file and then jump back to the beginning on page load (using JavaScript).  The will likely only work with a single chunk.

I found this workaround here, which has implementation details: http://stackoverflow.com/questions/38443084/how-can-i-add-predefined-length-to-audio-recorded-from-mediarecorder-in-chrome/39971175#39971175

I am using it here, if helpful: https://github.com/common-nighthawk/hello-its-me/blob/master/public/js/message-create.js (line 68)
#23 I am the author of this stackoverflow answer/workaround, but it will make the recorded media "seekable" only in the webpage that uses this trick, it doesn't change the file whatsoever.  
Actually, it is just implementing the *reading all of the audio/video data up to the target location in order to find it* from #17.

So your webm file still won't have the correct cues sections.

A real working workaround is to use `ffmpeg -i noCues.webm output.webm`, but this isn't doable currently on front-side, and I don't know of a js mkv's metadata parser/encoder yet which could allow us to do it from front.

Status: WontFix
This issue was discussed in the Spec [1] and we were of the opinion that
a polyfill/complementary solution would be enough.  Concretely, ts-ebml [2]
can be used to (re)construct the Cues/SeekHeads both live and as a post-
processing step. All this, of course, notwithstanding solutions or
workarounds applied to the playback (such as e.g. #23/24).  In light of
this, I'm marking this issue as WontFix.


[1] https://github.com/w3c/mediacapture-record/issues/119
[2] https://www.npmjs.com/package/ts-ebml
Beyond just seeking, i'm afraid that the current state of webm recording drives transcoding software like ffmpeg mad: the mkv header apparently contains no framerate information (is this related ? https://bugs.chromium.org/p/chromium/issues/detail?id=589561 ).

$ LANG=C mkvinfo original_chrome_58_linux.webm 
+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ EBML maximum ID length: 4
|+ EBML maximum size length: 8
|+ Doc type: webm
|+ Doc type version: 4
|+ Doc type read version: 2
+ Segment, size unknown
|+ Segment information
| + Timecode scale: 1000000
| + Multiplexing application: Chrome
| + Writing application: Chrome
|+ Segment tracks
| + A track
|  + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
|  + Track UID: 37541427745733400
|  + Track type: audio
|  + Codec ID: A_OPUS
|  + CodecPrivate, length 19
|  + Audio track
|   + Sampling frequency: 48000
|   + Channels: 1
| + A track
|  + Track number: 2 (track ID for mkvmerge & mkvextract: 1)
|  + Track UID: 563566770589222
|  + Track type: video
|  + Codec ID: V_VP8
|  + Video track
|   + Pixel width: 1280
|   + Pixel height: 720
|+ Cluster

and ffprobe/ffmpeg just falls back to 1000 fps (!!!), causing a lot of trouble when transcoding.

$ ffprobe original_chrome_58_linux.webm 
ffprobe version 3.3 Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 6.3.1 (GCC) 20170306
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-netcdf --enable-shared --enable-version3
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libavresample   3.  5.  0 /  3.  5.  0
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
Input #0, matroska,webm, from 'original_chrome_58_linux.webm':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
    Stream #0:1(eng): Video: vp8, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn, 1k tbc (default)

A current workaround is to fixate the framerate in the matroska container using mkvtoolnix:
mkvmerge -o out_mkvtoolnix.mkv --default-duration 1:30/1fps original_chrome_58_linux.webm

$ LANG=C mkvinfo  out_mkvtoolnix.mkv 
+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ EBML maximum ID length: 4
|+ EBML maximum size length: 8
|+ Doc type: matroska
|+ Doc type version: 4
|+ Doc type read version: 2
+ Segment, size 1656070
|+ Seek head (subentries will be skipped)
|+ EbmlVoid (size: 4029)
|+ Segment information
| + Timecode scale: 1000000
| + Multiplexing application: libebml v1.3.4 + libmatroska v1.4.7
| + Writing application: mkvmerge v11.0.0 ('Alive') 64bit
| + Duration: 10.466s (00:00:10.466)
| + Date: Wed May 10 16:20:15 2017 UTC
| + Segment UID: 0x1b 0x67 0x9b 0x6a 0x2a 0xfd 0x6c 0xec 0xac 0x73 0x69 0x72 0x30 0x2b 0xfd 0xc6
|+ Segment tracks
| + A track
|  + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
|  + Track UID: 37541427745733400
|  + Track type: audio
|  + Codec ID: A_OPUS
|  + Seek pre-roll: 80.000ms (80000000ns)
|  + CodecPrivate, length 19
|  + Audio track
|   + Sampling frequency: 48000
| + A track
|  + Track number: 2 (track ID for mkvmerge & mkvextract: 1)
|  + Track UID: 563566770589222
|  + Track type: video
|  + MinCache: 1
|  + Codec ID: V_VP8
|  + Default duration: 33.333ms (30.000 frames/fields per second for a video track)
|  + Video track
|   + Pixel width: 1280
|   + Pixel height: 720
|   + Display width: 1280
|   + Display height: 720
|+ EbmlVoid (size: 1116)
|+ Cluster

Unfortunately, we still need to blind guess/force the real framerate that MediaRecorder used. In an ideal world, the produced webm file should have proper header information, but at least having the framerate would help.

Any thoughts ?
Interestingly, even if the current firefox 53 produced files do not contain any framerate either (nor duration for that matter -- seems to contradict comment #4), but ffprobe does not go nuts and falls back to a saner 60 fps.

fthiery@flowmixer:/tmp$ LANG=C mkvinfo original_firefox_53_linux.webm 
+ EBML head
|+ EBML version: 1
|+ EBML read version: 1
|+ EBML maximum ID length: 4
|+ EBML maximum size length: 8
|+ Doc type: webm
|+ Doc type version: 2
|+ Doc type read version: 2
+ Segment, size unknown
|+ Seek head (subentries will be skipped)
|+ Segment information
| + Timecode scale: 1000000
| + Duration: 0.000s (00:00:00.000)
| + Multiplexing application: QTmuxingAppLibWebM-0.0.1
| + Writing application: QTwritingAppLibWebM-0.0.1
|+ Segment tracks
| + A track
|  + Track number: 1 (track ID for mkvmerge & mkvextract: 0)
|  + Track UID: 1305419017
|  + Codec name: VP8
|  + Track type: video
|  + Codec ID: V_VP8
|  + Video track
|   + Pixel width: 1280
|   + Pixel height: 720
|+ Cluster
fthiery@flowmixer:/tmp$ LANG=C mkvinfo original_firefox_53_linux.webm ^C
fthiery@flowmixer:/tmp$ ffprobe original_firefox_53_linux.webm 
ffprobe version 3.3 Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 6.3.1 (GCC) 20170306
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-netcdf --enable-shared --enable-version3
  libavutil      55. 58.100 / 55. 58.100
  libavcodec     57. 89.100 / 57. 89.100
  libavformat    57. 71.100 / 57. 71.100
  libavdevice    57.  6.100 / 57.  6.100
  libavfilter     6. 82.100 /  6. 82.100
  libavresample   3.  5.  0 /  3.  5.  0
  libswscale      4.  6.100 /  4.  6.100
  libswresample   2.  7.100 /  2.  7.100
  libpostproc    54.  5.100 / 54.  5.100
Input #0, matroska,webm, from 'original_firefox_53_linux.webm':
  Metadata:
    encoder         : QTmuxingAppLibWebM-0.0.1
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Video: vp8, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)

original_firefox_53_linux.webm
2.7 MB View Download
Sorry, here is the proper linux chrome file sample
original_chrome_58_linux.webm
2.1 MB View Download
Note that the ffmpeg remuxing workaround is not viable anymore; i created a ticket in ffmpeg: https://trac.ffmpeg.org/ticket/6386

The attached html test file allows to produce small fragments, which seem to always be detected as 1000 fps after remuxing.
test_simple_record.html
4.3 KB View Download
Last, i would like to report that h264 produced webm do not have the same issue, so it is probably more VP8/ffmpeg related. Sorry for the noise
Cc: mlamouri@chromium.org
 Issue 771910  has been merged into this issue.
Sign in to add a comment