MediaRecorder outputs encoded with h264 only contain a single gop
Reported by
m...@mux.com,
Apr 27 2018
|
|||||||
Issue descriptionChrome Version : 66.0.3359.117 (Official Build) (64-bit) OS Version: OS X 10.13 URLs (if applicable) : https://codepen.io/mmcc/pen/ELyzYe Other browsers tested: Add OK or FAIL after other browsers where you have tested this issue: Safari: Firefox: OK IE/Edge: N/A What steps will reproduce the problem? 1. Start a recording, then download the result. 2. `ffprobe -show_frames [recorded_file] | grep pict_type=I` 3. You should see just one iframe printed out. What is the expected result? Multiple iframes. For reference, the same video encoded using VP9 instead produces an output with 18 iframes. What happens instead of that? Just one iframe. We've had customers send us inputs as long as *2.5 hours* with a single iframe at the beginning. Please provide any additional information below. Attach a screenshot if possible. We tried configuring the `timeslice` argument in hopes that would actually create a new gop each time, but saw no difference. We also considered that this may be a hardware encoder issue, as all of our testing was done with MacBooks of similar specs. We tested on a Dell XPS13 and someone's windows desktop (GTX1070, if that helps, no QuickSync) and saw the same results. However, we just tested on a low-powered Chromebook and found 8 wonderful iframes in the output. I honestly don't know at this point if that should make me happy or sad.
,
Apr 27 2018
,
Apr 27 2018
,
Apr 27 2018
The sw encoder is meant to produce an I frame every 1000 frames [1,2], whereas the hw encoder is using the standard settings, which depend on the platform, so it might be a matter of changing those settings, potentially having to wire them. From the description, I see that a "low-powered Chromebook" probably uses the sw path (chrome://gpu has a section indicating what "accelerated" encoding is available), so the amount of i-frames is as expected. @emircan@ because I think I recall PeerConnection might want to also wire those settings...? [1] https://cs.chromium.org/chromium/src/content/renderer/media_recorder/h264_encoder.cc?sq=package:chromium&dr&l=137 [2] https://github.com/cisco/openh264/wiki/TypesAndStructures#id21
,
Apr 27 2018
VideoEncodeAccelerator::Encode() has a |bool force_keyframe| parameter that causes keyframe production. PeerConnection sets this to true in some intervals and network changes through WebRTC's video engine. All VideoEncodeAccelerator respects this. I think, the ideal solution is to change MediaRecorder's VEAEncoder to ask for keyframe in a certain interval. It is never calling that right now and let's HW encoder do the default[0]. mcasas@ is there any related spec discussion on this? Would you like to keep it in sync with SW, such that force a keyframe every 1000 frames? [0] https://cs.chromium.org/chromium/src/content/renderer/media_recorder/vea_encoder.cc?sq=package:chromium&dr&l=234
,
Apr 27 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e39cab4d53bdca08d52958d762dadde50e053532 commit e39cab4d53bdca08d52958d762dadde50e053532 Author: Emircan Uysaler <emircan@chromium.org> Date: Fri Apr 27 23:09:33 2018 Force keyframe in regular intervals using MediaRecorder's VEAEncoder This CL makes sure that we produce a keyframe for every 100 frames when MediaRecorder uses HW encoder. Bug: 837450 Change-Id: Ifa1a674012085aa122a8e907ca1f88cb8a949e55 Reviewed-on: https://chromium-review.googlesource.com/1033181 Commit-Queue: Emircan Uysaler <emircan@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Cr-Commit-Position: refs/heads/master@{#554552} [modify] https://crrev.com/e39cab4d53bdca08d52958d762dadde50e053532/content/renderer/media_recorder/vea_encoder.cc [modify] https://crrev.com/e39cab4d53bdca08d52958d762dadde50e053532/content/renderer/media_recorder/vea_encoder.h
,
Apr 27 2018
matt@mux.com please verify the fix when you get a chance in your setup too. It should be available in canary soon.
,
Apr 28 2018
Great! Thank you so much for tackling this so quickly. I should be able to give it a try over the weekend.
,
Apr 30 2018
Tried verifying the fix on mac 10.13.3 using chrome version #68.0.3415.0. Attached a screen cast for reference. Following are the steps followed to reproduce the issue. ------------ 1. Navigated to url: https://codepen.io/mmcc/pen/ELyzYe to start a recording and downloading the result. 2. Observed that upon clicking the "download button", a 404 error was generated. emircan@ - Could you please provide any other sample url/test file to verify the fix from TE-end. Thanks...!!
,
Apr 30 2018
Ah yeah, that 404 is because the stop button wasn't wired up to "finish" the recording process very well. I just confirmed in Chrome Canary [Version 68.0.3415.0 (Official Build) canary (64-bit)] that h264 recording outputs are well-formed. Thanks all!
,
Apr 30 2018
Just for confirmation in case anyone's curious, attached is an output from 68.0.3415.0 (RecordedVideo.blob - https://www.dropbox.com/s/3blf8ftwm5lumf0/RecordedVideo.blob?dl=0) ``` % ffprobe -show_frames RecordedVideo.blob | grep pict_type=I ffprobe version 3.4.2 Copyright (c) 2007-2018 the FFmpeg developers built with Apple LLVM version 9.0.0 (clang-900.0.39.2) configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, matroska,webm, from 'RecordedVideo.blob': Metadata: encoder : Chrome Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default) Stream #0:1(eng): Video: h264 (Baseline), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], 1k tbr, 1k tbn, 2k tbc (default) pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I pict_type=I ``` Also including an output from the current stable version of Chrome (66.0.3359.139) (RecordedVideo-stable.blob - https://www.dropbox.com/s/ksa782vxywyo6w6/RecordedVideo-stable.blob?dl=0): ``` % ffprobe -show_frames RecordedVideo-stable.blob | grep pict_type=I ffprobe version 3.4.2 Copyright (c) 2007-2018 the FFmpeg developers built with Apple LLVM version 9.0.0 (clang-900.0.39.2) configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, matroska,webm, from 'RecordedVideo-stable.blob': Metadata: encoder : Chrome Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default) Stream #0:1(eng): Video: h264 (Baseline), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], 1k tbr, 1k tbn, 2k tbc (default) pict_type=I ```
,
Jun 7 2018
[bulk-edit: disregard if N/A] Can the owner please set milestone to this bug if applicable? |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by m...@mux.com
, Apr 27 2018