New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 837450 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

MediaRecorder outputs encoded with h264 only contain a single gop

Reported by m...@mux.com, Apr 27 2018

Issue description

Chrome Version       : 66.0.3359.117 (Official Build) (64-bit)
OS Version: OS X 10.13
URLs (if applicable) : https://codepen.io/mmcc/pen/ELyzYe
Other browsers tested:
  Add OK or FAIL after other browsers where you have tested this issue:
     Safari:
    Firefox: OK
    IE/Edge: N/A

What steps will reproduce the problem?
1. Start a recording, then download the result.
2. `ffprobe -show_frames [recorded_file] | grep pict_type=I`
3. You should see just one iframe printed out.

What is the expected result?
Multiple iframes. For reference, the same video encoded using VP9 instead produces an output with 18 iframes.


What happens instead of that?
Just one iframe. We've had customers send us inputs as long as *2.5 hours* with a single iframe at the beginning.


Please provide any additional information below. Attach a screenshot if
possible.

We tried configuring the `timeslice` argument in hopes that would actually create a new gop each time, but saw no difference.

We also considered that this may be a hardware encoder issue, as all of our testing was done with MacBooks of similar specs. We tested on a Dell XPS13 and someone's windows desktop (GTX1070, if that helps, no QuickSync) and saw the same results. However, we just tested on a low-powered Chromebook and found 8 wonderful iframes in the output. I honestly don't know at this point if that should make me happy or sad.

 

Comment 1 by m...@mux.com, Apr 27 2018

I meant to change this from just being a Mac bug since it's reproducible on multiple OS's. So far the Chromebook is the only one that we've gotten a reasonable output from. 
Cc: chcunningham@chromium.org mcasas@chromium.org
Components: Blink>MediaRecording
Labels: -Pri-3 Pri-2
Labels: Needs-Triage-M66

Comment 4 by mcasas@chromium.org, Apr 27 2018

Owner: emir...@chromium.org
Status: Untriaged (was: Unconfirmed)
The sw encoder is meant to produce an I frame every 1000 frames [1,2],
whereas the hw encoder is using the standard settings, which depend on
the platform, so it might be a matter of changing those settings, 
potentially having to wire them.  From the description, I see that a
"low-powered Chromebook" probably uses the sw path (chrome://gpu has 
a section indicating what "accelerated" encoding is available), so the
amount of i-frames is as expected.

@emircan@ because I think I recall PeerConnection might want to also 
wire those settings...?



[1] https://cs.chromium.org/chromium/src/content/renderer/media_recorder/h264_encoder.cc?sq=package:chromium&dr&l=137
[2] https://github.com/cisco/openh264/wiki/TypesAndStructures#id21
Status: Assigned (was: Untriaged)
VideoEncodeAccelerator::Encode() has a |bool force_keyframe| parameter that causes keyframe production. PeerConnection sets this to true in some intervals and network changes through WebRTC's video engine. All VideoEncodeAccelerator respects this.

I think, the ideal solution is to change MediaRecorder's VEAEncoder to ask for keyframe in a certain interval. It is never calling that right now and let's HW encoder do the default[0]. mcasas@ is there any related spec discussion on this? Would you like to keep it in sync with SW, such that force a keyframe every 1000 frames?

[0] https://cs.chromium.org/chromium/src/content/renderer/media_recorder/vea_encoder.cc?sq=package:chromium&dr&l=234
Project Member

Comment 6 by bugdroid1@chromium.org, Apr 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e39cab4d53bdca08d52958d762dadde50e053532

commit e39cab4d53bdca08d52958d762dadde50e053532
Author: Emircan Uysaler <emircan@chromium.org>
Date: Fri Apr 27 23:09:33 2018

Force keyframe in regular intervals using MediaRecorder's VEAEncoder

This CL makes sure that we produce a keyframe for every 100 frames when
MediaRecorder uses HW encoder.

Bug:  837450 
Change-Id: Ifa1a674012085aa122a8e907ca1f88cb8a949e55
Reviewed-on: https://chromium-review.googlesource.com/1033181
Commit-Queue: Emircan Uysaler <emircan@chromium.org>
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Cr-Commit-Position: refs/heads/master@{#554552}
[modify] https://crrev.com/e39cab4d53bdca08d52958d762dadde50e053532/content/renderer/media_recorder/vea_encoder.cc
[modify] https://crrev.com/e39cab4d53bdca08d52958d762dadde50e053532/content/renderer/media_recorder/vea_encoder.h

Status: Fixed (was: Assigned)
matt@mux.com please verify the fix when you get a chance in your setup too. It should be available in canary soon.

Comment 8 by m...@mux.com, Apr 28 2018

Great! Thank you so much for tackling this so quickly. I should be able to give it a try over the weekend.
Cc: krajshree@chromium.org
Labels: Needs-Feedback
Tried verifying the fix on mac 10.13.3 using chrome version #68.0.3415.0.

Attached a screen cast for reference.

Following are the steps followed to reproduce the issue.
------------
1. Navigated to url: https://codepen.io/mmcc/pen/ELyzYe to start a recording and downloading the result.
2. Observed that upon clicking the "download button", a 404 error was generated.

emircan@ - Could you please provide any other sample url/test file to verify the fix from TE-end.

Thanks...!!
837450.mp4
1.7 MB View Download

Comment 11 by m...@mux.com, Apr 30 2018

Ah yeah, that 404 is because the stop button wasn't wired up to "finish" the recording process very well. I just confirmed in Chrome Canary [Version 68.0.3415.0 (Official Build) canary (64-bit)] that h264 recording outputs are well-formed. Thanks all!

Comment 12 by m...@mux.com, Apr 30 2018

Just for confirmation in case anyone's curious, attached is an output from 68.0.3415.0 (RecordedVideo.blob - https://www.dropbox.com/s/3blf8ftwm5lumf0/RecordedVideo.blob?dl=0)

```
% ffprobe -show_frames RecordedVideo.blob | grep pict_type=I
ffprobe version 3.4.2 Copyright (c) 2007-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from 'RecordedVideo.blob':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
    Stream #0:1(eng): Video: h264 (Baseline), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], 1k tbr, 1k tbn, 2k tbc (default)
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
pict_type=I
```

Also including an output from the current stable version of Chrome (66.0.3359.139) (RecordedVideo-stable.blob - https://www.dropbox.com/s/ksa782vxywyo6w6/RecordedVideo-stable.blob?dl=0):

```
% ffprobe -show_frames RecordedVideo-stable.blob | grep pict_type=I
ffprobe version 3.4.2 Copyright (c) 2007-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from 'RecordedVideo-stable.blob':
  Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)
    Stream #0:1(eng): Video: h264 (Baseline), yuv420p(progressive), 640x480 [SAR 1:1 DAR 4:3], 1k tbr, 1k tbn, 2k tbc (default)
pict_type=I
```
[bulk-edit: disregard if N/A] Can the owner please set milestone to this bug if applicable?

Sign in to add a comment