New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 735984 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug-Regression


Participants' hotlists:
Hotlist-1


Sign in to add a comment

Chrome crashed due to enormous memory usage of the MediaRecorder

Reported by andy200...@gmail.com, Jun 22 2017

Issue description

UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.109 Safari/537.36

Steps to reproduce the problem:
1. open attached index.html
2. run most hard scennario (1920x1080 120 fps)
3. check constantly increasing system memory usage

What is the expected behavior?
limited memory usage

What went wrong?
After recording started, one of chrome processes start allocating system memory constantly. When allocated more then 4GB process crashed.

Did this work before? Yes 59.0.3041.0 commit: ac041cfab01f509ef22e0f6fedbad3b421bbe20a

Does this work in other browsers? Yes

Chrome version: 59.0.3041.0  Channel: stable
OS Version: 10.0
Flash Version: 

I did some research and found exact two chrome builds between which this bug has appeared.

With snapshot recording working without mentioned issue:
https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/456330/

And with this one this issue occurs for the first time:
https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Win_x64/456333/

There are only 3 commits between them and the most likely candidate is:
https://chromium.googlesource.com/chromium/src/+/9d369903d6cdd84aaaf70e2a73ada508e2b34eaa

Please notice that despite the fact that my reduced test case is slightly artificial, this issue appearing on weak machines even with more real conditions. On my colleague laptop, recording start eating memory even with 1280x720 30fps and with CPU loaded by some another task (like skype screen sharing).

I didn't dig in the MediaRecorder sources, but from my feeling, this issue appearing when recording source (canvas in my case) generating more raw frames per second then encoder manage to process. And these frames start buffering, increasing memory usage very quick.
 
index.html
2.1 KB View Download
i'm the the colleague witness browser crash due to eaten memory with 1280x720x30fps on lenovo y510p 8G laptop (A strong i7 gaming machine)

thanks.

Cc: pbomm...@chromium.org rsch...@chromium.org abdulsyed@chromium.org tommi@chromium.org
Components: Blink>MediaRecording Blink>WebRTC
Labels: Performance-Memory
Status: Available (was: Unconfirmed)
Able to reproduce the issue on latest Chrome stable i.e., 59.0.3071.109. Working on bisect to see if this is a regression started in M59 or exist earlier as well. 
Cc: emir...@chromium.org mcasas@chromium.org
Labels: ReleaseBlock-Stable
Definitely this is a regression in M59, working on the bisect.
Cc: niklase@chromium.org
Cc: -emir...@chromium.org
Owner: emir...@chromium.org
Status: Assigned (was: Available)
Please find the bisect result below :

You are probably looking for a change made after 456330 (known good), but no lat
er than 456331 (first known bad).
CHANGELOG URL:
The script might not always return single CL as suspect as some perf builds migh
t get missing due to failure.
  https://chromium.googlesource.com/chromium/src/+log/ac041cfab01f509ef22e0f6fedbad3b421bbe20a..9d369903d6cdd84aaaf70e2a73ada508e2b34eaa
Labels: M-60 M-61 M-59
So far I wasn't able to reproduce the issue on Windows 10, Mac and Linux


Note : I was only able to reproduce this issue on my Corp Windows 7 machine and was across all Channels i.e., Chrome beta(60.0.3112.40), Dev(61.0.3135.4) and Canary(61.0.3138.0)
Labels: -ReleaseBlock-Stable
Thanks for the bisect.
 
var ctx = canvas.getContext('2d', {alpha: false});
Changing this line in your demo as above should fix the problem for you. With the suspected CL, we started encoding alpha channel for VPX as well. Although you are not drawing any alpha channel, when you don't add {alpha: false}, capturer don't know what alpha contains. It adds an extra stress as we start encoding 120+120 fps. I am removing Release-Block-Stable based on that.

There isn't any extra allocated memory for alpha channel encoding[0]. I will look if something is wrong in vpx_encoder calls but it looks like we have more frames than we can process here as you explained. We cannot throw away frames in canvas capture as per spec, but maybe we can do in media recorder. 

[0] https://codereview.chromium.org/2691373005/diff/300001/content/renderer/media_recorder/video_track_recorder.cc
I have 3 step solution that will address this issue:
- When alpha is there, half the duration parameter that is set for encoding a frame. Since we are actually encoding N+N fps, this helps reducing the time spent in encoder. Tests show that encoder doesn't queue up way too many frames this way.
- Add a counter in media_recorder such that we limit the number of queued frames to a certain number. This will cover all edge cases.
- {alpha: false} usage doesn't seem to be popular. Canvas context defaults to RGBA, so we always assume to have alpha. We can go through the full alpha channel values in canvas capture and drop the alpha plane if all zeros.
Project Member

Comment 10 by bugdroid1@chromium.org, Jun 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/894d66f402ebb97afe82052fa834612feffa8e28

commit 894d66f402ebb97afe82052fa834612feffa8e28
Author: Emircan Uysaler <emircan@chromium.org>
Date: Thu Jun 29 18:05:36 2017

Limit number of frames kept alive by VideoTrackRecorder

This CL addresses issues where we see increasing memory usage when
VideoTrackRecorder cannot keep up with the number of incoming frames.
- Limit the number of frames sent for encode to 30.
- Change duration when there is alpha encode to account for the double
load.

Bug:  735984 
Test: Added unittest and tested HTML demo given in the bug.
Change-Id: I470422ca2538eb46cd24d36560f3d35b4ae28d06
Reviewed-on: https://chromium-review.googlesource.com/546996
Commit-Queue: Emircan Uysaler <emircan@chromium.org>
Reviewed-by: Miguel Casas <mcasas@chromium.org>
Cr-Commit-Position: refs/heads/master@{#483421}
[modify] https://crrev.com/894d66f402ebb97afe82052fa834612feffa8e28/content/renderer/media_recorder/video_track_recorder.cc
[modify] https://crrev.com/894d66f402ebb97afe82052fa834612feffa8e28/content/renderer/media_recorder/video_track_recorder.h
[modify] https://crrev.com/894d66f402ebb97afe82052fa834612feffa8e28/content/renderer/media_recorder/video_track_recorder_unittest.cc
[modify] https://crrev.com/894d66f402ebb97afe82052fa834612feffa8e28/content/renderer/media_recorder/vpx_encoder.cc

What's the status here? More work planned in M62?
There is more work planned but it should land before 62 cut.
Labels: M-62
Ok, bumping it to M62 then.
Cc: -rsch...@chromium.org
What's the current status of this?
Labels: -M-62 M-63
Ping.
Labels: -Pri-2 M-64 Pri-3
Current fix in #10 limits the number of frames sent to encode, and fixes the high memory usage. I kept the bug open because I want to add another layer where we keep the frames in a LIFO queue, see https://cs.chromium.org/chromium/src/content/renderer/media_recorder/video_track_recorder.cc?rcl=5814e9f72db71eccce938f8ee67756ce960a2c11&l=82. I haven't had time to work on this, updating labels accordingly.
Labels: -M-59 -M-60 -M-61 -M-63 -M-64 M-62
Status: Fixed (was: Assigned)
I had another look at this. Changing the queue from FIFO to LIFO that I suggested as next step can be problematic with the current VEA encoder design. Since VEA posts the VideoFrame onto other threads and processes, they might be queuing up where VEA implementation is and canceling tasks from VideoTrackRecorder would be no use. It is a simpler and more controlled solution as it is. #10 limits the number of frames and solves the memory issue. 

Sign in to add a comment