New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 761852 link

Starred by 7 users

Issue metadata

Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Memory leak observed in Mickey devices when playing videos

Project Member Reported by sduraisamy@chromium.org, Sep 4 2017

Issue description

Memory leak observed in Mickey devices when playing videos.

Sample video URL: http://plnkr.co/edit/nXKFQi5aXXbjnLMIfQOY?p=preview

Please refer at https://bugs.chromium.org/p/chromium/issues/detail?id=731808#c88

Please see the comment below:

I'm still observing this issue on a Mickey on 60.0.3112.112 though it may be a different decoder. Rapid memory leak which eventually results in app/tab crash.

Device log attached, quite a few of the following error:

[2630:4049:0830/101711.770934:ERROR:v4l2_slice_video_decode_accelerator.cc(1450)] DecodeBufferTask(): Setting error state:4
 
veyron_mickey_R60.3112.112.tgz
151 KB Download
Krishna, can you please repro using Mickey and upload logs?
Installed M-60 test image of build 9592.85.0, chrome 60.0.3112.112 on a Veyron-Mickey; started a user session and directed the browser to "http://plnkr.co/edit/nXKFQi5aXXbjnLMIfQOY?p=preview".  There was a brief "Loading..." message, and then a red box saying "Import error.  Plunk not found".

Do I have the correct URL?

Can you please use the html file at https://bugs.chromium.org/p/chromium/issues/detail?id=735643 ?
Labels: -Pri-2 Pri-1
Owner: owenlin@chromium.org
Owen, can you please look at the logs? A few enterprise customers are reporting this issue.
avkodipelli@, have we seen this behavior in our testing? Might be good to file a feedback report if we can get to repro this.
I wasn't able to get the plnkr video to ever play, but I was able to try a couple of 4k video examples; neither one had an out-of-control memory leak.

A 10-minute repeating loop of casino ads went through a predictable memory cycle (as reported by task manager); it would start at approximately 100mb, climb to ~370mb, then fall back to ~100mb.

A youtube live cam stream gradually grew from ~60mb to ~420mb over a period of 4 hours, then bounced between ~350mb and ~420mb thereafter.
 

Comment 7 by josh@arreya.com, Sep 11 2017

Here's another log and crash ID from a Fievel that seems to be displaying the same behavior. 

crash/fe04995fa11e9729
Google_Veyron_Fievel.6588.237.0
Veyron_Fievel.6588.237.0_20170911-163454.tgz
884 KB Download
Hi Josh, can you please upload the video as well?
Labels: videoshortlist
Status: Started (was: Assigned)
Owner: sduraisamy@chromium.org
Status: Assigned (was: Started)
I used minnie on TOT for testing. I tried to use the Memory leak extension https://bugs.chromium.org/p/chromium/issues/detail?id=735643

But it stop downloading the video at the beginning. So I use the https://bugs.chromium.org/p/chromium/issues/detail?id=731808#c58. I think it is the HTML version of the extension.

However, I have run the test for two days but it still not crash. The browser memory usage increases to 480M, and the renderer process uses about 60M.

Comparing to what it is said "rapid memory leak", I cannot reproduce the issue. 

Hi Raj, Can you be more specific about how to reproduce the issue?


Owner: owenlin@chromium.org
Owen, can you please use the URL - https://pwtvboudreaux.arreya.com/#2984 and see if you are able to repro?
Yes, I can reproduce the issue on TOT minnie with  https://pwtvboudreaux.arreya.com/#2984

Comment 13 by wal...@arreya.com, Sep 26 2017

Raj / Owen - Any updates on this issue?
Owner: dalecur...@chromium.org
After adding more log, the video failed because the calculated dpb_size is 0.

https://cs.chromium.org/chromium/src/media/gpu/h264_decoder.cc?q=h264_decoder.cc&sq=package:chromium&dr&l=1115

Then chromium is switching to using software decoding. Different from  crbug.com/731808 . The memory leak is not related to the switching. 

After disabling the hardware accelerator by adding "--disable-accelerated-video-codec", the issue is still reproducible.

Maybe the leak is in the software decoder. Assign to Dale for triaging. 
Cc: dalecur...@chromium.org
Owner: chcunningham@chromium.org
chcunningham@ is working on the ffmpeg roll right now, so over to him. Probably as a first cut we should download the video in question and run it with lsan enabled to see if there are true leaks or just an accumulation of resources.

I'll also take a look over the next couple days to see if anything sticks out.

owenlin@ can you describe the test you're running to find the leak beyond just using the URL in c#12?
Checked the video clips used by that site, none of them trigger any formal leaks that can be detected by lsan. I'ave had the site running for a long time on Linux now without any issues, so this may be some CrOS specific graphics issue.

@owenlin: Can you please respond to my Q in c#15 for test clarity? Are you able to reproduce this on Linux?

Comment 17 by owenlin@google.com, Sep 29 2017

No, I didn't do anything else to reproduce the issue.

Besides, I tested it on my desktop chrome browser and no leaking is observed. I'll try to see if this issue can be reproduce on other CrOS devices. 
Owner: owenlin@chromium.org
Owner: sduraisamy@chromium.org
Ah, just noticed the video is changed. The URL will be redirected to #3708 (instead of #2984). And I cannot reproduce the issue on veyron_minnie anymore.

I didn't kept the video.
Raj, can you help us relocate the content in #12.

Comment 20 by josh@arreya.com, Sep 29 2017

@owenlin @dalecurtis My mistake, I changed the content for our own testing. I switched back to the old content and will not change it. https://pwtvboudreaux.arreya.com/#2984
Ah, thanks for updating. I'll rerun the lsan and desktop tests on my side.
Double checked none of the test media trigger any lsan failures, so there's no actual leak of decoded resources as far as I can tell. Rerunning the desktop test now to see if anything shows up.
Had it running on desktop for hours now without issue; perpetually hovers around ~105mb, so still seems like some CrOS specific graphics issue.
Thanks. I can reproduce the issue again. However, there is something different: the video doesn't played but it still crashed. Here is some of my observations:

1. It behave different on different platforms.
   - Cyan (CrOS Intel) - The video doesn't played but it also not crashed.
   - Minnie (CrOS Rockchip ARM) - The video not played and it got crashed in the end.
   - On desktop - The video played and not crashed

2. The OOM is not related to video playback.
   - I checked the histogram (Media.GpuVideoDecoderInitializeStatus)
     There is no video decoder get initialized.
   - I tried to play several videos it prefetched, none of them leaking the memory.

3. The OOM could be related to cache and/or prefetch:

   - On Cyan: The memory usage keeps increasing to 900MB while the number of data transferred bytes increases. 
     After the data transferred stop, the memory keeps going down to 118M.

   - On Minnie: It crashed before data transferred stop.
 
   - On Desktop: (The video will be cached, so I use a incognito mode to reload the page). 
     The video starts after 2T data transferred.

Since the OOM is not related to video playback, please help triage the issue to the right owner.

Comment 25 by josh@arreya.com, Oct 2 2017

@owenlin, @dalecurtis, FYI the URL has changed to https://disabled-pwtvboudreaux.arreya.com/
Labels: -videoshortlist
Still an issue? This is out of SLO for a P1 bug, which must have a milestone and be fixed in 30days, please drop to P3 or close as fixed.

Sign in to add a comment