New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 660862 link

Starred by 9 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug



Sign in to add a comment

Chrome "Aw Snap!" (out of memory) issue running simple GPU test

Reported by jer...@duckware.com, Oct 31 2016

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36

Steps to reproduce the problem:
1. Visit http://www.vsynctester.com
2. Click on "GPU memory usage tester" link at bottom of page
3. Click on "Run GPU Memory Test" link in the new page
4. If test completes, run gpu test (step 3) again, until crash happens

What is the expected behavior?

What went wrong?
In the middle of the test, Chrome crashes with an "Aw, Snap!" screen

Crashed report ID: 2a08bc0a-6e4a-4ce9-bbbc-83955b8c8cba

How much crashed? Just one tab

Is it a problem with a plugin? No 

Did this work before? N/A 

Chrome version: 56.0.2905.1  Channel: canary
OS Version: 6.1 (Windows 7, Windows Server 2008 R2)
Flash Version: Shockwave Flash 23.0 r0

Running Windows "Task Manager" during the test shows Chrome memory growing and dropping, growing and dropping, and finally growing to a point (>2GB) that is apparently an out of memory error.
 
Labels: M-56
Cc: kavvaru@chromium.org
Components: Internals>GPU
Labels: Needs-Feedback
Unable to reproduce the issue on windows 7, Linux Ubuntu 14.04 and Mac 10.12.1 using chrome version 56.0.2924.18 and canary 57.0.2949.0 with the below steps

1. Visit http://www.vsynctester.com
2. Clicked on "GPU memory usage tester" link at bottom of page
3. Clicked on "Run GPU Memory Test" link in the new page
4. Ran the gpu memory test almost 6 times.Not observed any crash.
5.Observed the test completed approx: 78 to secs and CPU usage increased upto 8 in task manager.

jerryj@Could you please confirm anything missed here in triaging the issue?
Request you please try the issue by upgrading the chrome to latest dev and update the thread if the issue still persists.

Thanks,

Comment 3 by jer...@duckware.com, Dec 13 2016

kavvaru/2: Yes, still can reproduce (every time).  Use 32-bit Chrome on Win7.  In task manager, look at the "Memory (Private Working Set)" for the renderer process.  Memory usage just goes up and up unbounded.
Labels: -Needs-Feedback

Comment 5 by cts...@gmail.com, Jan 2 2017

Crashed with: Version 55.0.2883.87 m (64-bit)
Cc: lukasza@chromium.org tkonch...@chromium.org
Labels: -M-56 M-57
Owner: junov@chromium.org
Status: Assigned (was: Unconfirmed)
Able to reproduce the issue on win10 chrome version 55.0.2883.87 and canary 57.0.2969.0 with 32 bit chrome - tab crashes while running the test

Crash ID 47e25c46-653e-474b-9a18-c907b06680fb (Server ID: ab2c5fe080000000)

Stack Trace:
Thread 0 CRASHED [EXCEPTION_ACCESS_VIOLATION_READ @ 0x0000001c ] MAGIC SIGNATURE THREAD
Stack Quality92%Show frame trust levels
0x61a6fe52	(chrome_child.dll -sksurface_base.h:108 )	SkSurface_Base::getCachedCanvas()
0x61bec4b1	(chrome_child.dll -recordingimagebuffersurface.cpp:107 )	blink::RecordingImageBufferSurface::fallBackToRasterCanvas(blink::RecordingImageBufferSurface::FallbackReason)
0x61beb5c3	(chrome_child.dll -recordingimagebuffersurface.cpp:163 )	blink::RecordingImageBufferSurface::newImageSnapshot(blink::AccelerationHint,blink::SnapshotReason)
0x61beb56c	(chrome_child.dll -imagebuffer.cpp:188 )	blink::ImageBuffer::newSkImageSnapshot(blink::AccelerationHint,blink::SnapshotReason)
0x61beb4f8	(chrome_child.dll -imagebuffer.cpp:193 )	blink::ImageBuffer::newImageSnapshot(blink::AccelerationHint,blink::SnapshotReason)
0x61beb4a7	(chrome_child.dll -canvasrenderingcontext2d.cpp:601 )	blink::CanvasRenderingContext2D::getImage(blink::AccelerationHint,blink::SnapshotReason)
0x61beb39f	(chrome_child.dll -htmlcanvaselement.cpp:1306 )	blink::HTMLCanvasElement::getSourceImageForCanvas(blink::SourceImageStatus *,blink::AccelerationHint,blink::SnapshotReason,blink::FloatSize const &)
0x61872fa0	(chrome_child.dll -baserenderingcontext2d.cpp:1107 )	blink::BaseRenderingContext2D::drawImage(blink::ExecutionContext *,blink::CanvasImageSource *,double,double,double,double,double,double,double,double,blink::ExceptionState &)
0x6187250b	(chrome_child.dll -baserenderingcontext2d.cpp:938 )	blink::BaseRenderingContext2D::drawImage(blink::ExecutionContext *,blink::CSSImageValueOrHTMLImageElementOrHTMLVideoElementOrHTMLCanvasElementOrImageBitmapOrOffscreenCanvas const &,double,double,double,double,blink::ExceptionState &)
0x61872201	(chrome_child.dll -v8canvasrenderingcontext2d.cpp:2113 )	blink::CanvasRenderingContext2DV8Internal::drawImage2Method
0x61873528	(chrome_child.dll -v8canvasrenderingcontext2d.cpp:2220 )	blink::CanvasRenderingContext2DV8Internal::drawImageMethodCallback
0x7762dc2d	(ntdll.dll + 0x0003dc2d )	RtlAllocateHeap
0x6198caf8	(chrome_child.dll -execution.cc:139 )	v8::internal::`anonymous namespace'::Invoke

This crash can be seen from M53 builds to latest canary

Link to the builds:
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27renderer%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27SkSurface_Base%3A%3AgetCachedCanvas%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

Possible suspect : https://codereview.chromium.org/2557783002

Please reassign if this is not related to your change.
57.0.2969.0	0.00%	2	canary
57.0.2950.4	0.21%	115	Dev
56.0.2924.28	0.24%	131	Beta
55.0.2883.87	24.29%	13079	Stable

Through code search on recordingimagebuffersurface.cpp suspecting https://codereview.chromium.org/2557783002

Please reassign if this is not related to your change.

32-bit bisect: r371207 works, r371214 crashes.  Changes in range are https://chromium.googlesource.com/chromium/src/+log/56f66725674e6e4795e02b16c928372450061129..45f46be8dd032d12b198ca4e81d60b652b27dce6:

  45f46be Make undo-smart-delete-word.html to use w3c test harness
  2e91ed9 command_buffer: Fix setting samplers with bound uniforms
  d6193f3 Suppress handle leaks in base::SharedMemory::ShareToProcessCommon
  426a0c0 Disable ExtensionTagsTest.PreAndPostExistingTaskProviding on Mac (as well).
  1443072 Exclude all ash test on DrMemory
  44af544 Call DOMFocusOut in fast/dom/HTMLDocument/set-focus-on-valid-element.html
  b25744b Enable Oilpan on ToT

For me, running r371207 results in the Memory for the "--type=renderer" process hovering around 300K, but in r371214, the Memory for the "--type=renderer" process cycles (low, low, low, med, low, high, low, higher, low, higher still, low, extremely high, low, near max, low; then crash).  Looks like failed garbage collection?

Cc: junov@chromium.org
Components: -Internals>GPU Blink>JavaScript>GC
Owner: ----
Status: Available (was: Assigned)
This is indeed a garbage collection issue.  The regression was most likely caused by b25744b "Enable Oilpan on ToT"

I think the problem could be that the tight animation loop in that page is leaving no idle time for GC.  Also the canvas uses externally allocated memory (not on the JS heap) for its pixel buffers. So there may be something wrong or sub-optimal with the mechanisms that trigger GC under these conditions.

What I am observing in the dev tools timeline is that major GCs are being triggered as expected in the beginning. As the sizes of the temporary canvas objects get larger, the GCs should become more frequent but they are not.  It looks like the calls to v8::Isolate::GetCurrent()->AdjustAmountOfExternalAllocatedMemory() do not have the same effect as before.

@jerryj: If you want to work around this issue for now, you can do this: before the line that calls makeCanvas(), set the size of the old canvas to 0.  This will cause the canvas to release its pixel buffer before the object goes out of scope. That way, the pixel buffers don't have to wait for garbage collection to be deallocated. Another approach would be to recycle (resize) the same canvas over and over rather than creating a new one at each iteration of the test.
@junov: The problem is that Chrome is very slow with large images (96MB GPU limit issue), which causes a bug in the Chrome's garbage collection algorithm to be exposed.

The problem Chrome has with large images is a very well known problem discussed at length in other issues (and remains unfixed).  The animation loop is doing nothing -- drawing a single large image every 17ms.  That leaves plenty of idle time, as evidenced by the very low CPU behavior in every other web browser.

Clearly, the garbage collection algorithm bug needs to be addressed (why can't it see process memory going over 2G; or, why can't it detect the failed memory allocation, gc and try again?), but Chrome's inability to work with large images must also be addressed (in the other issue, you said this was coming -- is there a timeframe for this?)

Owner: haraken@chromium.org
Kentaro could you find an appropriate owner?
Thanks!
Components: -Blink>JavaScript>GC Infra>Client>Oilpan
Cc: -kavvaru@chromium.org haraken@chromium.org
Owner: junov@chromium.org
Maybe are we missing calling v8::AdjustAmountOfExternalAllocatedMemory when allocating a big image?

Then a V8 GC won't be triggered. Thus an Oilpan GC won't be triggered.

Components: -Infra>Client>Oilpan Blink>MemoryAllocator>GarbageCollection
I agree that the core issue of some memory allocation not being accounted for must be fixed.

But shouldn't there also be a 'on memory allocation failure, GC, and try memory allocation again'?
Project Member

Comment 18 by sheriffbot@chromium.org, Feb 21 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Assigned (was: Untriaged)
Does this still reproduce somewhere somehow? The system changed quite a bit in the last few months.

(Triaging old issues)
Cc: -junov@chromium.org
Owner: ----
Status: Available (was: Assigned)

Sign in to add a comment