Chrome "Aw Snap!" (out of memory) issue running simple GPU test
Reported by jer...@duckware.com, Oct 31 2016
UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36 Steps to reproduce the problem: 1. Visit http://www.vsynctester.com 2. Click on "GPU memory usage tester" link at bottom of page 3. Click on "Run GPU Memory Test" link in the new page 4. If test completes, run gpu test (step 3) again, until crash happens What is the expected behavior? What went wrong? In the middle of the test, Chrome crashes with an "Aw, Snap!" screen Crashed report ID: 2a08bc0a-6e4a-4ce9-bbbc-83955b8c8cba How much crashed? Just one tab Is it a problem with a plugin? No Did this work before? N/A Chrome version: 56.0.2905.1 Channel: canary OS Version: 6.1 (Windows 7, Windows Server 2008 R2) Flash Version: Shockwave Flash 23.0 r0 Running Windows "Task Manager" during the test shows Chrome memory growing and dropping, growing and dropping, and finally growing to a point (>2GB) that is apparently an out of memory error.
Dec 9 2016,
Dec 13 2016,
Unable to reproduce the issue on windows 7, Linux Ubuntu 14.04 and Mac 10.12.1 using chrome version 56.0.2924.18 and canary 57.0.2949.0 with the below steps 1. Visit http://www.vsynctester.com 2. Clicked on "GPU memory usage tester" link at bottom of page 3. Clicked on "Run GPU Memory Test" link in the new page 4. Ran the gpu memory test almost 6 times.Not observed any crash. 5.Observed the test completed approx: 78 to secs and CPU usage increased upto 8 in task manager. jerryj@Could you please confirm anything missed here in triaging the issue? Request you please try the issue by upgrading the chrome to latest dev and update the thread if the issue still persists. Thanks,
Dec 13 2016,
kavvaru/2: Yes, still can reproduce (every time). Use 32-bit Chrome on Win7. In task manager, look at the "Memory (Private Working Set)" for the renderer process. Memory usage just goes up and up unbounded.
Dec 14 2016,
Jan 2 2017,
Crashed with: Version 55.0.2883.87 m (64-bit)
Jan 3 2017,
Able to reproduce the issue on win10 chrome version 55.0.2883.87 and canary 57.0.2969.0 with 32 bit chrome - tab crashes while running the test Crash ID 47e25c46-653e-474b-9a18-c907b06680fb (Server ID: ab2c5fe080000000) Stack Trace: Thread 0 CRASHED [EXCEPTION_ACCESS_VIOLATION_READ @ 0x0000001c ] MAGIC SIGNATURE THREAD Stack Quality92%Show frame trust levels 0x61a6fe52 (chrome_child.dll -sksurface_base.h:108 ) SkSurface_Base::getCachedCanvas() 0x61bec4b1 (chrome_child.dll -recordingimagebuffersurface.cpp:107 ) blink::RecordingImageBufferSurface::fallBackToRasterCanvas(blink::RecordingImageBufferSurface::FallbackReason) 0x61beb5c3 (chrome_child.dll -recordingimagebuffersurface.cpp:163 ) blink::RecordingImageBufferSurface::newImageSnapshot(blink::AccelerationHint,blink::SnapshotReason) 0x61beb56c (chrome_child.dll -imagebuffer.cpp:188 ) blink::ImageBuffer::newSkImageSnapshot(blink::AccelerationHint,blink::SnapshotReason) 0x61beb4f8 (chrome_child.dll -imagebuffer.cpp:193 ) blink::ImageBuffer::newImageSnapshot(blink::AccelerationHint,blink::SnapshotReason) 0x61beb4a7 (chrome_child.dll -canvasrenderingcontext2d.cpp:601 ) blink::CanvasRenderingContext2D::getImage(blink::AccelerationHint,blink::SnapshotReason) 0x61beb39f (chrome_child.dll -htmlcanvaselement.cpp:1306 ) blink::HTMLCanvasElement::getSourceImageForCanvas(blink::SourceImageStatus *,blink::AccelerationHint,blink::SnapshotReason,blink::FloatSize const &) 0x61872fa0 (chrome_child.dll -baserenderingcontext2d.cpp:1107 ) blink::BaseRenderingContext2D::drawImage(blink::ExecutionContext *,blink::CanvasImageSource *,double,double,double,double,double,double,double,double,blink::ExceptionState &) 0x6187250b (chrome_child.dll -baserenderingcontext2d.cpp:938 ) blink::BaseRenderingContext2D::drawImage(blink::ExecutionContext *,blink::CSSImageValueOrHTMLImageElementOrHTMLVideoElementOrHTMLCanvasElementOrImageBitmapOrOffscreenCanvas const &,double,double,double,double,blink::ExceptionState &) 0x61872201 (chrome_child.dll -v8canvasrenderingcontext2d.cpp:2113 ) blink::CanvasRenderingContext2DV8Internal::drawImage2Method 0x61873528 (chrome_child.dll -v8canvasrenderingcontext2d.cpp:2220 ) blink::CanvasRenderingContext2DV8Internal::drawImageMethodCallback 0x7762dc2d (ntdll.dll + 0x0003dc2d ) RtlAllocateHeap 0x6198caf8 (chrome_child.dll -execution.cc:139 ) v8::internal::`anonymous namespace'::Invoke This crash can be seen from M53 builds to latest canary Link to the builds: https://crash.corp.google.com/browse?q=product.name%3D%27Chrome%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27renderer%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27SkSurface_Base%3A%3AgetCachedCanvas%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000 Possible suspect : https://codereview.chromium.org/2557783002 Please reassign if this is not related to your change.
Jan 3 2017,
57.0.2969.0 0.00% 2 canary 57.0.2950.4 0.21% 115 Dev 56.0.2924.28 0.24% 131 Beta 55.0.2883.87 24.29% 13079 Stable Through code search on recordingimagebuffersurface.cpp suspecting https://codereview.chromium.org/2557783002 Please reassign if this is not related to your change.
Jan 3 2017,
32-bit bisect: r371207 works, r371214 crashes. Changes in range are https://chromium.googlesource.com/chromium/src/+log/56f66725674e6e4795e02b16c928372450061129..45f46be8dd032d12b198ca4e81d60b652b27dce6: 45f46be Make undo-smart-delete-word.html to use w3c test harness 2e91ed9 command_buffer: Fix setting samplers with bound uniforms d6193f3 Suppress handle leaks in base::SharedMemory::ShareToProcessCommon 426a0c0 Disable ExtensionTagsTest.PreAndPostExistingTaskProviding on Mac (as well). 1443072 Exclude all ash test on DrMemory 44af544 Call DOMFocusOut in fast/dom/HTMLDocument/set-focus-on-valid-element.html b25744b Enable Oilpan on ToT
Jan 4 2017,
For me, running r371207 results in the Memory for the "--type=renderer" process hovering around 300K, but in r371214, the Memory for the "--type=renderer" process cycles (low, low, low, med, low, high, low, higher, low, higher still, low, extremely high, low, near max, low; then crash). Looks like failed garbage collection?
Jan 6 2017,
This is indeed a garbage collection issue. The regression was most likely caused by b25744b "Enable Oilpan on ToT" I think the problem could be that the tight animation loop in that page is leaving no idle time for GC. Also the canvas uses externally allocated memory (not on the JS heap) for its pixel buffers. So there may be something wrong or sub-optimal with the mechanisms that trigger GC under these conditions. What I am observing in the dev tools timeline is that major GCs are being triggered as expected in the beginning. As the sizes of the temporary canvas objects get larger, the GCs should become more frequent but they are not. It looks like the calls to v8::Isolate::GetCurrent()->AdjustAmountOfExternalAllocatedMemory() do not have the same effect as before.
Jan 6 2017,
@jerryj: If you want to work around this issue for now, you can do this: before the line that calls makeCanvas(), set the size of the old canvas to 0. This will cause the canvas to release its pixel buffer before the object goes out of scope. That way, the pixel buffers don't have to wait for garbage collection to be deallocated. Another approach would be to recycle (resize) the same canvas over and over rather than creating a new one at each iteration of the test.
Jan 7 2017,
@junov: The problem is that Chrome is very slow with large images (96MB GPU limit issue), which causes a bug in the Chrome's garbage collection algorithm to be exposed. The problem Chrome has with large images is a very well known problem discussed at length in other issues (and remains unfixed). The animation loop is doing nothing -- drawing a single large image every 17ms. That leaves plenty of idle time, as evidenced by the very low CPU behavior in every other web browser. Clearly, the garbage collection algorithm bug needs to be addressed (why can't it see process memory going over 2G; or, why can't it detect the failed memory allocation, gc and try again?), but Chrome's inability to work with large images must also be addressed (in the other issue, you said this was coming -- is there a timeframe for this?)
Feb 7 2017,
Kentaro could you find an appropriate owner? Thanks!
Feb 7 2017,
Feb 7 2017,
Maybe are we missing calling v8::AdjustAmountOfExternalAllocatedMemory when allocating a big image? Then a V8 GC won't be triggered. Thus an Oilpan GC won't be triggered.
Feb 10 2017,
Feb 10 2017,
I agree that the core issue of some memory allocation not being accounted for must be fixed. But shouldn't there also be a 'on memory allocation failure, GC, and try memory allocation again'?
Feb 21 2018,
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
May 21 2018,
Does this still reproduce somewhere somehow? The system changed quite a bit in the last few months. (Triaging old issues)
Sign in to add a comment