Chrome_Android: Crash Report - [GPU hang] base::internal::LockImpl::Lock |
|||||||||||
Issue descriptionProduct name: Chrome_Android Magic Signature: [GPU hang] base::internal::LockImpl::Lock Current link: https://crash.corp.google.com/browse?q=product.name%3D'Chrome_Android'%20AND%20product.version%3D'55.0.2868.0'%20AND%20custom_data.ChromeCrashProto.ptype%3D'gpu-process'%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D'%5BGPU%20hang%5D%20base%3A%3Ainternal%3A%3ALockImpl%3A%3ALock'&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#reports Search properties: product.name: Chrome_Android product.version: 55.0.2868.0 custom_data.chromecrashproto.ptype: gpu-process Crash Stacktrace: Thread 0 MAGIC SIGNATURE THREAD 0x400a56ab (libc.so + 0x0004b6ab ) 0x40183d02 (libutils.so + 0x0001ed02 ) 0x400a56b9 (libc.so + 0x0004b6b9 ) 0x40183f35 (libutils.so + 0x0001ef35 ) 0x68b4bb16 (libchrome.so -lock_impl_posix.cc:62 ) base::internal::LockImpl::Lock() 0x68b4bb65 (libchrome.so -lock_impl_posix.cc:68 ) base::internal::LockImpl::Unlock() 0x68b0e849 (libchrome.so -callback_internal.h:130 ) Notify<void (base::MemoryPressureListener::*)(base::MemoryPressureListener::MemoryPressureLevel), base::MemoryPressureListener::MemoryPressureLevel&> 0x68b4bb7b (libchrome.so -lock_impl_posix.cc:69 ) base::internal::LockImpl::Unlock() 0x68b0e6c8 (libchrome.so -observer_list_threadsafe.h:171 ) Notify<void (base::MemoryPressureListener::*)(base::MemoryPressureListener::MemoryPressureLevel), base::MemoryPressureListener::MemoryPressureLevel&> 0x68b4bb16 (libchrome.so -lock_impl_posix.cc:62 ) base::internal::LockImpl::Lock() 0x68b4bb65 (libchrome.so -lock_impl_posix.cc:68 ) base::internal::LockImpl::Unlock() 0x68b0eb63 (libchrome.so -lock.h:27 ) Notify 0x68b0e46f (libchrome.so + 0x002e746f ) 0x6c3c3eb3 (libchrome.so + 0x03b9ceb3 ) 0x68b0eaf4 (libchrome.so -memory_pressure_listener.cc:39 ) Notify 0x68b0eadd (libchrome.so -memory_pressure_listener.cc:36 ) Notify 0x68b0ed00 (libchrome.so -memory_pressure_listener.cc:126 ) base::MemoryPressureListener::NotifyMemoryPressure(base::MemoryPressureListener::MemoryPressureLevel) 0x42bba66a (libdvm.so + 0x000e066a ) 0x4337c58f (dalvik-zygote (deleted) + 0x0006f58f ) 0x402b9cdf (libandroid_runtime.so + 0x000b7cdf ) 0x65b0b118 (dalvik-jit-code-cache (deleted) + 0x00000118 ) 0x42b72c80 (libdvm.so + 0x00098c80 ) 0x438ae9cf (dalvik-heap (deleted) + 0x0001e9cf ) 0x42cfdaa4 (libdvm.so + 0x00223aa4 ) 0x42cfdc28 (libdvm.so + 0x00223c28 ) 0x438ae9cf (dalvik-heap (deleted) + 0x0001e9cf ) 0x4330d2af (dalvik-zygote (deleted) + 0x000002af ) 0x438aea4f (dalvik-heap (deleted) + 0x0001ea4f ) 0x4337f1ff (dalvik-zygote (deleted) + 0x000721ff ) 0x4337f1ff (dalvik-zygote (deleted) + 0x000721ff ) 0x438ae9cf (dalvik-heap (deleted) + 0x0001e9cf ) 0x58f2472f (dalvik-LinearAlloc (deleted) + 0x000b572f ) 0x5b0f16bd (framework2.odex + 0x0046c6bd ) 0x58e7b627 (dalvik-LinearAlloc (deleted) + 0x0000c627 ) 0x5a18cfff (core.odex + 0x00317fff ) 0x438aea8f (dalvik-heap (deleted) + 0x0001ea8f ) 0x42bc8af0 (libdvm.so + 0x000eeaf0 ) 0x42c1f8ef (libdvm.so + 0x001458ef ) 0x438aeabf (dalvik-heap (deleted) + 0x0001eabf ) 0x438aeabf (dalvik-heap (deleted) + 0x0001eabf ) 0x42bd0d79 (libdvm.so + 0x000f6d79 ) 0x438aeabf (dalvik-heap (deleted) + 0x0001eabf ) 0x58f2472f (dalvik-LinearAlloc (deleted) + 0x000b572f ) 0x42bc8af0 (libdvm.so + 0x000eeaf0 ) 0x4331d207 (dalvik-zygote (deleted) + 0x00010207 ) 0x438aeb33 (dalvik-heap (deleted) + 0x0001eb33 ) 0x438ae9e3 (dalvik-heap (deleted) + 0x0001e9e3 ) 0x58f2472f (dalvik-LinearAlloc (deleted) + 0x000b572f ) 0x42bed798 (libdvm.so + 0x00113798 ) 0x58f2472f (dalvik-LinearAlloc (deleted) + 0x000b572f ) ... 149 more 0x63732f6a (libwebcore.so + 0x00c1bf6a ) 0x65747378 (data@app@com.chrome.dev-2.apk@classes.dex + 0x001ab378 ) 0x61746f71 (libsavscmn.so + 0x00037f71 ) 0x6172662e (libsavscmn.so + 0x0001762e ) 0x65636165 (data@app@com.chrome.dev-2.apk@classes.dex + 0x0009a165 ) 0x6172662e (libsavscmn.so + 0x0001762e ) 0x632f6b71 (libwebcore.so + 0x007dfb71 ) 0x692e6d6e (libchrome.so -ipc_message_templates.h:89 ) ~MessageT 0x45545844 (dalvik-heap (deleted) + 0x01cb5844 ) 0x4c414e51 (dalvik-heap (deleted) + 0x08b84e51 ) 0x4f54535e (dalvik-heap (deleted) + 0x0bcb535e ) 0x45474151 (dalvik-heap (deleted) + 0x01be4151 ) 0x63616764 (libwebcore.so + 0x00aff764 ) 0x4d450078 (dalvik-heap (deleted) + 0x09bc0078 ) 0x54414c54 (dalvik-mark-stack (deleted) + 0x00d07c54 ) 0x535f4444 (dalvik-bitmap-2 (deleted) + 0x000e7444 ) 0x41524f53 (libomadrm.so + 0x000cdf53 ) 0x535f4546 (dalvik-bitmap-2 (deleted) + 0x000e7546 ) 0x4352554e (dalvik-zygote (deleted) + 0x0021854e ) 0x6574616b (data@app@com.chrome.dev-2.apk@classes.dex + 0x001aa16b ) 0x4d450063 (dalvik-heap (deleted) + 0x09bc0063 ) 0x54414c54 (dalvik-mark-stack (deleted) + 0x00d07c54 ) 0x535f4444 (dalvik-bitmap-2 (deleted) + 0x000e7444 ) 0x41524f53 (libomadrm.so + 0x000cdf53 ) 0x545f4546 (dalvik-mark-stack (deleted) + 0x00ee7546 ) 0x45475240 (dalvik-heap (deleted) + 0x01be5240 ) 0x61726f73 (libsavscmn.so + 0x00017f73 ) 0x4f434552 (dalvik-heap (deleted) + 0x0bba4552 ) 0x5241444d (dalvik-heap (deleted) + 0x0eb8444d ) 0x54535f58 (dalvik-mark-stack (deleted) + 0x00e28f58 ) 0x4741524e (dalvik-heap (deleted) + 0x03b8524e ) 0x61726f73 (libsavscmn.so + 0x00017f73 ) 0x64537477 (libSamsungPkcs11Wrapper.so + 0x000b6477 ) 0x64726142 (libvideoeditor_core.so + 0x00041142 ) 0x444e40ff (dalvik-heap (deleted) + 0x00c540ff ) 0x44494f51 (dalvik-heap (deleted) + 0x00c04f51 ) 0x4f52505e (dalvik-heap (deleted) + 0x0bc9505e ) 0x5452454f (dalvik-mark-stack (deleted) + 0x00e1754f ) 0x4f575f58 (dalvik-heap (deleted) + 0x0bce5f58 ) 0x50534b51 (dalvik-heap (deleted) + 0x0cca4b51 ) 0x52444e40 (dalvik-heap (deleted) + 0x0ebb4e40 ) 0x5f44494e (Roboto-Regular.ttf + 0x0000c94e ) 0x4b434f52 (dalvik-heap (deleted) + 0x07ba4f52 ) 0x415f4f4d (libomadrm.so + 0x0019df4d ) 0x504d4f42 (dalvik-heap (deleted) + 0x0cc44f42 ) 0x4c5f5440 (dalvik-heap (deleted) + 0x08d65440 ) 0x554f5940 (dalvik-mark-stack (deleted) + 0x01de8940 ) 0x49465f53 (dalvik-heap (deleted) + 0x05bd5f53 ) 0x65747378 (data@app@com.chrome.dev-2.apk@classes.dex + 0x001ab378 ) 0x69622f6c (libchrome.so -serialization_forward.h:82 ) blink::mojom::WebBluetoothServiceProxy::RemoteServerGetPrimaryServices(content::WebBluetoothDeviceId const&, blink::mojom::WebBluetoothGATTQueryQuantity, base::Optional<device::BluetoothUUID> const&, base::Callback<void (blink::mojom::WebBluetoothError, mojo::Array<mojo::InlinedStructPtr<blink::mojom::WebBluetoothRemoteGATTService> >), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) This Crash observed in below builds: 55.0.2868.0 11.63% 40 55.0.2860.0 17.44% 60 54.0.2840.34 3.49% 12 54.0.2840.25 10.17% 35 54.0.2840.16 2.33% 8 54.0.2830.0 33.72% 116 54.0.2826.2 20.06% 69 52.0.2743.98 0.58% 2 51.0.2704.81 0.58% 2 Here is the link with OS version,GPU Vendor and CPU architecture: https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Android%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27gpu-process%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27%5BGPU%20hang%5D%20base%3A%3Ainternal%3A%3ALockImpl%3A%3ALock%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,osversion,cpuarchitecture Crashes are seeing below versions of Android: 1 Android KitKat 77.91% 268 2 Android Jelly Bean 16.57% 57 3 Android Marshmallow 3.20% 11 4 Android Lollipop 2.03% 7 5 Android ??? 0.29% 1 Notes: 1.This is not a recent regression and this crash started in M51 (51.0.2704.81) 2.Observed 40 crashes from 33 different clients with 0.91% on latest canary and dev 55.0.2868.0 as of now. Latest changes done by bcwhite@ on file lock_impl_posix.cc https://chromium.googlesource.com/chromium/src/+/d9705964f525c278a5939b6fcd29a10f732149d0%5E%21/base/synchronization/lock_impl_posix.cc bcwhite@, could you please take a look and help us to find correct owner if it is not related your changes.
,
Sep 29 2016
If it's been occurring since M51, it's not my changes which are only present in M54 and above. If I remove the OS limitation, I see 88% of the crashes with that signature coming from "Chrome" (i.e. desktop). Is the stack symbolization valid? I know there have been issues with that lately. (https://bugs.chromium.org/p/chromium/issues/detail?id=647747) 0x68b4bb16 (libchrome.so -lock_impl_posix.cc:62 ) base::internal::LockImpl::Lock() 0x68b4bb65 (libchrome.so -lock_impl_posix.cc:68 ) base::internal::LockImpl::Unlock() I can't see Unlock() calling Lock() given than the former has only one line of code: int rv = pthread_mutex_unlock(&native_handle_); Assuming this really is a deadlock, petrcermak has done recent work on the MemoryPressure notifications so assigning to him.
,
Sep 29 2016
Most of those crashes are on x86. As we found out later, crashes on x86 are utterly broken and should not be trusted (see Issue 647747). However, if you filter out x86 you might get some more reasonable results.
,
Oct 2 2016
Users experienced this crash on the following builds: Android Beta 54.0.2840.42 - 1.09 CPM, 45 reports, 37 clients (signature [GPU hang] base::internal::LockImpl::Lock) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Oct 3 2016
I think that the deadlock is in the code dealing with synchronous observers added in https://codereview.chromium.org/1749073002/. The patch author is not a Googler; he's from Intel. Does anyone know if/how I can share this bug (with Restrict-View-Google label) with them?
,
Oct 3 2016
Assigning and cc-ing to CL reviewers.
,
Oct 22 2016
Users experienced this crash on the following builds: Android Dev 56.0.2895.3 - 1.94 CPM, 5 reports, 3 clients (signature [GPU hang] base::internal::LockImpl::Lock) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Oct 24 2016
Nothing about this bug seems private. I'm not sure what that issue is here, something is wrong inside v8 when we call the low Mem notification?
,
Oct 25 2016
The stack doesn't look right, however, the code on the stack is not related to v8, nor do I see how it would be possible to create a deadlock here. Primiano, can you do some magic with the stack to tell us what's wrong here? Or do we just happen to get a storm of low memory notifications and that keeps the GPU process busy? Note that there should also not be v8 running in a gpu process...
,
Oct 25 2016
- Dejavu, this seem to me very similar to Issue 622034 and Issue 580215 (which have similarly garbage stacks). +aelias - I don't see how v8 is related at all here. Am I missing some report? Can you point it out explicitly if that's the case? - Most of those crashes have non-chrome libs (libc.so) on the top of the stacktrace. I provided a thorough description in Issue 480835 about why this situation is broken and what we could do to fix it. I am not going to add anything more here. - There is something else broken in the triage process here: crash/ should have an indication of which stack unwinding method has been used and should show a red warning saying "this stacktrace is best effort, very likely is garbage" when falling back on stackwalk (stackwalk == bad, untrustworthy, FP == good, CFI == excellent). - This is a case for: stackwalk == bad. I glanced through the reports, they are really nonsense. For instance report dc68d85b00000000 starts with something reasonable (MemoryPressureListener) but then down on the same stack has disk_cache::SimpleBackendImpl::DoomEntry, blink::OfflineAudioContext::suspendContext. Which simply doesn't make sense. One possible action I can help with if we *really really* care about this bug is: somebody give me a dump of the /system/lib for the *extact signature* for one of those devices in the cluster. I can generate cfi for those system libraries and resymbolize the crash. If everything uses CFI we'd hopefully get a more reasonable crash dump. However, this costs time. Honestly I'm getting these manual symbolization on a ~weekly basis. I'd be really happy if we solved the general problem of having a reasonable crash reporting story on Android. Summary: until we have a reasonable crash reporting story on Android, all those kinds of crashes are impossible to work on. I reported this situation few weeks ago (+amineer).
,
Oct 25 2016
,
Oct 25 2016
There are 8 total crashes of this type in the latest M56 dev release, which puts it at ~#50 overall crash. If the stacktraces are garbage, I don't think it's worth burning primiano@'s time on them. Marking as WontFix.
,
Oct 25 2016
FYI debugging android gpu hangs, make sure you are looking at the right thread. The magic signature for reports that I looked here at aren't even on the gpu "main" thread, which on android is not thread 0. b/30505328 |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by sheriffbot@chromium.org
, Sep 28 2016