GPU Hang after MediaCodec.setSurface() on some Marshmallow+ devices. |
||||||||||
Issue descriptionhttps://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Android%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27gpu-process%27%20AND%20custom_data.ChromeCrashProto.gpu.gl_vendor%3D%27ARM%27%20AND%20custom_data.ChromeCrashProto.gpu.gl_renderer%3D%27Mali-450%20MP%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27GPU%20hang%27%20AND%20product.Version%3D%2756.0.2924.59%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,processuptime,devicemodel Mostly on M, mostly on ARM, mostly this one huawei device which we don't have. Most has this log ~15 seconds (the watchdog timeout) before the crash: MediaCodec: [HTTP://WEBADDRESS.ELIDED] setting surface generation to 27443217 Could this be oom as well (ie same as crbug.com/600454), or something else?
,
Jan 21 2017
blacklist-able?
,
Jan 21 2017
Yeah, we'll have to plumb some blacklist bits, but is otherwise fairly easy since it's M+ only: https://cs.chromium.org/chromium/src/media/filters/gpu_video_decoder.cc?l=229 You filtered the crashes to Mali-450MP, any reason why? If we blacklist all M,N Mali-450MP that will affect ~0.7% of M,N watch time.
,
Jan 21 2017
> You filtered the crashes to Mali-450MP, any reason why? Err... pasted wrong link? Removing ARM and Mali, 344 out of 383 reports contain libstagefright_foundation.so https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Android%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27gpu-process%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27GPU%20hang%27%20AND%20product.Version%3D%2756.0.2924.59%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D vs that + libstagefright_foundation.so: https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Android%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27gpu-process%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27GPU%20hang%27%20AND%20product.Version%3D%2756.0.2924.59%27%20OMIT%20RECORD%20IF%20SUM(thread.StackTrace.StackFrame.Module.CodeFile%3D%27%2Fsystem%2Flib%2Flibstagefright_foundation.so%27)%20%3D%200&ignore_case=false&enable_rewrite=true&omit_field_name=thread.StackTrace.StackFrame.Module.CodeFile&omit_field_value=%2Fsystem%2Flib%2Flibstagefright_foundation.so&omit_field_opt=%3D#samplereports
,
Jan 21 2017
So should be ok to block m57 then?
,
Jan 21 2017
Sure, we can figure out some blacklisting here. In case it's a video specific issue, here's the one SFW video I could find in crash URLs: https://endlessvideo.com/watch?v=kiYVifQ90kE
,
Jan 21 2017
,
Jan 24 2017
Looks like the #1 failing device is the ALE-L21 (actually the whole series of L21 devices account for 61% of crashes). We've had issues with this device in the past. I think liberato@ was going to order one per issue 650863. liberato@, did you ever order this? I'll grab it from you tomorrow if so.
,
Jan 24 2017
I didn't. I think that i fixed 650863 then promptly forgot to order the phone. Sorry about that.
,
Jan 24 2017
liberato@ hadn't ordered one, I have one on the way and will be here Friday. Will see if I can repro and if this is an issue we can workaround or if we need to blacklist it. Given the long tail (~40%) of crashes, I'm hoping there is something we could do to prevent this hang.
,
Jan 31 2017
Just got the device updated to M, ALE-L21 + setSurface == immediate failure. MediaCodec gives and takes a few buffers after the switch but then hangs shortly after and never returns any more input or output buffers. Seems like it's straight up broken. CTS test helpfully says "TODO" :| https://android.googlesource.com/platform/cts/+/master/tests/tests/media/src/android/media/cts/AdaptivePlaybackTest.java#918 Will look into blacklisting options.
,
Jan 31 2017
Looks like we'll have to go with blacklisting by Mali-450MP, just using *-L21 will only capture ~45% of the crashes versus ~72% with the GL renderer filter. Unfortunately to do that I need to plumb a gpu preferences bit since we can't easily access the GL renderer value from media/filters. Will start on that tomorrow.
,
Feb 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8a5efb12a76b24de92a20a23f56d20aa9e22338b commit 8a5efb12a76b24de92a20a23f56d20aa9e22338b Author: dalecurtis <dalecurtis@chromium.org> Date: Tue Feb 07 02:17:21 2017 Make setOutputSurface, encrypted capabilities. Blacklist hi6210sft. No need for platform specific define for encrypted hardware decode support when we have a capability system; thus make support for encrypted streams a capability. We now have a device with broken MediaCodec.setOutputSurface(), so make it a capability and blacklist the hardware platform common to 99% of the crashes (hi6210sft). BUG= 683401 TEST=p8lite no longer hangs on fullscreen transition. CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Review-Url: https://codereview.chromium.org/2677983004 Cr-Commit-Position: refs/heads/master@{#448514} [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/base/android/java/src/org/chromium/media/MediaCodecUtil.java [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/base/android/media_codec_util.cc [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/base/android/media_codec_util.h [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/filters/gpu_video_decoder.cc [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/gpu/android_video_decode_accelerator.cc [modify] https://crrev.com/8a5efb12a76b24de92a20a23f56d20aa9e22338b/media/video/video_decode_accelerator.h
,
Feb 11 2017
Looks good on dev, marking as merge request.
,
Feb 11 2017
Your change meets the bar and is auto-approved for M57. Please go ahead and merge the CL to branch 2987 manually. Please contact milestone owner if you have questions. Owners: amineer@(clank), cmasso@(bling), ketakid@(cros), govind@(desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Feb 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e3f399dec4ba3384929172a2ca92ab92a2a5132e commit e3f399dec4ba3384929172a2ca92ab92a2a5132e Author: Dale Curtis <dalecurtis@chromium.org> Date: Sat Feb 11 01:01:24 2017 Merge M57: "Make setOutputSurface, encrypted capabilities. Blacklist hi6210sft." No need for platform specific define for encrypted hardware decode support when we have a capability system; thus make support for encrypted streams a capability. We now have a device with broken MediaCodec.setOutputSurface(), so make it a capability and blacklist the hardware platform common to 99% of the crashes (hi6210sft). BUG= 683401 TEST=p8lite no longer hangs on fullscreen transition. CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel Review-Url: https://codereview.chromium.org/2677983004 Cr-Commit-Position: refs/heads/master@{#448514} (cherry picked from commit 8a5efb12a76b24de92a20a23f56d20aa9e22338b) Review-Url: https://codereview.chromium.org/2693653003 . Cr-Commit-Position: refs/branch-heads/2987@{#459} Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943} [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/base/android/java/src/org/chromium/media/MediaCodecUtil.java [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/base/android/media_codec_util.cc [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/base/android/media_codec_util.h [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/filters/gpu_video_decoder.cc [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/gpu/android_video_decode_accelerator.cc [modify] https://crrev.com/e3f399dec4ba3384929172a2ca92ab92a2a5132e/media/video/video_decode_accelerator.h
,
Feb 11 2017
We can merge back to M56 if there's going to be another update, it's currently ~12% of GPU crashes on stable. +amineer to make the call.
,
Feb 15 2017
I'll apply a Merge-Review-56 tag to this to keep it on my radar for respins, but as of now, nothing planned.
,
Mar 9 2017
No more 56. |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by dalecur...@chromium.org
, Jan 20 2017