veyron_minnie: crash in vop_update_primary_plane during resume |
|||||||||
Issue descriptionChrome Version: 49.0.2623.75 beta Chrome OS Version: 7834.52.0 (Official Build) beta-channel veyron_minnie Chrome OS Platform: veyron_minnie Steps To Reproduce: (1) suspend (2) resume (3) Expected Result: No crash Actual Result: Crash. How frequently does this problem reproduce? (Always, sometimes, hard to reproduce?) Sometimes. What is the impact to the user, and is there a workaround? If so, what is it? Crash on resume. System reboots. Please provide any additional information below. Attach a screen shot or log if possible. Tomasz, can you take a look at this one? Feedback report: https://feedback.corp.google.com/product/208/neutron?lView=rd&lReport=6888217296 https://crash.corp.google.com/browse?q=ReportID%3D%27d912d87400000000%27 [334986.828219] calling display-subsystem.4+ @ 27422, parent: none, cb: platform_pm_resume [334986.828260] rockchip-vop ff940000.vop: Attached to iommu domain [334986.996651] rockchip-edp rockchip-edp: unable to config video [334986.996715] rockchip-edp rockchip-edp: vop LIT output to edp [334986.996818] Unable to handle kernel NULL pointer dereference at virtual address 00000044 [334986.996831] pgd = c4154000 [334986.996839] [00000044] *pgd=00000000 [334986.996853] Internal error: Oops: 5 [#1] SMP ARM [334986.996862] Modules linked in: nls_iso8859_1 nls_cp437 vfat fat rfcomm ip6t_REJECT hci_uart uinput i2c_dev bluetooth cdc_ether usbnet r8152 uvcvideo videobuf2_vmalloc zram fuse nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables cros_ec_accel kfifo_buf iio_trig_sysfs brcmfmac brcmutil cfg80211 joydev snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device ppp_async ppp_generic slhc tun [334986.997011] CPU: 1 PID: 27422 Comm: cat Tainted: G W 3.14.0 #1 [334986.997025] task: ed14e000 ti: c3138000 task.ti: c3138000 [334986.997041] PC is at vop_update_primary_plane+0x30/0x7c [334986.997053] LR is at vop_crtc_mode_set_base+0x24/0x50 [334986.997066] pc : [<c03b9f04>] lr : [<c03b9f74>] psr: 60000113 [334986.997066] sp : c3139b68 ip : c3139ba8 fp : c3139ba4 [334986.997082] r10: 00000174 r9 : 00000000 r8 : 0000005d [334986.997093] r7 : 001c033c r6 : 00000000 r5 : 00000000 r4 : ecb48018 [334986.997104] r3 : 00000000 r2 : 00000000 r1 : 00000000 r0 : ecb49388 [334986.997116] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [334986.997129] Control: 10c5387d Table: 0415406a DAC: 00000015 [334986.997139] ... [334986.999718] [<c03b9f04>] (vop_update_primary_plane) from [<c03b9f74>] (vop_crtc_mode_set_base+0x24/0x50) [334986.999737] [<c03b9f74>] (vop_crtc_mode_set_base) from [<c03ba524>] (vop_crtc_mode_set+0x584/0x654) [334986.999759] [<c03ba524>] (vop_crtc_mode_set) from [<c03931e4>] (drm_crtc_helper_set_mode+0x300/0x4e0) [334986.999780] [<c03931e4>] (drm_crtc_helper_set_mode) from [<c039342c>] (drm_helper_resume_force_mode+0x68/0x150) [334986.999800] [<c039342c>] (drm_helper_resume_force_mode) from [<c03b6714>] (rockchip_drm_sys_resume+0xb4/0xcc) [334986.999819] [<c03b6714>] (rockchip_drm_sys_resume) from [<c03ecd50>] (platform_pm_resume+0x50/0x5c) [334986.999839] [<c03ecd50>] (platform_pm_resume) from [<c03f2f5c>] (dpm_run_callback+0x48/0x84) [334986.999857] [<c03f2f5c>] (dpm_run_callback) from [<c03f3634>] (device_resume+0x158/0x19c) [334986.999874] [<c03f3634>] (device_resume) from [<c03f4500>] (dpm_resume+0xe8/0x220) [334986.999890] [<c03f4500>] (dpm_resume) from [<c03f4808>] (dpm_resume_end+0x1c/0x28) [334986.999908] [<c03f4808>] (dpm_resume_end) from [<c016716c>] (suspend_devices_and_enter+0x164/0x21c) [334986.999927] [<c016716c>] (suspend_devices_and_enter) from [<c0167330>] (pm_suspend+0x10c/0x22c) [334986.999943] [<c0167330>] (pm_suspend) from [<c0166010>] (state_store+0xb8/0xcc) [334986.999960] [<c0166010>] (state_store) from [<c0329c1c>] (kobj_attr_store+0x1c/0x28) [334986.999979] [<c0329c1c>] (kobj_attr_store) from [<c0270388>] (sysfs_kf_write+0x4c/0x58) [334986.999998] [<c0270388>] (sysfs_kf_write) from [<c0273684>] (kernfs_fop_write+0x100/0x148) [334987.000017] [<c0273684>] (kernfs_fop_write) from [<c02124a4>] (vfs_write+0xe4/0x178) [334987.000034] [<c02124a4>] (vfs_write) from [<c0212a88>] (SyS_write+0x64/0xbc) [334987.000053] [<c0212a88>] (SyS_write) from [<c01061e0>] (ret_fast_syscall+0x0/0x30) Looks like this crash has been present for a while. I see crashes containing 'vop_update_primary_plane' in the following versions, stretching back to 2015/11: 7647.84.0 23.33% 150 7647.78.0 8.55% 55 7647.73.0 13.69% 88 7520.67.0 11.66% 75 7520.63.0 12.29% 79 7520.62.0 2.95% 19 7520.55.0 1.24% 8 7390.68.0 21.62% 139 7390.61.0 0.93% 6 7262.52.0 1.56% 10
,
Jan 11 2017
Looks like nobody was able to take a look... Since it looks like the crashes are still quite common, I'm going to investigate.
,
Jan 11 2017
,
Jan 17 2017
This is third day of running suspend_stress test almost continuously on my minnie and I couldn't hit this issue. However I uploaded the following 3 CLs for prevention: https://chromium-review.googlesource.com/c/428770/ https://chromium-review.googlesource.com/c/428771/ https://chromium-review.googlesource.com/c/428772/ PTAL.
,
Jan 17 2017
+Doug, for his long history of ghost bug busting.
,
Jan 18 2017
,
Jan 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/804b7334dcf5825b948b1b5118f1800353ce6d13 commit 804b7334dcf5825b948b1b5118f1800353ce6d13 Author: Mark Yao <mark.yao@rock-chips.com> Date: Mon Jul 20 08:25:20 2015 UPSTREAM: drm/rockchip: vop: restore vop registers when resume The registers will be reset to default values when whole power domain off, so restore registers from regsbak. Signed-off-by: Mark Yao <mark.yao@rock-chips.com> (cherry picked from commit 77faa1619a5ae9ed600b0836bc1eec57bad1895b) BUG= chromium:595281 TEST=suspend_stress_test Change-Id: I47d60a92314e80551082b9268cbf4d5f6852e88f Signed-off-by: Tomasz Figa <tfiga@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/428770 Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> [modify] https://crrev.com/804b7334dcf5825b948b1b5118f1800353ce6d13/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
,
Jan 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/e31c7edc0cd95333f34cf0da055b6b7fec7a68ec commit e31c7edc0cd95333f34cf0da055b6b7fec7a68ec Author: Chris Zhong <zyw@rock-chips.com> Date: Sat Aug 27 03:39:38 2016 BACKPORT: FROMLIST: drm/rockchip: vop: make vop register setting take effect The setting of vop registers need a reg_done writing to take effect. In vop_enable the vop return to work by by restoring registers, but the registers do not take effect immediately, it should a vop_cfg_done after it. The same thing is needed by windows_disabled in vop_crtc_disable. Signed-off-by: Chris Zhong <zyw@rock-chips.com> (am from https://patchwork.kernel.org/patch/9302253/) BUG= chromium:595281 TEST=suspend_stress_test [tfiga: The call needed in vop_crtc_disable() in upstream, is not necessary in v3.14, because each plane is disabled separately with its own call to vop_cfg_done().] Change-Id: Ic10638b59779d581478498c00f98b1d1cd7e037c Signed-off-by: Tomasz Figa <tfiga@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/428771 Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> [modify] https://crrev.com/e31c7edc0cd95333f34cf0da055b6b7fec7a68ec/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
,
Jan 18 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/b8a2f4d004b7b435243aa70a616abaddc8323a7d commit b8a2f4d004b7b435243aa70a616abaddc8323a7d Author: Tomasz Figa <tfiga@chromium.org> Date: Tue Jan 17 09:12:59 2017 CHROMIUM: drm/rockchip: vop: Do not update primary plane without framebuffer For some reason vop_update_primary_plane() might get called without a framebuffer attached to primary plane. To avoid a crash from a NULL pointer dereference let's just bail out from the function. This should be safe, as initially after resume we should have all planes disabled in hardware. We should investigate the root cause, but since this seems to be a quite common crash on production devices and it does not seem to reproduce locally, we can work it around by this change. BUG= chromium:595281 TEST=suspend_stress_test Change-Id: I30c766869a627ce5a20d4e91e8e4d8b035631134 Signed-off-by: Tomasz Figa <tfiga@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/428772 Reviewed-by: Daniel Kurtz <djkurtz@chromium.org> [modify] https://crrev.com/b8a2f4d004b7b435243aa70a616abaddc8323a7d/drivers/gpu/drm/rockchip/rockchip_drm_vop.c
,
Jan 18 2017
Nice! We think this is fixed on R57, then? ...presumably we'll need to wait until it rolls out to beta channel and look for instances of this crash in crash/ ? ...then we can mark Verified?
,
Jan 19 2017
Well, at least it should be worked around, i.e. doesn't cause any unwanted behavior. Looking at the API, the function crashing shouldn't really be called with NULL crtc->primary->fb, so I'd suspect we have a bug somewhere caused by some of the backports to the core DRM code. I couldn't yet find anything relevant in the code, though...
,
Feb 23 2017
It's not reproducible in Chrome OS 9202.37.0, 57.0.2987.75. |
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by tfiga@chromium.org
, Mar 23 2016Status: Available (was: Assigned)