New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 664406 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression



Sign in to add a comment

[nyan_big] Kernel crash - Device reboots when suspending by closing the lid since R56-8973.0.0(Wed Nov 09)

Project Member Reported by ka...@chromium.org, Nov 11 2016

Issue description

Few test suites show nyan_big devices reboot on close-open lid.
https://screenshot.googleplex.com/zjqaTU7BL0k
https://screenshot.googleplex.com/AVzTtBJ2ydJ

Sample test logs at: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/85026933-chromeos-test/chromeos1-row2-rack4-host6/

Kernel crash is observed. Attaching crash file.

ChromeOS diff: https://crosland.corp.google.com/log/8972.0.0..8973.0.0
Could https://chromium-review.googlesource.com/#/c/408664/ be the cause?
 
kernel.20161110.162143.0.kcrash
120 KB Download

Comment 1 by ka...@chromium.org, Nov 11 2016

Cc: h...@chromium.org

Comment 2 by ka...@chromium.org, Nov 11 2016

Cc: dbehr@chromium.org
Labels: M-56
Reproducible in 8975.0.0/56.0.2907.0 on big on my desk. 

Crash id: a038cd0500000000

log:    https://pantheon.corp.google.com/storage/browser/chromiumos-test-logs/bugfiles/cr/664406/

Comment 4 by h...@chromium.org, Nov 11 2016

Cc: marc...@chromium.org
From console-ramoops

[   94.031475] kernel BUG at /mnt/host/source/src/third_party/kernel/v3.10/drivers/gpu/drm/drm_crtc.c:498!
[   94.031483] Internal error: Oops - BUG: 0 [#1] SMP ARM
[   94.031490] Modules linked in: uinput snd_hda_codec_hdmi snd_soc_tegra30_i2s snd_soc_tegra_pcm uvcvideo snd_soc_tegra_max98090 snd_soc_tegra_utils videobuf2_vmalloc videobuf2_memops videobuf2_core snd_hda_tegra snd_hda_controller snd_hda_codec snd_soc_tegra30_ahub rfcomm i2c_dev snd_soc_max98090 fuse zram(C) nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device mwifiex_sdio mwifiex btmrvl_sdio btmrvl cfg80211 bluetooth cdc_ether usbnet joydev
[   94.031636] CPU: 0 PID: 1200 Comm: DrmThread Tainted: G         C   3.10.18 #13
[   94.031644] task: eda36400 ti: ecaa6000 task.ti: ecaa6000
[   94.031656] PC is at drm_framebuffer_free_bug+0x14/0x18
[   94.031663] LR is at __drm_framebuffer_unreference+0x64/0x78
[   94.031670] pc : [<c04d736c>]    lr : [<c04d73d4>]    psr: 60010013
[   94.031670] sp : ecaa7d98  ip : ecaa7da8  fp : ecaa7da4
[   94.031678] r10: 000000af  r9 : 00000018  r8 : ed806a30
[   94.031685] r7 : ed806800  r6 : ed806800  r5 : ed8069e4  r4 : ec076880
[   94.031691] r3 : 00000000  r2 : 00000000  r1 : c097d5e0  r0 : ec076884
[   94.031699] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   94.031705] Control: 30c5387d  Table: adb9f280  DAC: fffffffd
[   94.031712] Process DrmThread (pid: 1200, stack limit = 0xecaa6240)
[   94.031719] Stack: (0xecaa7d98 to 0xecaa8000)
[   94.031726] 7d80:                                                       ecaa7dc4 ecaa7da8
[   94.031733] 7da0: c04d73d4 c04d7364 00000000 ed806800 ed806a30 ec076880 ecaa7de4 ecaa7dc8
[   94.031741] 7dc0: c04d7434 c04d737c ed16e61c ec076880 ed16e600 ed16e67c ecaa7e0c ecaa7de8
[   94.031749] 7de0: c04db380 c04d73f4 c04db2e0 00000004 ecaa7e44 ed16e600 ed806800 c00464af
[   94.031757] 7e00: ecaa7ef4 ecaa7e10 c04cd4fc c04db2ec 000004b0 0000e201 00000001 c097b57c
[   94.031765] 7e20: ee4b63f4 00000000 ecaa7e6c ecaa6010 ed806848 acc9d31c c04db2e0 c0848f54
[   94.031772] 7e40: 80400034 00000018 0000010b ed5ea130 c03152e4 ed5ea130 00006191 022df000
[   94.031779] 7e60: eedc0990 eedc0990 ecaa7e9c ecaa7e78 c02fd590 c02fd14c ed5ea130 eedc0990
[   94.031786] 7e80: 00000000 ed5ea184 00000008 eedc0990 ecaa7eb4 ecaa7ea0 c03152e4 c0685bac
[   94.031793] 7ea0: 00100020 ed5ea130 ecaa7ecc ecaa7eb8 c0315380 c03152b0 00000007 271ae70e
[   94.031801] 7ec0: 40000028 271ae70e ecaa7f54 acc9d31c ed855838 edacd000 acc9d31c c00464af
[   94.031808] 7ee0: 00000001 00000047 ecaa7f6c ecaa7ef8 c031277c c04cd1c0 ecaa7f14 ecaa7f08
[   94.031815] 7f00: c031e4a4 c0e083c8 ecaa7f54 ecaa7f18 c0302fbc c031e47c 00000020 00000000
[   94.031822] 7f20: ec883f00 ec883f08 ecaa7f54 271ae70e ecaa6000 7fff0000 eeb87000 271ae70e
[   94.031829] 7f40: 00000036 00000000 c0e083c8 edacd000 acc9d31c c00464af 00000001 00000047
[   94.031836] 7f60: ecaa7fa4 ecaa7f70 c0312900 c03122e0 00000001 271ae70e c0208850 00000000
[   94.031843] 7f80: acc9d31c c00464af 00000036 c02061a4 ecaa6000 00000800 00000000 ecaa7fa8
[   94.031850] 7fa0: c020618c c031289c 00000000 acc9d31c 00000047 c00464af acc9d31c 00000000
[   94.031857] 7fc0: 00000000 acc9d31c c00464af 00000036 00000047 00000018 b84c1c00 00000ddf
[   94.031864] 7fe0: b04c2f38 acc9d2fc b04ba52f b026c206 60010030 00000047 ac127408 0df0d99d
[   94.031878] [<c04d736c>] (drm_framebuffer_free_bug+0x14/0x18) from [<c04d73d4>] (__drm_framebuffer_unreference+0x64/0x78)
[   94.031887] [<c04d73d4>] (__drm_framebuffer_unreference+0x64/0x78) from [<c04d7434>] (__drm_framebuffer_unregister+0x4c/0x50)
[   94.031897] [<c04d7434>] (__drm_framebuffer_unregister+0x4c/0x50) from [<c04db380>] (drm_mode_rmfb+0xa0/0xf4)
[   94.031907] [<c04db380>] (drm_mode_rmfb+0xa0/0xf4) from [<c04cd4fc>] (drm_ioctl+0x348/0x4d8)
[   94.031922] [<c04cd4fc>] (drm_ioctl+0x348/0x4d8) from [<c031277c>] (do_vfs_ioctl+0x4a8/0x5bc)
[   94.031933] [<c031277c>] (do_vfs_ioctl+0x4a8/0x5bc) from [<c0312900>] (SyS_ioctl+0x70/0xa8)
[   94.031944] [<c0312900>] (SyS_ioctl+0x70/0xa8) from [<c020618c>] (__sys_trace_return+0x0/0x14)
[   94.031953] Code: e92dd800 e24cb004 e52de004 e8bd4000 (e7f001f2) 
[   94.031962] ---[ end trace bbe0407e87b1ced9 ]---
[   94.034018] Kernel panic - not syncing: Fatal exception

Comment 5 by h...@chromium.org, Nov 11 2016

Labels: ReleaseBlock-Dev
Owner: h...@chromium.org
Status: Assigned (was: Untriaged)

Comment 6 by h...@chromium.org, Nov 11 2016

Regarding the first question in the bug report by kalin@

"Could https://chromium-review.googlesource.com/#/c/408664/ be the cause?"

This is not possible. That change only modified the string containing the version number (from "24.01" to "24.02"), that's all. It did not in fact update the driver to a different binary or change any functionality.

If we know for sure that R56-8973.0.0 was the first build that had the problem then I would start bisecting kernel 3.10 changes. There are about 10 of them.

Comment 7 by h...@chromium.org, Nov 11 2016

Labels: -Type-Bug Type-Bug-Regression

Comment 8 by h...@chromium.org, Nov 11 2016

Cc: gurcheta...@chromium.org
gurchetan@ -- I have finished bisecting the 9 kernel CLs. It looks like the culprit is this patch.

commit 4b44bc14603ac35032b60b808f2d3b8e98a85121
Author: Gurchetan Singh <gurchetansingh@chromium.org>
Date:   Wed Nov 2 18:13:58 2016 -0700

    CHROMIUM: drm: Replace crtc fb with primary plane fb in v3.10
    
    This applies CL:195617 to the v3.10 kernel.  Here's what was done:
    
    "This patch uses the coccinelle rules from commit f4510a275 to replace
    crtc->fb with crtc->primary->fb.
    
    For completeness, the rules are:
    
    @@ struct drm_crtc C; @@
    -   (C).fb
    +   C.primary->fb
    @@ struct drm_crtc *C; @@
    -   (C)->fb
    +   C->primary->fb
    @@ struct drm_mode_set C; @@
    -   (C).crtc->fb
    +   C.crtc->primary->fb
    @@ struct drm_mode_set *C; @@
    -   (C)->crtc->fb
    +   C->crtc->primary->fb
    
    To run the script by hand:
    spatch -sp_file ${RULES_FILE} --in-place --include-headers \
    --all-includes -dir ${KERNEL_DIR} -I ${KERNEL_DIR}/include \
    -I ${KERNEL_DIR}/drivers/gpu/drm"
    
    BUG= chromium:661400 
    TEST=Tested on glimmer and nyan.
    
    Checked if Chrome boots and ran modetest.
    The returned planes are reported as overlays.
    
    Change-Id: Ifd7b734dae2447c933b47e8f4413f3c580486013
    Reviewed-on: https://chromium-review.googlesource.com/407605
    Commit-Ready: Gurchetan Singh <gurchetansingh@chromium.org>
    Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
    Reviewed-by: Dominik Behr <dbehr@chromium.org>


Owner: gurcheta...@chromium.org
If this doesn't happen on glimmer, we probably missing additional nyan backports.  Taking a loot
Steps to reproduce:

1) Log into the board
2) Close the lid.
3) Wait 5 secs
4) Open the lid, see reboot


Couldn't reproduce on glimmer, can on nyan_big.  Maybe we need the tegra analogue of

https://chromium-review.googlesource.com/#/c/407597/

??
Is this issue seen in the latest ToT builds?
Yes, this issue is seen on ToT builds.  https://patchwork.kernel.org/patch/5119071/ should fix this.  Need to cherry-pick other patches to make history nice though.
Project Member

Comment 13 by bugdroid1@chromium.org, Nov 12 2016

Labels: merge-merged-chromeos-3.10
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/ef34b5842c53dec2947da1ae0aaaae6f7d0add12

commit ef34b5842c53dec2947da1ae0aaaae6f7d0add12
Author: Daniel Vetter <daniel.vetter@ffwll.ch>
Date: Wed Apr 23 13:15:32 2014

BACKPORT: drm/tegra: restrict plane loops to legacy planes

In Matt Ropers primary plane series a set of prep patches like

commit af2b653bfb4ef40931b4d101ca842ce0c5da57ef
Author: Matt Roper <matthew.d.roper@intel.com>
Date:   Tue Apr 1 15:22:32 2014 -0700

    drm/i915: Restrict plane loops to only operate on overlay planes (v2)

ensured that all exisiting users of the mode_config->plane_list
wouldn't change behaviour. Unfortunately tegra seems to have fallen
through the cracks. Fix it.

This regression was introduced in

commit e13161af80c185ecd8dc4641d0f5df58f9e3e0af
Author: Matt Roper <matthew.d.roper@intel.com>
Date:   Tue Apr 1 15:22:38 2014 -0700

    drm: Add drm_crtc_init_with_planes() (v2)

The result was that we've unref'ed the fb for the primary plane twice,
leading to a use-after free bug. This is because the drm core will
already set crtc->primary->fb to NULL and do the unref for us, and the
crtc disable hook is called by the drm crtc helpers for exactly this
case.

Aside: Now that the fbdev helpers clean up planes there's no longer a
need to do this in drivers. So this could probably be nuked entirely
in linux-next.

Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Thierry Reding <treding@nvidia.com>

Conflicts:
	drivers/gpu/drm/tegra/dc.c

BUG= chromium:661400 ,  chromium:664406 
TEST=nyan_big doesn't restart after closing the lid when logged into
     Chrome

(cherry picked from commit 2b4c36612efac173397756398000921a7771fdda)
Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>

Change-Id: Ia6485202de4d091d52e04ce492431d6fcd53ace3
Reviewed-on: https://chromium-review.googlesource.com/410272
Commit-Ready: Gurchetan Singh <gurchetansingh@chromium.org>
Tested-by: Gurchetan Singh <gurchetansingh@chromium.org>
Reviewed-by: Haixia Shi <hshi@chromium.org>

[modify] https://crrev.com/ef34b5842c53dec2947da1ae0aaaae6f7d0add12/drivers/gpu/drm/tegra/dc.c

Comment 15 by ka...@chromium.org, Nov 14 2016

Status: Verified (was: Fixed)
Right. Thanks for the fast resolution.
Marking as Verified.

Sign in to add a comment