New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 892760 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Oct 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

Fleex: kernel lockdep error during boot

Project Member Reported by yuzhao@chromium.org, Oct 5

Issue description

Chrome OS Version: 11102.0
Chrome OS Platform: Fleex

Please specify Cr-* of the system to which this bug/feature applies (add
the label below).

Steps To Reproduce:
(1) Enable lockdep when building kernel
(2) boot
(3)

Expected Result:
No lockdep error

Actual Result:
[   14.635895] ======================================================
[   14.642783] WARNING: possible circular locking dependency detected
[   14.649688] 4.14.73 #5 Not tainted
[   14.653491] ------------------------------------------------------
[   14.660380] wpa_supplicant/621 is trying to acquire lock:
[   14.666395]  (&sb->s_type->i_mutex_key#4){++++}, at: [<ffffffff82651d8f>] start_creating+0xb2/0x163
[   14.676499] 
[   14.676499] but task is already holding lock:
[   14.683014]  (&mvm->mutex){+.+.}, at: [<ffffffffc06beab6>] iwl_mvm_mac_add_interface+0xc1/0x802 [iwlmvm]
[   14.693636] 
[   14.693636] which lock already depends on the new lock.
[   14.693636] 
[   14.702752] 
[   14.702752] the existing dependency chain (in reverse order) is:
[   14.711097] 
[   14.711097] -> #7 (&mvm->mutex){+.+.}:
[   14.716934]        __mutex_lock_common+0x145/0x1854
[   14.722376]        __mutex_lock+0x16/0x1c
[   14.726872]        iwl_mvm_tzone_get_temp+0x9f/0x1f8 [iwlmvm]
[   14.733298]        thermal_zone_get_temp+0xe5/0x105
[   14.738748]        thermal_zone_device_update+0xe9/0x5db
[   14.744683]        thermal_zone_device_register+0xa94/0xf2b
[   14.750915]        iwl_mvm_thermal_initialize+0x3e2/0x57b [iwlmvm]
[   14.757839]        iwl_op_mode_mvm_start+0x1d7d/0x2697 [iwlmvm]
[   14.764476]        _iwl_op_mode_start+0x159/0x1f9 [iwlwifi]
[   14.770714]        iwl_opmode_register+0xb0/0x120 [iwlwifi]
[   14.776950]        iio_find_channel_from_si+0x2c/0x10e [industrialio]
[   14.784147]        do_one_initcall+0x281/0x458
[   14.789117]        do_init_module+0x1a8/0x42c
[   14.793973]        load_module+0x5d2a/0x6f36
[   14.798734]        SyS_finit_module+0x23c/0x266
[   14.803797]        do_syscall_64+0x1af/0x202
[   14.808555]        entry_SYSCALL_64_after_hwframe+0x42/0xb7
[   14.814770] 
[   14.814770] -> #6 (&tz->lock){+.+.}:
[   14.820424]        __mutex_lock_common+0x145/0x1854
[   14.825864]        __mutex_lock+0x16/0x1c
[   14.830335]        thermal_zone_get_temp+0x97/0x105
[   14.835771]        thermal_zone_device_update+0xe9/0x5db
[   14.841695]        thermal_zone_device_register+0xa94/0xf2b
[   14.847923]        pkg_thermal_cpu_online+0x40d/0x608
[   14.853568]        cpuhp_invoke_callback+0x337/0x75d
[   14.859115]        cpuhp_thread_fun+0x311/0x65f
[   14.864178]        smpboot_thread_fn+0x579/0x8ab
[   14.869337]        kthread+0x272/0x287
[   14.873529]        ret_from_fork+0x3a/0x50
[   14.878095] 
[   14.878095] -> #5 (cpuhp_state-up){+.+.}:
[   14.884231]        cpuhp_lock_acquire+0x33/0x36
[   14.889299]        cpuhp_issue_call+0x11c/0x3f9
[   14.894364]        __cpuhp_setup_state_cpuslocked+0x149/0x24e
[   14.900784]        __cpuhp_setup_state+0x43/0x5b
[   14.905951]        page_writeback_init+0x38/0x55
[   14.911112]        start_kernel+0x6f9/0x801
[   14.915793]        secondary_startup_64+0xa5/0xb0
[   14.921049] 
[   14.921049] -> #4 (cpuhp_state_mutex){+.+.}:
[   14.927479]        __mutex_lock_common+0x145/0x1854
[   14.932922]        __mutex_lock+0x16/0x1c
[   14.937395]        __cpuhp_setup_state_cpuslocked+0x5c/0x24e
[   14.943721]        __cpuhp_setup_state+0x43/0x5b
[   14.948884]        page_alloc_init+0x26/0x30
[   14.953650]        start_kernel+0x2c6/0x801
[   14.958317]        secondary_startup_64+0xa5/0xb0
[   14.963561] 
[   14.963561] -> #3 (cpu_hotplug_lock.rw_sem){++++}:
[   14.970579]        cpus_read_lock+0x39/0x8f
[   14.975241]        apply_wqattrs_lock+0xe/0x1d
[   14.980197]        apply_workqueue_attrs+0x17/0x30
[   14.985548]        __alloc_workqueue_key+0x615/0xadb
[   14.991083]        i915_gem_init_userptr+0x95/0xd1
[   14.996438]        i915_gem_init+0xf5/0x1a2
[   15.001101]        i915_driver_load+0x1b84/0x3073
[   15.006358]        pci_device_probe+0x1a5/0x2e3
[   15.011411]        driver_probe_device+0x54c/0x88f
[   15.016762]        __driver_attach+0xf8/0x174
[   15.021622]        bus_for_each_dev+0x129/0x185
[   15.026684]        bus_add_driver+0x2c9/0x547
[   15.031546]        driver_register+0x21d/0x29f
[   15.036503]        do_one_initcall+0x281/0x458
[   15.041477]        kernel_init_freeable+0x413/0x569
[   15.046921]        kernel_init+0x11/0x116
[   15.051395]        ret_from_fork+0x3a/0x50
[   15.055961] 
[   15.055961] -> #2 (&dev->struct_mutex){+.+.}:
[   15.062486]        __mutex_lock_common+0x145/0x1854
[   15.067939]        __mutex_lock+0x16/0x1c
[   15.072426]        i915_mutex_lock_interruptible+0x1b2/0x21a
[   15.078737]        i915_gem_fault+0x356/0xe41
[   15.083607]        __do_fault+0x79/0x115
[   15.087995]        handle_mm_fault+0x1839/0x2070
[   15.093152]        __do_page_fault+0x404/0x815
[   15.098129]        page_fault+0x45/0x50
[   15.102404] 
[   15.102404] -> #1 (&mm->mmap_sem){++++}:
[   15.108447]        __might_fault+0xcb/0x119
[   15.113102]        _copy_to_user+0x23/0x9a
[   15.117672]        filldir+0x1dc/0x43b
[   15.121866]        dir_emit_dots+0x10d/0x264
[   15.126629]        dcache_readdir+0x71/0x26b
[   15.131402]        iterate_dir+0x1fd/0x407
[   15.135969]        SyS_getdents+0x212/0x3cd
[   15.140634]        do_syscall_64+0x1af/0x202
[   15.145392]        entry_SYSCALL_64_after_hwframe+0x42/0xb7
[   15.151613] 
[   15.151613] -> #0 (&sb->s_type->i_mutex_key#4){++++}:
[   15.158912]        lock_acquire+0x297/0x346
[   15.163585]        down_write+0x39/0x76
[   15.167870]        start_creating+0xb2/0x163
[   15.172628]        debugfs_create_dir+0x17/0x197
[   15.177813]        iwl_mvm_vif_dbgfs_register+0x14d/0xb53 [iwlmvm]
[   15.184723]        iwl_mvm_mac_add_interface+0x622/0x802 [iwlmvm]
[   15.191582]        drv_add_interface+0x2e9/0x3fd [iwl7000_mac80211]
[   15.198629]        ieee80211_do_open+0x37d/0x15c7 [iwl7000_mac80211]
[   15.205721]        __dev_open+0xf6/0x1d5
[   15.210111]        __dev_change_flags+0x184/0x48a
[   15.215378]        dev_change_flags+0x78/0x166
[   15.220340]        devinet_ioctl+0x562/0x121c
[   15.225211]        sock_do_ioctl+0x72/0x99
[   15.229780]        sock_ioctl+0x316/0x34e
[   15.234252]        vfs_ioctl+0x72/0x91
[   15.238442]        do_vfs_ioctl+0x872/0x1090
[   15.243214]        SyS_ioctl+0x52/0x77
[   15.247403]        do_syscall_64+0x1af/0x202
[   15.252162]        entry_SYSCALL_64_after_hwframe+0x42/0xb7
[   15.258384] 
[   15.258384] other info that might help us debug this:
[   15.258384] 
[   15.267315] Chain exists of:
[   15.267315]   &sb->s_type->i_mutex_key#4 --> &tz->lock --> &mvm->mutex
[   15.267315] 
[   15.279449]  Possible unsafe locking scenario:
[   15.279449] 
[   15.286060]        CPU0                    CPU1
[   15.291116]        ----                    ----
[   15.296178]   lock(&mvm->mutex);
[   15.299796]                                lock(&tz->lock);
[   15.306025]                                lock(&mvm->mutex);
[   15.312440]   lock(&sb->s_type->i_mutex_key#4);
[   15.317500] 
[   15.317500]  *** DEADLOCK ***
[   15.317500] 
[   15.324109] 2 locks held by wpa_supplicant/621:
[   15.329162]  #0:  (rtnl_mutex){+.+.}, at: [<ffffffff836cfe8a>] devinet_ioctl+0x23f/0x121c
[   15.338316]  #1:  (&mvm->mutex){+.+.}, at: [<ffffffffc06beab6>] iwl_mvm_mac_add_interface+0xc1/0x802 [iwlmvm]
[   15.349422] 
[   15.349422] stack backtrace:
[   15.354292] CPU: 1 PID: 621 Comm: wpa_supplicant Not tainted 4.14.73 #5
[   15.361679] Hardware name: Google Fleex/Fleex, BIOS Google_Fleex.11102.0.2018_09_26_1558 09/25/2018
[   15.371792] Call Trace:
[   15.374533]  dump_stack+0xe2/0x148
[   15.378347]  ? _atomic_dec_and_lock+0x1ad/0x1ad
[   15.383402]  ? print_circular_bug+0x28f/0x2c8
[   15.388270]  __lock_acquire+0x23a9/0x2990
[   15.392743]  ? lock_acquire+0x346/0x346
[   15.397036]  ? rcu_read_lock_sched_held+0xab/0x15b
[   15.402381]  ? __lock_is_held+0x68/0xe6
[   15.406659]  ? lock_release+0x7ec/0x7ec
[   15.410939]  ? cpumask_test_cpu+0xcf/0xcf
[   15.415433]  lock_acquire+0x297/0x346
[   15.419530]  ? start_creating+0xb2/0x163
[   15.423909]  ? lock_downgrade+0x601/0x601
[   15.428387]  down_write+0x39/0x76
[   15.432108]  ? start_creating+0xb2/0x163
[   15.436490]  start_creating+0xb2/0x163
[   15.440690]  debugfs_create_dir+0x17/0x197
[   15.445292]  iwl_mvm_vif_dbgfs_register+0x14d/0xb53 [iwlmvm]
[   15.451642]  ? iwl_free_resp+0x6a/0x6a [iwlmvm]
[   15.456714]  ? lockdep_init_map+0x11a/0x595
[   15.461386]  ? perf_fetch_caller_regs+0xa8/0xa8
[   15.466439]  ? trace_softirqs_off+0x419/0x419
[   15.471299]  ? init_timer_key+0x168/0x1dd
[   15.475781]  iwl_mvm_mac_add_interface+0x622/0x802 [iwlmvm]
[   15.482087]  drv_add_interface+0x2e9/0x3fd [iwl7000_mac80211]
[   15.488558]  ? trace_drv_return_void+0x1e0/0x1e0 [iwl7000_mac80211]
[   15.495610]  ieee80211_do_open+0x37d/0x15c7 [iwl7000_mac80211]
[   15.502115]  __dev_open+0xf6/0x1d5
[   15.505915]  __dev_change_flags+0x184/0x48a
[   15.510582]  ? dev_get_flags+0x17b/0x17b
[   15.514956]  ? lock_release+0x7ec/0x7ec
[   15.519231]  dev_change_flags+0x78/0x166
[   15.523606]  devinet_ioctl+0x562/0x121c
[   15.527891]  ? inet_ifa_byprefix+0x125/0x125
[   15.532661]  sock_do_ioctl+0x72/0x99
[   15.536650]  sock_ioctl+0x316/0x34e
[   15.540549]  vfs_ioctl+0x72/0x91
[   15.544161]  do_vfs_ioctl+0x872/0x1090
[   15.548357]  ? selinux_file_ioctl+0x347/0x444
[   15.553221]  ? ioctl_preallocate+0x22d/0x22d
[   15.557991]  ? selinux_file_free_security+0x68/0x68
[   15.563452]  ? __sys_recvmsg+0x129/0x1f5
[   15.567833]  ? trace_hardirqs_on_caller+0x218/0x6d0
[   15.573278]  ? security_file_ioctl+0x80/0x97
[   15.578038]  SyS_ioctl+0x52/0x77
[   15.581640]  do_syscall_64+0x1af/0x202
[   15.585837]  ? ioctl_file_clone+0x208/0x208
[   15.590524]  entry_SYSCALL_64_after_hwframe+0x42/0xb7
[   15.596164] RIP: 0033:0x7eb052a65967
[   15.600166] RSP: 002b:00007ffcf2057dd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[   15.608630] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007eb052a65967
[   15.616601] RDX: 00007ffcf2057de0 RSI: 0000000000008914 RDI: 0000000000000009
[   15.624563] RBP: 00007ffcf2057e30 R08: 00005ab3357befe0 R09: 00007eb052d1bbd8
[   15.632534] R10: 0000000000000031 R11: 0000000000000246 R12: 0000000000000009
[   15.640502] R13: 00005ab3357c2770 R14: 0000000000000001 R15: 00005ab3357c2790


How frequently does this problem reproduce? (Always, sometimes, hard to
reproduce?)
Always

What is the impact to the user, and is there a workaround? If so, what is
it?

Please provide any additional information below. Attach a screen shot or
log if possible.

For graphics-related bugs, please copy/paste the contents of the about:gpu
page at the end of this report.

 
crbug.com/833992 looks similar.


Project Member

Comment 2 by bugdroid1@chromium.org, Oct 18

Labels: merge-merged-chromeos-4.14
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/faa1045a76826b1516b15342133b5892529a226e

commit faa1045a76826b1516b15342133b5892529a226e
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date: Thu Oct 18 21:53:20 2018

UPSTREAM: drm/i915: Call i915_gem_init_userptr() before taking struct_mutex

We don't need struct_mutex to initialise userptr (it just allocates a
workqueue for itself etc), but we do need struct_mutex later on in
i915_gem_init() in order to feed requests onto the HW.

This should break the chain

[  385.697902] ======================================================
[  385.697907] WARNING: possible circular locking dependency detected
[  385.697913] 4.14.0-CI-Patchwork_7234+ #1 Tainted: G     U
[  385.697917] ------------------------------------------------------
[  385.697922] perf_pmu/2631 is trying to acquire lock:
[  385.697927]  (&mm->mmap_sem){++++}, at: [<ffffffff811bfe1e>] __might_fault+0x3e/0x90
[  385.697941]
               but task is already holding lock:
[  385.697946]  (&cpuctx_mutex){+.+.}, at: [<ffffffff8116fe8c>] perf_event_ctx_lock_nested+0xbc/0x1d0
[  385.697957]
               which lock already depends on the new lock.

[  385.697963]
               the existing dependency chain (in reverse order) is:
[  385.697970]
               -> #4 (&cpuctx_mutex){+.+.}:
[  385.697980]        __mutex_lock+0x86/0x9b0
[  385.697985]        perf_event_init_cpu+0x5a/0x90
[  385.697991]        perf_event_init+0x178/0x1a4
[  385.697997]        start_kernel+0x27f/0x3f1
[  385.698003]        verify_cpu+0x0/0xfb
[  385.698006]
               -> #3 (pmus_lock){+.+.}:
[  385.698015]        __mutex_lock+0x86/0x9b0
[  385.698020]        perf_event_init_cpu+0x21/0x90
[  385.698025]        cpuhp_invoke_callback+0xca/0xc00
[  385.698030]        _cpu_up+0xa7/0x170
[  385.698035]        do_cpu_up+0x57/0x70
[  385.698039]        smp_init+0x62/0xa6
[  385.698044]        kernel_init_freeable+0x97/0x193
[  385.698050]        kernel_init+0xa/0x100
[  385.698055]        ret_from_fork+0x27/0x40
[  385.698058]
               -> #2 (cpu_hotplug_lock.rw_sem){++++}:
[  385.698068]        cpus_read_lock+0x39/0xa0
[  385.698073]        apply_workqueue_attrs+0x12/0x50
[  385.698078]        __alloc_workqueue_key+0x1d8/0x4d8
[  385.698134]        i915_gem_init_userptr+0x5f/0x80 [i915]
[  385.698176]        i915_gem_init+0x7c/0x390 [i915]
[  385.698213]        i915_driver_load+0x99e/0x15c0 [i915]
[  385.698250]        i915_pci_probe+0x33/0x90 [i915]
[  385.698256]        pci_device_probe+0xa1/0x130
[  385.698262]        driver_probe_device+0x293/0x440
[  385.698267]        __driver_attach+0xde/0xe0
[  385.698272]        bus_for_each_dev+0x5c/0x90
[  385.698277]        bus_add_driver+0x16d/0x260
[  385.698282]        driver_register+0x57/0xc0
[  385.698287]        do_one_initcall+0x3e/0x160
[  385.698292]        do_init_module+0x5b/0x1fa
[  385.698297]        load_module+0x2374/0x2dc0
[  385.698302]        SyS_finit_module+0xaa/0xe0
[  385.698307]        entry_SYSCALL_64_fastpath+0x1c/0xb1
[  385.698311]
               -> #1 (&dev->struct_mutex){+.+.}:
[  385.698320]        __mutex_lock+0x86/0x9b0
[  385.698361]        i915_mutex_lock_interruptible+0x4c/0x130 [i915]
[  385.698403]        i915_gem_fault+0x206/0x760 [i915]
[  385.698409]        __do_fault+0x1a/0x70
[  385.698413]        __handle_mm_fault+0x7c4/0xdb0
[  385.698417]        handle_mm_fault+0x154/0x300
[  385.698440]        __do_page_fault+0x2d6/0x570
[  385.698445]        page_fault+0x22/0x30
[  385.698449]
               -> #0 (&mm->mmap_sem){++++}:
[  385.698459]        lock_acquire+0xaf/0x200
[  385.698464]        __might_fault+0x68/0x90
[  385.698470]        _copy_to_user+0x1e/0x70
[  385.698475]        perf_read+0x1aa/0x290
[  385.698480]        __vfs_read+0x23/0x120
[  385.698484]        vfs_read+0xa3/0x150
[  385.698488]        SyS_read+0x45/0xb0
[  385.698493]        entry_SYSCALL_64_fastpath+0x1c/0xb1
[  385.698497]
               other info that might help us debug this:

[  385.698505] Chain exists of:
                 &mm->mmap_sem --> pmus_lock --> &cpuctx_mutex

[  385.698517]  Possible unsafe locking scenario:

[  385.698522]        CPU0                    CPU1
[  385.698526]        ----                    ----
[  385.698529]   lock(&cpuctx_mutex);
[  385.698553]                                lock(pmus_lock);
[  385.698558]                                lock(&cpuctx_mutex);
[  385.698564]   lock(&mm->mmap_sem);
[  385.698568]
                *** DEADLOCK ***

[  385.698574] 1 lock held by perf_pmu/2631:
[  385.698578]  #0:  (&cpuctx_mutex){+.+.}, at: [<ffffffff8116fe8c>] perf_event_ctx_lock_nested+0xbc/0x1d0
[  385.698589]
               stack backtrace:
[  385.698595] CPU: 3 PID: 2631 Comm: perf_pmu Tainted: G     U          4.14.0-CI-Patchwork_7234+ #1
[  385.698602] Hardware name:                  /NUC6CAYB, BIOS AYAPLCEL.86A.0040.2017.0619.1722 06/19/2017
[  385.698609] Call Trace:
[  385.698615]  dump_stack+0x5f/0x86
[  385.698621]  print_circular_bug.isra.18+0x1d0/0x2c0
[  385.698627]  __lock_acquire+0x19c3/0x1b60
[  385.698634]  ? generic_exec_single+0x77/0xe0
[  385.698640]  ? lock_acquire+0xaf/0x200
[  385.698644]  lock_acquire+0xaf/0x200
[  385.698650]  ? __might_fault+0x3e/0x90
[  385.698655]  __might_fault+0x68/0x90
[  385.698660]  ? __might_fault+0x3e/0x90
[  385.698665]  _copy_to_user+0x1e/0x70
[  385.698670]  perf_read+0x1aa/0x290
[  385.698675]  __vfs_read+0x23/0x120
[  385.698682]  ? __fget+0x101/0x1f0
[  385.698686]  vfs_read+0xa3/0x150
[  385.698691]  SyS_read+0x45/0xb0
[  385.698696]  entry_SYSCALL_64_fastpath+0x1c/0xb1
[  385.698701] RIP: 0033:0x7ff1c46876ed
[  385.698705] RSP: 002b:00007fff13552f90 EFLAGS: 00000293 ORIG_RAX: 0000000000000000
[  385.698712] RAX: ffffffffffffffda RBX: ffffc90000647ff0 RCX: 00007ff1c46876ed
[  385.698718] RDX: 0000000000000010 RSI: 00007fff13552fa0 RDI: 0000000000000005
[  385.698723] RBP: 000056063d300580 R08: 0000000000000000 R09: 0000000000000060
[  385.698729] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000046
[  385.698734] R13: 00007fff13552c6f R14: 00007ff1c6279d00 R15: 00007ff1c6279a40

Testcase: igt/perf_pmu
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20171122172621.16158-1-chris@chris-wilson.co.uk
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
(cherry picked from commit ee48700dd57d9ce783ec40f035b324d0b75632e4)
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
(cherry picked from commit ef78970ace74bb606e641a1f27ea36547716a775)

BUG= chromium:892760 
TEST=Built with USE="debug" and booted to verify there was no lockdep error

Change-Id: I63043041caa735d0e58fb9bf63cd74656d481450
Signed-off-by: Yu Zhao <yuzhao@google.com>
Reviewed-on: https://chromium-review.googlesource.com/1282189
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Yu Zhao <yuzhao@chromium.org>
Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>

[modify] https://crrev.com/faa1045a76826b1516b15342133b5892529a226e/drivers/gpu/drm/i915/i915_gem.c

Status: Fixed (was: Assigned)

Sign in to add a comment