panic in bpf_register_prog_type |
||
Issue description
Observed when running an s390 qemu session.
Unable to handle kernel pointer dereference in virtual kernel address space
failing address: 0000000000b96000 TEID: 0000000000b96407
Fault in home space mode while using kernel ASCE.
AS:000000000107a007 R3:000000001fff0007 S:000000001ffea800 P:0000000000b96215
Oops: 0004 ilc:3 [#1] SMP DEBUG_PAGEALLOC
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.140-14502-g02408c75d39e #1
task: 000000001f028000 task.stack: 000000001f024000
Krnl PSW : 0704e00180000000 00000000006aa002 (__list_add+0x52/0xd8)
R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 EA:3
Krnl GPRS: 0000000000000018 0000000000d88918 0000000000b965d8 0000000000d88918
0000000000d88918 00000000000008a4 0000000000ea41d8 000000000106d008
000000000106d030 0000000000e63038 0000000000d88918 0000000000d88918
0000000000b965d8 0000000000954040 000000001f027d50 000000001f027d18
Krnl Code: 00000000006a9ff4: b92000ac cgr %r10,%r12
00000000006a9ff8: a7840015 brc 8,6aa022
#00000000006a9ffc: e3c0a0080024 stg %r12,8(%r10)
>00000000006aa002: e3a0c0000024 stg %r10,0(%r12)
00000000006aa008: e3b0c0080024 stg %r11,8(%r12)
00000000006aa00e: e3c0b0000024 stg %r12,0(%r11)
00000000006aa014: e340f0c00004 lg %r4,192(%r15)
00000000006aa01a: ebaff0a00004 lmg %r10,%r15,160(%r15)
Call Trace:
([<07000000000008a2>] 0x7000000000008a2)
[<0000000000e2d0e0>] register_kprobe_prog_ops+0x28/0x40
[<00000000001001da>] do_one_initcall+0xa2/0x1b0
[<0000000000e12ef6>] kernel_init_freeable+0x28e/0x348
[<00000000009430e6>] kernel_init+0x2e/0x128
[<0000000000951bca>] kernel_thread_starter+0x6/0xc
[<0000000000951bc4>] kernel_thread_starter+0x0/0xc
INFO: lockdep is turned off.
Last Breaking-Event-Address:
[<0000000000295b06>] bpf_register_prog_type+0x16/0x20
Kernel panic - not syncing: Fatal exception: panic_on_oops
Bisect:
# bad: [37b41fda65d3b58416015043203bdbd81115c6d9] UPSTREAM: ASoC: dapm: Recalculate audio map forcely when card instantiated
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
git bisect start 'HEAD' 'v4.4'
# good: [56517fd182a2f36425336f2b7e5b5e09d30cda2c] BACKPORT: drm/gem: support BO freeing without dev->struct_mutex
git bisect good 56517fd182a2f36425336f2b7e5b5e09d30cda2c
# good: [954581d83d18436c7c64f8540064bff5859e2566] UPSTREAM: PCI: Recognize D3cold in pci_update_current_state()
git bisect good 954581d83d18436c7c64f8540064bff5859e2566
# good: [74ede0af32f565ac5b7222c8722cc7da6951bef4] x86/percpu: Fix this_cpu_read()
git bisect good 74ede0af32f565ac5b7222c8722cc7da6951bef4
# good: [84dc6431053a38b930140a1503f302b6a12fbda2] BACKPORT: drm/i915/skl+: nv12 workaround disable WM level 1-7
git bisect good 84dc6431053a38b930140a1503f302b6a12fbda2
# bad: [852f428b756e1a33789460cbc95d4546a996d8bc] Merge remote-tracking branch 'upstream/chromeos-4.4' into chromeos-4.4__release/core38-78
git bisect bad 852f428b756e1a33789460cbc95d4546a996d8bc
# good: [04e1e81cc82481c54d39a70b3347ef9de564983d] CHROMIUM: iwl7000: mvm: force TCM re-evaluation on TCM resume
git bisect good 04e1e81cc82481c54d39a70b3347ef9de564983d
# good: [d8ffda217653e134260a4013342abb9e4775eff8] CHROMIUM: iwl7000: chromeOS: Tweak backport headers for 4.4 support
git bisect good d8ffda217653e134260a4013342abb9e4775eff8
# bad: [5d6951fd1598768bc17069d0955516d6eac13dfb] UPSTREAM: drm/i915/psr: Nuke aux frame sync
git bisect bad 5d6951fd1598768bc17069d0955516d6eac13dfb
# bad: [ddfdfd5870fc11c562ec9f7641621403adfc9822] UPSTREAM: drm/i915/dp: Silence compiler for missing prototype
git bisect bad ddfdfd5870fc11c562ec9f7641621403adfc9822
# good: [cc968d74619e63abe870d79c7d694833c3a52b77] CHROMIUM: config: enable CROS_EC_THROTTLER for rockchip64/arm64
git bisect good cc968d74619e63abe870d79c7d694833c3a52b77
# bad: [7cd1abbacd1dfdc6ddfdd8ac49f39bbb7ad81fed] UPSTREAM: mm/mmap.c: mark protection_map as __ro_after_init
git bisect bad 7cd1abbacd1dfdc6ddfdd8ac49f39bbb7ad81fed
# good: [a492a707e79c291c41239914a0622bfe521a3550] CHROMIUM: alt-syscall: Allow Android access to sync(2)
git bisect good a492a707e79c291c41239914a0622bfe521a3550
# bad: [e0aa98ac77dc08b55827832faecddc784ec1527e] UPSTREAM: watchdog: booke_wdt: add __ro_after_init to booke_wdt_info
git bisect bad e0aa98ac77dc08b55827832faecddc784ec1527e
# bad: [02408c75d39e7f17fa2cb805f356198bfeece796] BACKPORT: bpf: mark all registered map/prog types as __ro_after_init
git bisect bad 02408c75d39e7f17fa2cb805f356198bfeece796
# good: [e30621c76c7e24facc6cf69cb790daeef1860147] UPSTREAM: x86: Apply more __ro_after_init and const
git bisect good e30621c76c7e24facc6cf69cb790daeef1860147
# first bad commit: [02408c75d39e7f17fa2cb805f356198bfeece796] BACKPORT: bpf: mark all registered map/prog types as __ro_after_init
Reverting this patch alone does not fix the problem; qemu-s390 with ToT chromeos-4.4 does not crash but fails silently. More bisects will be needed to find the second problem.
Either case, the crash suggests that BPF may no longer operate correctly in chromeos-4.4.
,
Dec 7
qemu command line:
qemu-system-s390x -kernel arch/s390/boot/bzImage \
-initrd rootfs.cpio \
-append 'rdinit=/sbin/init panic=-1' \
-m 512 -nographic -monitor null --no-reboot
with the attached initrd.
I don't know yet if chromeos-4.14 is affected. Still bisecting chromeos-4.4.
,
Dec 7
Another crash:
Unable to handle kernel pointer dereference in virtual kernel address space
failing address: 0000000000b96000 TEID: 0000000000b96407
Fault in home space mode while using kernel ASCE.
AS:000000000107a007 R3:000000001fff0007 S:000000001ffea800 P:0000000000b96215
Oops: 0004 ilc:3 [#1] SMP DEBUG_PAGEALLOC
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.140-14505-gd8756db0aba4-dirty #1
task: 000000001f028000 task.stack: 000000001f024000
Krnl PSW : 0704e00180000000 0000000000e1f6ba (ksysfs_init+0x82/0xd8)
R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 EA:3
Krnl GPRS: 0000000000000001 0000000000000024 000000001f26d680 0000000000b965e0
0000000000b965e0 000000003fc5e3a2 0000000000ea3858 000000000106d008
000000000106d030 0000000000e63008 0000000000000000 000000000107f068
0000000000000000 0000000000960f38 0000000000e1f68a 000000001f027d30
Krnl Code: 0000000000e1f6aa: c040ffebb79b larl %r4,b965e0
0000000000e1f6b0: b9040034 lgr %r3,%r4
#0000000000e1f6b4: e320b0000004 lg %r2,0(%r11)
>0000000000e1f6ba: e31040200024 stg %r1,32(%r4)
0000000000e1f6c0: c0e5ffae3244 brasl %r14,3e5b48
0000000000e1f6c6: 1222 ltr %r2,%r2
0000000000e1f6c8: b90400a2 lgr %r10,%r2
0000000000e1f6cc: a7840019 brc 8,e1f6fe
Call Trace:
([<0000000000e1f68a>] ksysfs_init+0x52/0xd8)
[<00000000001001da>] do_one_initcall+0xa2/0x1b0
[<0000000000e12ef6>] kernel_init_freeable+0x28e/0x348
[<00000000009430e6>] kernel_init+0x2e/0x128
[<0000000000951bca>] kernel_thread_starter+0x6/0xc
[<0000000000951bc4>] kernel_thread_starter+0x0/0xc
INFO: lockdep is turned off.
Last Breaking-Event-Address:
[<00000000003e6b96>] internal_create_group+0x26e/0x348
Kernel panic - not syncing: Fatal exception: panic_on_oops
This is at commit d8756db0aba49f46 ("UPSTREAM: kernel/ksysfs.c: add __ro_after_init to bin_attribute structure").
,
Dec 7
Forgot to mention: This is a boot problem; no PoC necessary.
,
Dec 7
Second bisect: # bad: [37b41fda65d3b58416015043203bdbd81115c6d9] UPSTREAM: ASoC: dapm: Recalculate audio map forcely when card instantiated # good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4 git bisect start 'HEAD' 'v4.4' # good: [56517fd182a2f36425336f2b7e5b5e09d30cda2c] BACKPORT: drm/gem: support BO freeing without dev->struct_mutex git bisect good 56517fd182a2f36425336f2b7e5b5e09d30cda2c # good: [954581d83d18436c7c64f8540064bff5859e2566] UPSTREAM: PCI: Recognize D3cold in pci_update_current_state() git bisect good 954581d83d18436c7c64f8540064bff5859e2566 # good: [74ede0af32f565ac5b7222c8722cc7da6951bef4] x86/percpu: Fix this_cpu_read() git bisect good 74ede0af32f565ac5b7222c8722cc7da6951bef4 # good: [84dc6431053a38b930140a1503f302b6a12fbda2] BACKPORT: drm/i915/skl+: nv12 workaround disable WM level 1-7 git bisect good 84dc6431053a38b930140a1503f302b6a12fbda2 # bad: [852f428b756e1a33789460cbc95d4546a996d8bc] Merge remote-tracking branch 'upstream/chromeos-4.4' into chromeos-4.4__release/core38-78 git bisect bad 852f428b756e1a33789460cbc95d4546a996d8bc # good: [04e1e81cc82481c54d39a70b3347ef9de564983d] CHROMIUM: iwl7000: mvm: force TCM re-evaluation on TCM resume git bisect good 04e1e81cc82481c54d39a70b3347ef9de564983d # good: [d8ffda217653e134260a4013342abb9e4775eff8] CHROMIUM: iwl7000: chromeOS: Tweak backport headers for 4.4 support git bisect good d8ffda217653e134260a4013342abb9e4775eff8 # bad: [5d6951fd1598768bc17069d0955516d6eac13dfb] UPSTREAM: drm/i915/psr: Nuke aux frame sync git bisect bad 5d6951fd1598768bc17069d0955516d6eac13dfb # bad: [ddfdfd5870fc11c562ec9f7641621403adfc9822] UPSTREAM: drm/i915/dp: Silence compiler for missing prototype git bisect bad ddfdfd5870fc11c562ec9f7641621403adfc9822 # good: [cc968d74619e63abe870d79c7d694833c3a52b77] CHROMIUM: config: enable CROS_EC_THROTTLER for rockchip64/arm64 git bisect good cc968d74619e63abe870d79c7d694833c3a52b77 # bad: [7cd1abbacd1dfdc6ddfdd8ac49f39bbb7ad81fed] UPSTREAM: mm/mmap.c: mark protection_map as __ro_after_init git bisect bad 7cd1abbacd1dfdc6ddfdd8ac49f39bbb7ad81fed # good: [a492a707e79c291c41239914a0622bfe521a3550] CHROMIUM: alt-syscall: Allow Android access to sync(2) git bisect good a492a707e79c291c41239914a0622bfe521a3550 # good: [e0aa98ac77dc08b55827832faecddc784ec1527e] UPSTREAM: watchdog: booke_wdt: add __ro_after_init to booke_wdt_info git bisect good e0aa98ac77dc08b55827832faecddc784ec1527e # bad: [d8756db0aba49f46e2781ebb7760da54caf1ca9f] UPSTREAM: kernel/ksysfs.c: add __ro_after_init to bin_attribute structure git bisect bad d8756db0aba49f46e2781ebb7760da54caf1ca9f # good: [d45d7900226f9ffaafd085dfdf7169aa4eb8e6e8] UPSTREAM: watchdog: pika_wdt: add __ro_after_init to ident git bisect good d45d7900226f9ffaafd085dfdf7169aa4eb8e6e8 # first bad commit: [d8756db0aba49f46e2781ebb7760da54caf1ca9f] UPSTREAM: kernel/ksysfs.c: add __ro_after_init to bin_attribute structure
,
Dec 7
There might be a patch related to s390 and __ro_after_init that needs to be applied for the ro_after_init changes to not take effect on that arch. Let me try and see if I can find it.
,
Dec 7
Turns out ro_after_init support was only added later to s390 ...
,
Dec 7
Turns out fixing this is too complex and by itself adds risk. We'll just drop the s390 boot test from chromeos-4.4.
,
Dec 7
There are several patches in arch/s390, but they don't apply cleanly. |
||
►
Sign in to add a comment |
||
Comment 1 by zsm@chromium.org
, Dec 7