New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 879810 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Sep 1
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 0
Type: Bug


Show other hotlists

Hotlists containing this issue:
Hotlist-1


Sign in to add a comment

grunt: kernel-crash boot loop @ R70-11022.0.0

Project Member Reported by djkurtz@chromium.org, Sep 1

Issue description

Chrome OS Version:  R70-11022.0.0
Chrome OS Platform: grunt

Steps To Reproduce:
(1) Boot R70-11022.0.0 test image from USB
(2) 
(3)

Expected Result:
System boots to sign-in screen and stays up.

Actual Result:
System reboots after ~35 seconds.


How frequently does this problem reproduce? (Always, sometimes, hard to
reproduce?)

Always.

What is the impact to the user, and is there a workaround? If so, what is
it?

Unusable canary channel images.

ramoops (attached) contains the following splat:
...
[   33.097095] kauditd_printk_skb: 109 callbacks suppressed
[   33.097097] audit: type=1400 audit(1535767590.095:120): avc:  granted  { execute } for  pid=1155 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.100696] audit: type=1400 audit(1535767590.099:121): avc:  granted  { execute } for  pid=1156 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.103576] audit: type=1400 audit(1535767590.102:122): avc:  granted  { execute } for  pid=1157 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.109658] audit: type=1400 audit(1535767590.108:123): avc:  granted  { execute } for  pid=1158 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.109868] audit: type=1400 audit(1535767590.108:124): avc:  granted  { execute } for  pid=1158 comm="sshd-pre" path="/bin/dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.111732] audit: type=1400 audit(1535767590.110:125): avc:  granted  { execute } for  pid=1160 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.144796] audit: type=1400 audit(1535767590.143:126): avc:  granted  { execute } for  pid=1165 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.147859] audit: type=1400 audit(1535767590.146:127): avc:  granted  { execute } for  pid=1166 comm="init" name="dash" dev="mmcblk1p3" ino=57431 scontext=u:r:cros_init:s0 tcontext=u:object_r:sh_exec:s0 tclass=file
[   33.155009] audit: type=1400 audit(1535767590.153:128): avc:  granted  { execute } for  pid=1167 comm="minijail0" path="/sbin/minijail0" dev="mmcblk1p3" ino=65558 scontext=u:r:minijail:s0 tcontext=u:object_r:cros_minijail_exec:s0 tclass=file
[   33.159115] audit: type=1400 audit(1535767590.158:129): avc:  granted  { execute } for  pid=1167 comm="minijail0" name="sslh-fork" dev="mmcblk1p3" ino=18701 scontext=u:r:minijail:s0 tcontext=u:object_r:cros_unconfined_exec:s0 tclass=file
[   33.253266] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[   33.255025] cz-da7219-max98357a AMD7219:00: ASoC: CODEC DAI HiFi not registered
[   33.255036] cz-da7219-max98357a AMD7219:00: devm_snd_soc_register_card(acpd7219m98357) failed: -517
[   33.258828] cz-da7219-max98357a AMD7219:00: ASoC: CODEC DAI adau7002-hifi not registered
[   33.258839] cz-da7219-max98357a AMD7219:00: devm_snd_soc_register_card(acpd7219m98357) failed: -517
[   33.262644] da7219 i2c-DLGS7219:00: Using default clk name: da7219-dai-clks
[   33.276012] (NULL device *): codec dai name = da7219-hifi
[   33.276697] cz-da7219-max98357a AMD7219:00: da7219-hifi <-> designware-i2s.1.auto mapping ok
[   33.276748] cz-da7219-max98357a AMD7219:00: da7219-hifi <-> designware-i2s.2.auto mapping ok
[   33.276797] cz-da7219-max98357a AMD7219:00: HiFi <-> designware-i2s.3.auto mapping ok
[   33.276843] cz-da7219-max98357a AMD7219:00: adau7002-hifi <-> designware-i2s.3.auto mapping ok
[   33.276893] cz-da7219-max98357a AMD7219:00: adau7002-hifi <-> designware-i2s.2.auto mapping ok
[   33.278291] input: acpd7219m98357 Headset Jack as /devices/platform/AMD7219:00/sound/card1/input5
[   33.315106] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[   33.315117] piix4_smbus 0000:00:14.0: Using register 0x02 for SMBus port selection
[   33.366554] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input6
[   33.366681] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input7
[   33.424833] ath10k_pci 0000:01:00.0: refusing to write mmio out of bounds at 0x0d134400 - 0x0d134404 (max 0x00200000)
[   33.424852] BUG: unable to handle kernel NULL pointer dereference at 0000000000000040
[   33.424876] IP: ath10k_ce_src_ring_base_addr_set+0x56/0x81 [ath10k_core]
[   33.424880] PGD 0 P4D 0 
[   33.424886] Oops: 0000 [#1] PREEMPT SMP NOPTI
[   33.424909] gsmi: Log Shutdown Reason 0x03
[   33.424913] Modules linked in: ath10k_pci(+) ath10k_core snd_hda_codec_hdmi i2c_piix4 mac80211 acpi_als ath snd_soc_adau7002 snd_soc_max98357a snd_soc_acp_da7219mx98357_mach snd_hda_intel snd_soc_da7219 designware_i2s snd_hda_codec snd_hwdep acp_audio_dma snd_hda_core cfg80211 iio_trig_sysfs cros_ec_sensors cros_ec_sensors_ring cros_ec_sensors_core industrialio_triggered_buffer kfifo_buf industrialio smsc95xx usbnet mii joydev
[   33.424946] CPU: 0 PID: 1173 Comm: udevd Not tainted 4.14.67 #4
[   33.424950] Hardware name: Google Grunt/Grunt, BIOS Google_Grunt.10974.0.0 08/16/2018
[   33.424954] task: ffff9a2b4d5b2b80 task.stack: ffffac98012d4000
[   33.424969] RIP: 0010:ath10k_ce_src_ring_base_addr_set+0x56/0x81 [ath10k_core]
[   33.424973] RSP: 0018:ffffac98012d79f0 EFLAGS: 00010246
[   33.424977] RAX: 0000000000000000 RBX: 000000000d134400 RCX: 89545d20444a6100
[   33.424980] RDX: ffff9a2b6ec18660 RSI: ffff9a2b6ec107c0 RDI: ffff9a2b6ec107c0
[   33.424983] RBP: ffffac98012d7a18 R08: fffffffffffffd08 R09: 00000000ffff0a00
[   33.424987] R10: 0000000000000007 R11: ffffffff82a96d91 R12: ffff9a2b413f8608
[   33.424990] R13: 0000000000034400 R14: 0000000000000000 R15: ffff9a2b413f1900
[   33.424994] FS:  000079f60b8b8800(0000) GS:ffff9a2b6ec00000(0000) knlGS:0000000000000000
[   33.424998] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   33.425001] CR2: 0000000000000040 CR3: 000000010d5f8000 CR4: 00000000001406f0
[   33.425004] Call Trace:
[   33.425023]  ath10k_ce_deinit_pipe+0x31/0x94 [ath10k_core]
[   33.425035]  ath10k_pci_ce_deinit+0x1d/0x33 [ath10k_pci]
[   33.425044]  ath10k_pci_probe+0x461/0x6b4 [ath10k_pci]
[   33.425052]  ? ath10k_pci_qca988x_chip_reset+0x14f/0x14f [ath10k_pci]
[   33.425059]  ? ath10k_pci_remove+0x67/0x67 [ath10k_pci]
[   33.425069]  pci_device_probe+0xa4/0x117
[   33.425078]  driver_probe_device+0x25e/0x2a9
[   33.425084]  __driver_attach+0x7a/0x9e
[   33.425090]  ? driver_attach+0x22/0x22
[   33.425095]  bus_for_each_dev+0x76/0xb0
[   33.425101]  bus_add_driver+0x102/0x1d3
[   33.425106]  ? 0xffffffffc01fc000
[   33.425110]  driver_register+0x8c/0xca
[   33.425114]  ? 0xffffffffc01fc000
[   33.425139]  init_module+0x1e/0x1000 [ath10k_pci]
[   33.425144]  do_one_initcall+0x117/0x1ed
[   33.425165]  ? consume_skb+0x12/0x2b
[   33.425171]  ? __kmalloc+0xee/0x17d
[   33.425176]  ? kfree+0x97/0x417
[   33.425181]  ? do_init_module+0x22/0x19f
[   33.425185]  do_init_module+0x58/0x19f
[   33.425190]  load_module+0x1c85/0x1f31
[   33.425198]  ? kernel_read_file+0x109/0x171
[   33.425203]  ? kernel_read_file_from_fd+0x46/0x6f
[   33.425207]  SyS_finit_module+0xb7/0xbe
[   33.425213]  do_syscall_64+0x67/0x75
[   33.425219]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[   33.425223] RIP: 0033:0x79f60afb2139
[   33.425226] RSP: 002b:00007ffe553bac88 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   33.425231] RAX: ffffffffffffffda RBX: 00005b245ab09080 RCX: 000079f60afb2139
[   33.425234] RDX: 0000000000000000 RSI: 000079f60b84940b RDI: 0000000000000010
[   33.425237] RBP: 00007ffe553bacd0 R08: 0000000000000000 R09: 00005b245aaeb650
[   33.425240] R10: 0000000000000010 R11: 0000000000000246 R12: 0000000000000000
[   33.425243] R13: 00005b245ab09440 R14: 000079f60b84940b R15: 0000000000000000
[   33.425247] Code: 8b 50 14 8b 58 18 29 d3 0f af de 01 d3 8b 31 01 de 49 8b 44 24 18 4c 8b 58 08 44 89 f2 e8 7f 99 19 c3 49 6b c5 48 49 8b 44 04 60 <4c> 8b 58 40 4d 85 db 74 17 4c 89 ff 89 de 4c 89 f2 5b 41 5c 41 
[   33.425303] RIP: ath10k_ce_src_ring_base_addr_set+0x56/0x81 [ath10k_core] RSP: ffffac98012d79f0
[   33.425306] CR2: 0000000000000040
[   33.425362] ---[ end trace 2dc77585e5903943 ]---
[   33.433950] Kernel panic - not syncing: Fatal exception
[   33.433970] Kernel Offset: 0x1800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[   33.433990] gsmi: Log Shutdown Reason 0x02
[   33.439081] ACPI MEMORY or I/O RESET_REG.


 
grunt-R70-11022.0.0-bootloop-console-ramoops
48.8 KB Download
My Canary channel dogfood grunt just updated to the latest R70-11021.0.0, and it seems fine.

So, it is probably a kernel change that landed after, in the range:

https://chromium.googlesource.com/chromiumos/third_party/kernel/+log/52c742d7395c35e2bb295f20b3e4390293c67214..50112079d890f4c89c836b4a53f96d07c81e9f37?n=10000


Project Member

Comment 2 by bugdroid1@chromium.org, Sep 1

Labels: merge-merged-chromeos-4.14
The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/kernel/+/6566dd73b0b8335559806e2ac6864dd4635edec4

commit 6566dd73b0b8335559806e2ac6864dd4635edec4
Author: Brian Norris <briannorris@chromium.org>
Date: Sat Sep 01 02:45:06 2018

Revert "FROMLIST: ath10k: Set DMA address mask to 35 bit for WCN3990"

This reverts commit 7350ca6fd865b4a99e2102bd91b077da69abc724.

Reason for revert: it crashes on QCA6174A Wifi

Original change's description:
> FROMLIST: ath10k: Set DMA address mask to 35 bit for WCN3990
> 
> WCN3990 is a 37-bit target but can address memory range
> only upto 35 bits. The 36th bit is used to control the
> smmu/iommu translation and the 37th bit is used by the
> internal bus masters to access the wifi subsystem internal
> SRAM. With the DMA mask set to 37-bit, the host driver
> can get 37-bit dma address, which leads to incorrect
> address access in the target.
> 
> Hence the host driver can use addresses upto 35-bit
> for WCN3990. Fix the dma mask for wcn3990 to 35-bit,
> instead of 37-bit.
> 
> Tested HW: WCN3990
> Tested FW: WLAN.HL.2.0-01188-QCAHLSWMTPLZ-1
> 
> BUG=b:78530936
> TEST=wifi_matfunc on QCA6174A; basic sanity on WCN3990
> 
> Change-Id: Ice1ed50e39f85b771d05d73aadb261275fcaa919
> Signed-off-by: Rakesh Pillai <pillair@codeaurora.org>
> (am from https://patchwork.kernel.org/patch/10557295/)
> Reviewed-on: https://chromium-review.googlesource.com/1137962
> Commit-Ready: Brian Norris <briannorris@chromium.org>
> Tested-by: Brian Norris <briannorris@chromium.org>
> Reviewed-by: Govind Singh <govinds@qualcomm.corp-partner.google.com>
> Reviewed-by: Brian Norris <briannorris@chromium.org>

BUG=b:78530936,  chromium:879810 
TEST=build; test grunt

Change-Id: I5894b71fbfba60c20a13d1bdf5d63294a0f7e473
Reviewed-on: https://chromium-review.googlesource.com/1200293
Trybot-Ready: Brian Norris <briannorris@chromium.org>
Reviewed-by: Brian Norris <briannorris@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>

[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/hw.h
[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/htt_tx.c
[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/hw.c
[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/ce.c
[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/ce.h
[modify] https://crrev.com/6566dd73b0b8335559806e2ac6864dd4635edec4/drivers/net/wireless/ath/ath10k/snoc.c

Status: Verified (was: Started)
Labels: Kernel-4.14 M-71
I thought the bad CL made it into the R70 branch, but it just barely missed the branch (good!). So we're all good now.

I just wanted to note that, in case anybody else was worrying about the same.

Sign in to add a comment