Issue metadata
Sign in to add a comment
|
[usb_detect] logging_GenerateCrashFiles.KERNEL test is failing due to missing ['kcrash', 'meta'] files on linux-3.8 systems |
||||||||||||||||||||||
Issue descriptionChrome Version: 10108.0.0 OS: ChromeOS Devices: Daisy, Link, Panther, Pit, Peppy and Speedy. Wmatrix link: https://wmatrix.googleplex.com/unfiltered?hide_missing=True&tests=logging_GenerateCrashFiles.KERNEL This issue is a regression from R64-10108.0.0 From debug logs: 11/08 08:16:25.853 DEBUG| abstract_ssh:0381| Using Rsync. 11/08 08:16:25.853 DEBUG| utils:0212| Running 'rsync -L --timeout=1800 --rsh='/usr/bin/ssh -a -x -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22' -az --no-o --no-g root@chromeos1-row1-rack4-host3:"/tmp/autoserv-55kxMo/sysinfo.pickle" "/tmp/tmpnwrTun"' 11/08 08:16:27.281 DEBUG| test:0202| before_iteration_hook ends running. 11/08 08:16:27.282 DEBUG| test:0365| before_iteration_hooks completed 11/08 08:16:27.282 DEBUG| test:0379| starting test(run_once()), test details follow () 11/08 08:16:27.304 DEBUG| ssh_host:0296| Running (ssh) 'sync' from 'execute|_call_run_once_with_retry|_call_run_once|run_once|run|run_very_slowly' 11/08 08:16:47.320 DEBUG| ssh_host:0296| Running (ssh) 'ls /var/spool/crash' from 'execute|_call_run_once_with_retry|_call_run_once|run_once|run|run_very_slowly' 11/08 08:16:48.458 DEBUG| ssh_host:0296| Running (ssh) 'echo BUG > /sys/kernel/debug/provoke-crash/DIRECT' from 'execute|_call_run_once_with_retry|_call_run_once|run_once|run|run_very_slowly' 11/08 08:17:06.506 DEBUG| utils:0280| [stderr] Write failed: Broken pipe 11/08 08:17:06.511 DEBUG|logging_GenerateCr:0067| Crash invoked! 11/08 08:17:06.511 DEBUG| utils:0212| Running 'ping chromeos1-row1-rack4-host3 -w1 -c1' 11/08 08:17:06.627 DEBUG| utils:0280| [stdout] PING chromeos1-row1-rack4-host3.cros.corp.google.com (172.27.212.81) 56(84) bytes of data. 11/08 08:17:06.628 DEBUG| utils:0280| [stdout] 64 bytes from chromeos-rack8e-host2.mtv.corp.google.com (172.27.212.81): icmp_seq=1 ttl=55 time=58.9 ms 11/08 08:17:06.629 DEBUG| utils:0280| [stdout] 11/08 08:17:06.629 DEBUG| utils:0280| [stdout] --- chromeos1-row1-rack4-host3.cros.corp.google.com ping statistics --- 11/08 08:17:06.630 DEBUG| utils:0280| [stdout] 1 packets transmitted, 1 received, 0% packet loss, time 0ms 11/08 08:17:06.630 DEBUG| utils:0280| [stdout] rtt min/avg/max/mdev = 58.999/58.999/58.999/0.000 ms 11/08 08:17:06.655 DEBUG| ssh_host:0296| Running (ssh) 'sync' from 'execute|_call_run_once_with_retry|_call_run_once|run_once|run|run_very_slowly' 11/08 08:17:17.935 DEBUG| ssh_host:0296| Running (ssh) 'ls /var/spool/crash' from '_call_run_once_with_retry|_call_run_once|run_once|check_missing_crash_files|run|run_very_slowly' 11/08 08:17:19.039 INFO |logging_GenerateCr:0035| Crash files diff: set([]) 11/08 08:17:19.040 DEBUG| test:0389| Test failed due to set(['kcrash', 'meta']) files not generated.. Exception log follows the after_iteration_hooks. I am able to reproduce this issue manually on lab devices. Daisy : chromeos1-row1-rack4-host3 link: chromeos1-row1-rack3-host3 Panther: chromeos1-row2-rack4-host4 Pit: chromeos1-row1-rack4-host5 Peppy: chromeos1-row1-rack4-host2 Speedy: chromeos1-row1-rack3-host2 Will check with recovery images.
,
Nov 8 2017
Looking at https://crosland.corp.google.com/log/10107.0.0..10108.0.0 https://chromium-review.googlesource.com/c/chromiumos/platform2/+/740867 could be related. +bmgordon@ Checked with sontis@ and there are no kcrash files referenced in the messages log
,
Nov 8 2017
Taking a look now.
,
Nov 8 2017
,
Nov 8 2017
I can reproduce the missing kcrash files on a link, but they're still missing even without crrev.com/c/740867. crrev.com/c/753516 also looks potentially related, since it moves the pstore path. I'll double-check that one next.
,
Nov 8 2017
that CL moved the mount point, but left a bind mount in place that *should* have kept backwards compat. i'm posting CLs to migrate things over bit by bit otherwise i'd have to land a whole bunch at once :/.
,
Nov 8 2017
It seems like /sys/fs/pstore doesn't exist on link, so the new init script fails. Running the mount command by hand, I get this: # mount -n -t pstore -o nodev,nosuid,noexec pstore /sys/fs/pstore mount: mount point /sys/fs/pstore does not exist Taking a quick look, it looks like all the newly failing devices are on kernel 3.8.11. Is there something changed about pstore in newer kernels?
,
Nov 8 2017
ah yes, that'd be it. the /sys/fs/pstore path is new to linux-3.9. i'll backport the simple patch to 3.8.
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/193f1c589b3c8e7a06e2a7a8e56fe039a43bb86e commit 193f1c589b3c8e7a06e2a7a8e56fe039a43bb86e Author: Josh Boyer <jwboyer@redhat.com> Date: Fri Nov 10 17:16:36 2017 UPSTREAM: pstore: Create a convenient mount point for pstore Using /dev/pstore as a mount point for the pstore filesystem is slightly awkward. We don't normally mount filesystems in /dev/ and the /dev/pstore file isn't created automatically by anything. While this method will still work, we can create a persistent mount point in sysfs. This will put pstore on par with things like cgroups and efivarfs. BUG= chromium:782821 , chromium:781536 TEST=precq passes Signed-off-by: Josh Boyer <jwboyer@redhat.com> Acked-by: Kees Cook <keescook@chromium.org> Signed-off-by: Tony Luck <tony.luck@intel.com> (cherry picked from commit fb0af3f2b1b613e5ea75426d454c7e5b1d1eef49) Change-Id: I8ed29909ac26f5a2e171c05e0640d731419f1233 Reviewed-on: https://chromium-review.googlesource.com/759277 Commit-Ready: Mike Frysinger <vapier@chromium.org> Tested-by: Benjamin Gordon <bmgordon@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-by: Benjamin Gordon <bmgordon@chromium.org> [modify] https://crrev.com/193f1c589b3c8e7a06e2a7a8e56fe039a43bb86e/Documentation/ABI/testing/pstore [modify] https://crrev.com/193f1c589b3c8e7a06e2a7a8e56fe039a43bb86e/fs/pstore/inode.c
,
Nov 10 2017
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by sontis@chromium.org
, Nov 8 2017Components: OS>Kernel
Labels: ReleaseBlock-Stable