New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 880171 link

Starred by 5 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Sep 5
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocked on:
issue 880497
issue 880504



Sign in to add a comment

Chrome PFQ and pre-CQ keep failing

Project Member Reported by chuntsen@chromium.org, Sep 4

Issue description

https://cros-goldeneye.corp.google.com/chromeos/console/listChromePfqBuild#/

tricky-chrome-pfq and peppy-chrome-pfq showed:
Command <mosys memory spd print geometry -s size_mb> failed, rc=-6, Command returned non-zero exit status	
 
Cc: jorgelo@chromium.org edcourtney@chromium.org achuith@chromium.org egranata@chromium.org
Labels: -Pri-2 Pri-0
Owner: chuntsen@chromium.org
Status: Assigned (was: Untriaged)
In particular, it looks like build image is failing. Here are some suspicious logs:

INFO    : Extracting the kernel command line from /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/vmlinuz.image
sh: line 1: 11344 Segmentation fault      (core dumped) mattrib -h -r -s s:/ldlinux.sys 2> /dev/null
syslinux: failed to create ldlinux.sys
ERROR   : Mon Sep  3 19:04:15 PDT 2018
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 11122:  '--arch=amd64' '--image_type=usb' '--to=/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/chromiumos_base_image.bin' '--from=/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/rootfs_dir/boot' '--vmlinuz=/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/rootfs_dir/boot/vmlinuz' '--to_offset=127926272' '--to_size=33554432' '--kernel_partition='/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/vmlinuz.image'' '--install_syslinux'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  update_bootloaders.sh:203:main(), called: die_err_trap  
ERROR   : 
ERROR   : Command failed:
ERROR   :   Command 'sudo syslinux -d /syslinux "${ESP_DEV}"' exited with nonzero code: 1
umount: /tmp/esp.Rh73DU: not mounted.
WARNING : Initial unmount failed. Possibly crosbug.com/23443. Retrying
umount: /tmp/esp.Rh73DU: not mounted.
INFO    : Unmounting image from /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/stateful_dir and /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/rootfs_dir
INFO    : Unmounting temporary rootfs /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/rootfs_dir//build/rootfs.
Cleaning up /usr/local symlinks for /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/stateful_dir/dev_image
ERROR   : Mon Sep  3 19:04:21 PDT 2018
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 9341:  '/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1' 'chromiumos_base_image.bin' '--adjust_part='
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  cros_make_image_bootable:443:main(), called: make_image_bootable '/mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/chromiumos_base_image.bin' 
ERROR   :  cros_make_image_bootable:1:make_image_bootable(), called: die 'cros_make_image_bootable failed.' 
ERROR   : 
ERROR   : Error was:
ERROR   :   cros_make_image_bootable failed.

AFAICT this is affecting a lot of PFQs (including Android PFQ, but most chrome PFQs). 

PFQ is high priority I think so bumping this to P0 and CC'ing ChromeOS sheriffs and gardeners.
Cc: hirono@chromium.org
Cc: amoylan@chromium.org
Cc: hidehiko@chromium.org
This looks not limited to Chrome PFQ. I hit the same issue on my pre-cq try.

https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8936345568402580784

I seem to have this on my local build.
Summary: Chrome PFQ and pre-CQ keep failing (was: Chrome PFQ keep failing)
The error message:
ERROR   :   Command 'sudo syslinux -d /syslinux "${ESP_DEV}"' exited with nonzero code: 1
started from 11031.0.0
Cc: -hirono@chromium.org oka@chromium.org
Cc: hirono@chromium.org
Wait, are the errors in OP and c#1 happening at the same time?
Not at the same time. The bug in c#1 seemed to be fixed.
If c#1 seems to be fixed, then mosys will likely be fixed soon by https://chromium-review.googlesource.com/c/chromiumos/platform/mosys/+/1203572.
Cc: mukai@chromium.org
Blockedon: 880504
Blockedon: 880497
This:

... Segmentation fault      (core dumped) mattrib -h -r -s s:/ldlinux.sys 2> /dev/null
syslinux: failed to create ldlinux.sys

no longer happens for me after sync this morning.
Labels: -Pri-0 Pri-1
Yeah, so I think this is actually another case of the mosys crash.
Status: Fixed (was: Assigned)
I think we can probably close this bug since the main issues (mosys and the build_image failures) have been fixed? Please re-open if there's some work pending.
FWIW, I ran into the symptoms described in comment #1 for a few days - i.e. mattrib crashes. mattrib is a symlink to mtools and my problems went away after I did "sudo emerge mtools" inside the chroot. 


INFO    : Extracting the kernel command line from /mnt/host/source/src/build/images/caroline/R71-11032.0.0-rc1/vmlinuz.image
sh: line 1: 11344 Segmentation fault      (core dumped) mattrib -h -r -s s:/ldlinux.sys 2> /dev/null
syslinux: failed to create ldlinux.sys

Sign in to add a comment