We also see this on the release branch for 64:
https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos_release%2Fveyron_mickey-release_release-R64-10176.B%2F22%2F%2B%2Frecipes%2Fsteps%2FArchive%2F0%2Fstdout

And this happened to Soraka on 10225:
https://luci-milo.appspot.com/buildbot/chromeos/soraka-release/707

Gwendal, given this seems to be dying around the filesystem stuff is this something in your jurisdiction? If not please feel free to reassign.

We really need to fix this as if it happens on releases it means we will be skipping boards when it does.

See it again here:
https://luci-milo.appspot.com/buildbot/chromeos/elm-release/2127

Repro on https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_minnie-release/builds/2144

INFO    : Creating FS for partition 8 with format ext4.
INFO    : Creating FS for partition 12 with format vfat.
INFO    : Creating FS for partition 3 with format ext2.
INFO    : Creating FS for partition 1 with format ext4.
mount: wrong fs type, bad option, bad superblock on /dev/loop3,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so.
ERROR   : mount failed: device=/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/factory_install_shim.bin target=/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/esp format=vfat ro/rw=rw options=offset=127926272
ERROR   : Tue Apr 24 06:25:46 PDT 2018
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 926:  '--from' '/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/factory_install_shim.bin' '--rootfs_mountpt' '/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/rootfs' '--stateful_mountpt' '/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/stateful' '--esp_mountpt' '/mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/esp'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  mount_gpt_image.sh:478:main(), called: mount_image  
ERROR   :  mount_gpt_image.sh:393:mount_image(), called: die 'Failed to mount all partitions in /mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/factory_install_shim.bin' 
ERROR   : 
ERROR   : Error was:
ERROR   :   Failed to mount all partitions in /mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/factory_install_shim.bin
INFO    : Unmounting image from /mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/stateful and /mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/rootfs
INFO    : Unmounting temporary rootfs /mnt/host/source/src/build/images/veyron_minnie/R68-10613.0.0-a3/rootfs//build/rootfs.


Looking at the serial console on the VM:
Apr 24 06:25:46 cros-beefy405-c2 kernel: [27993.358796] EXT4-fs (loop2): mounted filesystem with ordered data mode. Opts: (null)
[27993.469294] FAT-fs (loop3): bogus number of reserved sectors
Apr 24 06:25:46 cros-beefy405-c2 kernel: [27993.469294] FAT-fs (loop3): bogus number of reserved sectors
Apr 24 06:25:46 cros-beefy405-c2 kernel: [27993.475175] FAT-fs (loop3): Can't find a valid FAT filesystem

The BIOS parameter block (BPB) is not valid, part of the partition super block.

I can not find the factory image in the artifact, but the EFI partition may not be formatted properly or is not present at all.

The factory shim is following same process as build_image so I'm not sure what may go wrong... 

does this only fail on devices with cros_embedded set? (b/72749926)

#6: no, elm, veyron_minnie do not have cros_embedded set.

INFO    : Creating FS for partition 12 with format vfat.

The build script did create EFI (12) before starting to mount.
And the message indicates a "sudo mkfs.vfat" should be called after that, with a mount (based on loop) then umount.

In kernel source, the bogus number means reserved sectors = 0, but mkfs.vfat should create at least 1 (for FAT16 - which is the default selected value for our EFI partitions).

I wonder if there's some cache/sync/flush issue, or mount_gpt_image has some bug, or the calculation of offset in mk_fs has overflow.

well, $(()) seems supporting 64 bit integer so overflow should not be a problem.

I think we need to find a way to reproduce, or get a such corrupted image to analyze what happened.

Or, meanwhile, we can print additional info in "Create FS ..." for offset and size it has created, to see if the EFI was then zeroed by creation of other partitions.

Seeing this on falco release:

https://luci-logdog.appspot.com/v/?s=chromeos/bb/chromeos/falco-release/3505/+/recipes/steps/Archive/0/stdout

 Issue 850555  has been merged into this issue.

We are seeing this on the 67 branch also, has anyone had a chance to look at this recently?