Ansar@ can you please confirm with AU.

Recovery logs are present at https://pantheon.corp.google.com/storage/browser/chromiumos-test-logs/bugfiles/cros/887672/?debugUI=CLOUD

Issue does *not* reproduce on Fizz (sion) using R70-11021.19.0

Diff between the two builds shows several coral FW changes:
https://crosland.corp.google.com/log/11021.19.0..11021.22.0

Also not reproducible on babytiger using ToT R71-11083.0.0

29d8c3ac looks like a candidate

Au also failed at post-install process..

Log info
---------------------------------------------------------------------
[0920/142605:INFO:image_properties_chromeos.cc(157)] lsb-release inside the old rootfs:
CHROMEOS_ARC_ANDROID_SDK_VERSION=25
CHROMEOS_ARC_VERSION=5013879
CHROMEOS_AUSERVER=https://tools.google.com/service/update2
CHROMEOS_BOARD_APPID={5A3AB642-2A67-470A-8F37-37E737A53CFC}
CHROMEOS_CANARY_APPID={90F229CE-83E2-4FAF-8479-E368A34938B1}
CHROMEOS_DEVSERVER=
CHROMEOS_RELEASE_APPID={5A3AB642-2A67-470A-8F37-37E737A53CFC}
CHROMEOS_RELEASE_BOARD=coral-signed-mp-v11keys
CHROMEOS_RELEASE_BRANCH_NUMBER=20
CHROMEOS_RELEASE_BUILDER_PATH=coral-release/R70-11021.20.0
CHROMEOS_RELEASE_BUILD_NUMBER=11021
CHROMEOS_RELEASE_BUILD_TYPE=Official Build
CHROMEOS_RELEASE_CHROME_MILESTONE=70
CHROMEOS_RELEASE_DESCRIPTION=11021.20.0 (Official Build) beta-channel coral-unibuild (astronaut babymega babytiger blacktip blacktip360 blue bruce coral epaulette lava nasher nasher360 porbeagle rabbid robo robo360 santa whitetip) 
CHROMEOS_RELEASE_KEYSET=mp-v11
CHROMEOS_RELEASE_MODELS=astronaut babymega babytiger blacktip blacktip360 blue bruce coral epaulette lava nasher nasher360 porbeagle rabbid robo robo360 santa whitetip
CHROMEOS_RELEASE_NAME=Chrome OS
CHROMEOS_RELEASE_PATCH_NUMBER=0
CHROMEOS_RELEASE_TRACK=beta-channel
CHROMEOS_RELEASE_UNIBUILD=1
CHROMEOS_RELEASE_VERSION=11021.20.0
DEVICETYPE=CHROMEBOOK
GOOGLE_RELEASE=11021.20.0

[0920/142605:INFO:action_processor.cc(116)] ActionProcessor: finished FilesystemVerifierAction with code ErrorCode::kSuccess
[0920/142605:INFO:action_processor.cc(143)] ActionProcessor: starting PostinstallRunnerAction
[0920/142605:INFO:postinstall_runner_action.cc(172)] Performing postinst (postinst at /tmp/.org.chromium.Chromium.3Wh0wR/postinst) installed on device /dev/mmcblk1p5 and mountable device /dev/mmcblk1p5
[0920/142605:INFO:postinstall_runner_action.cc(179)] Format file for new postinst is: data
[0920/142636:INFO:subprocess.cc(153)] Subprocess exited with si_status: 1
[0920/142636:INFO:subprocess.cc(157)] Subprocess output:
dm:dm bht[DEBUG] Setting block_count 473600
dm:dm bht[DEBUG] Setting depth to 3.
dm:dm bht[DEBUG] depth: 0 entries: 1
dm:dm bht[DEBUG] depth: 1 entries: 29
dm:dm bht[DEBUG] depth: 2 entries: 3700
PostInstall Configured: (B, /dev/mmcblk1p5, /dev/mmcblk1p4, /dev/mmcblk1p12)

 Current Kernel Info: sysname(Linux) nodename(localhost) release(4.4.153-15116-ga18af71a0d14) version(#1 SMP PREEMPT Tue Sep 18 12:57:30 PDT 2018) machine(x86_64)

lsb-release inside the new rootfs:
CHROMEOS_ARC_ANDROID_SDK_VERSION=25
CHROMEOS_ARC_VERSION=5017745
CHROMEOS_AUSERVER=https://tools.google.com/service/update2
CHROMEOS_BOARD_APPID={5A3AB642-2A67-470A-8F37-37E737A53CFC}
CHROMEOS_CANARY_APPID={90F229CE-83E2-4FAF-8479-E368A34938B1}
CHROMEOS_DEVSERVER=
CHROMEOS_RELEASE_APPID={5A3AB642-2A67-470A-8F37-37E737A53CFC}
CHROMEOS_RELEASE_BOARD=coral-signed-mp-v11keys
CHROMEOS_RELEASE_BRANCH_NUMBER=22
CHROMEOS_RELEASE_BUILDER_PATH=coral-release/R70-11021.22.0
CHROMEOS_RELEASE_BUILD_NUMBER=11021
CHROMEOS_RELEASE_BUILD_TYPE=Official Build
CHROMEOS_RELEASE_CHROME_MILESTONE=70
CHROMEOS_RELEASE_DESCRIPTION=11021.22.0 (Official Build) beta-channel coral-unibuild (astronaut babymega babytiger blacktip blacktip360 blue bruce coral epaulette lava nasher nasher360 porbeagle rabbid robo robo360 santa whitetip) 
CHROMEOS_RELEASE_KEYSET=mp-v11
CHROMEOS_RELEASE_MODELS=astronaut babymega babytiger blacktip blacktip360 blue bruce coral epaulette lava nasher nasher360 porbeagle rabbid robo robo360 santa whitetip
CHROMEOS_RELEASE_NAME=Chrome OS
CHROMEOS_RELEASE_PATCH_NUMBER=0
CHROMEOS_RELEASE_TRACK=beta-channel
CHROMEOS_RELEASE_UNIBUILD=1
CHROMEOS_RELEASE_VERSION=11021.22.0
DEVICETYPE=CHROMEBOOK
GOOGLE_RELEASE=11021.22.0

Set boot target to /dev/mmcblk1p5: Partition 5, Slot B
SetImage
KERNEL_CONFIG: console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 3788800 verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart=3788800 alg=sha1 root_hexdigest=7d49f544635e2fdff61b6b7980c9f0cfb7a60887 salt=b2d4ac80fc174f0ace6812adbca4eea8081341dbdefde42ddbc7c17142d8261f" noinitrd vt.global_cursor_default=0 kern_guid=%U add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic disablevmx=off  
Setting up verity.
Finished after 26 seconds.
Clearing network driver boot cache: /var/lib/preload-network-drivers.
Syncing filesystems before changing boot order...
Finished after 1 seconds.
Updating Partition Table Attributes using CgptManager...
Updated kernel 4 with Successful = 0 and NumTriesLeft = 6
Checking /mnt/stateful_partition/unencrypted permission.
Permission is ok.
Unlinked file /var/lib/ureadahead/opt.google.containers.android.rootfs.root.pack
Unlinked file /var/lib/ureadahead/pack
Starting firmware updater (//usr/sbin/chromeos-firmwareupdate --mode=autoupdate)
Command: //usr/sbin/chromeos-firmwareupdate --mode=autoupdate
Aborted (core dumped)
ERROR: Cannot get model from mosys.
ERROR: Execution failed: ./updater4.sh (error code = 1)
Finished after 4 seconds.
Failed Command: //usr/sbin/chromeos-firmwareupdate --mode=autoupdate - Exit Code 1
Firmware update failed (error code: 1).
Rolling back update due to failure installing required firmware.
Successfully updated GPT with all settings to rollback.
PostInstall Failed

log file attached.

Deleted: update_engine.20180920-141435.txt 79.9 KB

update_engine.20180920-141435.txt 79.9 KB View Download

Not reprod with TOT builds, successfully Auto updated to M71.

If mosys is failing that CL is probably suspect. Adding Jason who took over that CL for me.

What I don't get is why ToT is working -- if this were really that CL, there's no reason why ToT would fix itself unless another mosys CL landed with a fix.

Did someone find teh actual failing mosys command?

All R70 amd64 builds of mosys up to (but not including) 11021.23 were broken due to a bad seccomp filter. That was fixed by https://chromium.googlesource.com/chromiumos/platform/mosys/+/ba87f835e3aa63e07314dbfe182bc13a47c778bf which landed in 11021.23. Please retest with .23.

Recovery is working fine with 11021.23.0 build.

[Auto-generated comment by a script] We noticed that this issue is targeted for M-70; it appears the fix may have landed after branch point, meaning a merge might be required. Please confirm if a merge is required here - if so add Merge-Request-70 label, otherwise remove Merge-TBD label. Thanks.

and auto updated successfully to M70-11021.23.0 coral