New issue
Advanced search Search tips

Issue 902546 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Nov 17
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression



Sign in to add a comment

[Coral - Nasher360] Recovery is failing with unexpected error - Firmware update failed

Project Member Reported by pgangishetty@google.com, Nov 6

Issue description

Chrome Version: 72.0.3602.0
OS: 11232.0.0

What steps will reproduce the problem?
Try to recover the device using USB recovery stick.

What is the expected result?
Installation should be completed successfully.
 

What happens instead?
Fails with unexpected error.

From logs:
-------------------------------------
DO NOT REBOOT OR POWEROFF!
FAILED
ERROR: Failed writing firmware.
FAILED: Firmware updater stopped due to error.
Target image: models/coral_71ro_76rw/bios.bin (RO:Google_Coral.10068.71.0, RW/A:Google_Coral.10068.76.0, RW/B:Google_Coral.10068.76.0).
Loading current system firmware...
Current system: /tmp/fwupdater.oeCKH0 (RO:Google_Coral.10068.71.0, RW/A:Google_Coral.10068.71.0, RW/B:Google_Coral.10068.71.0).
Write protection: 0 (disabled; HW=1, SW=0).
FULL UPDATE: Updating whole firmware image(s), RO+RW.
Checking compatibility...
Executing: flashrom -w /tmp/fwupdater.WufJNa -p host    
Executing: flashrom -w /tmp/fwupdater.C0o46Z -p ec    
ERROR: Execution failed: ./updater5.sh (error code = 1)
Finished after 65 seconds.
Failed Command: //usr/sbin/chromeos-firmwareupdate --mode=recovery - Exit Code 1
Firmware update failed (error code: 1).
Rolling back update due to failure installing required firmware.
Successfully updated GPT with all settings to rollback.
PostInstall Failed

-----------------------------------------

Note:  Logs attached. 
 
Labels: -Pri-3 Pri-1
Labels: OS-Chrome
Did not notice issue on updating nasher360 device from M70 (11021.56.0, 70.0.3538.76 stable-channel) to M72 (11232.0.0, 72.0.3602.0 dev-channel). AU was successful.
Cc: mkarkada@chromium.org
AU was successful from M70 (11021.34.0, 70.0.3538.41 beta-channel) to M72 (11232.0.0, 72.0.3602.0 dev-channel) on epaulette device.
Did you give right URL to logs?
From the link I can only find success:

>> futility update -a .  --mode=recovery
>> Firmware updater started.
flashrom v0.9.9  : f238fa6 : Nov 02 2018 21:50:43 UTC on Linux 4.14.78-07952-gfd7ac64fb424 (x86_64)
Calibrating delay loop... OK.
coreboot table found at 0x79b2c000.
Erasing and writing flash chip... Verifying flash... VERIFIED.          
SUCCESS
flashrom v0.9.9  : f238fa6 : Nov 02 2018 21:50:43 UTC on Linux 4.14.78-07952-gfd7ac64fb424 (x86_64)
Calibrating delay loop... OK.
coreboot table found at 0x79b2c000.
Skip jumping to RO
Erasing and writing flash chip... Verifying flash... VERIFIED.          
SUCCESS
>> DONE: Firmware updater exited successfully.
Re: #7 - Oops.. sorry about that.  I have attached the correct recovery.log in the same link above.  
I looked at that device and found that 
  - HW WP is enabled 
  - host SW WP is Disabled
  - EC SW WP is Enabled

Logs from c#1: 

i think the below failure is expected, since host sw wp disabled, updater tried RO+RW update. host firmware updated successfully but failed during EC update because ec sw wp is in enabled state. 

>> futility update -a .  --mode=recovery
>> Firmware updater started.
flashrom v0.9.9  : f238fa6 : Nov 02 2018 21:50:43 UTC on Linux 4.4.162-15455-gda5fb0f6c59b (x86_64)
Calibrating delay loop... OK.
coreboot table found at 0x7ab33000.
WARNING: SPI Configuration Lockdown activated.
Erasing and writing flash chip... Verifying flash... VERIFIED.          
SUCCESS
flashrom v0.9.9  : f238fa6 : Nov 02 2018 21:50:43 UTC on Linux 4.4.162-15455-gda5fb0f6c59b (x86_64)
Calibrating delay loop... OK.
coreboot table found at 0x7ab33000.
WARNING: SPI Configuration Lockdown activated.
FAILED: RO_AT_BOOT is not clear.
Erasing and writing flash chip... Erasing and writing flash chip... ERASE FAILED at 0x00000000! Expected=0xff, Read=0x5e, failed byte count from 0x00000000-0x00000fff: 0xf8
 ERASE_FAILED
FAILED!
Uh oh. CROS_EC 2nd pass failed.
Your flash chip is in an unknown state.
Get help on IRC at irc.freenode.net (channel #flashrom) or
mail flashrom@flashrom.org with FAILED: your board name in the subject line!
-------------------------------------------------------------------------------
DO NOT REBOOT OR POWEROFF!
FAILED
ERROR: Failed writing firmware.
>> FAILED: Firmware updater stopped due to error.
>> Target image: models/coral_71ro_76rw/bios.bin (RO:Google_Coral.10068.71.0, RW/A:Google_Coral.10068.76.0, RW/B:Google_Coral.10068.76.0).
Loading current system firmware...
>> Current system: /tmp/fwupdater.oeCKH0 (RO:Google_Coral.10068.71.0, RW/A:Google_Coral.10068.71.0, RW/B:Google_Coral.10068.71.0).
>> Write protection: 0 (disabled; HW=1, SW=0).
>> FULL UPDATE: Updating whole firmware image(s), RO+RW.
Checking compatibility...
Executing: flashrom -w /tmp/fwupdater.WufJNa -p host    
Executing: flashrom -w /tmp/fwupdater.C0o46Z -p ec    
ERROR: Execution failed: ./updater5.sh (error code = 1)
Finished after 65 seconds.
Failed Command: //usr/sbin/chromeos-firmwareupdate --mode=recovery - Exit Code 1
Firmware update failed (error code: 1).


Brought the device back with R71 test image 11151.19.0:
Then:

1. Recovered with Recovery image 72.11238.0.0 & 72.11232.0.0 - both failed
2. R71 Beta Test image ---> R71 Beta Recovery image -- Recovery was successful.

At this point venkataraju@ looked at the device and enabled the host SW WP.

3. Then tried to recover from R71 Beta recovery image --> R72 Dev recovery image ---> successful recovery.  


Summary: [Coral - Nasher360] Recovery is failing with unexpected error - Firmware update failed (was: [Coral] Recovery is failing with unexpected error - Firmware update failed )
Labels: -ReleaseBlock-Dev
I've also made a patch so it'll work more like before:
https://chromium-review.googlesource.com/c/chromiumos/platform/vboot_reference/+/1322295
Project Member

Comment 14 by bugdroid1@chromium.org, Nov 15

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/50354b924b13e780a398c5d6b57c726992049d6e

commit 50354b924b13e780a398c5d6b57c726992049d6e
Author: Hung-Te Lin <hungte@chromium.org>
Date: Thu Nov 15 05:30:31 2018

futility: updater: Check EC/PD WP state again before updating

There are devices, especially during or after RMA, may have WP states
not synced; for example
 HW =      1
 SW (AP) = 0
 SW (EC) = 1

In this case, we can still update host firmware but not EC. This happens
more often on EC that needs an extra reboot to change WP states.

As a result, we do want to check real programmer again before updating
optional images.

BUG= chromium:902546 
TEST=make futil; tests/futility/run_test_scripts.sh $(pwd)/build/futility
BRANCH=None

Change-Id: I9a526cde19a1ab3c41afecb4f7247bd941edc3f4
Signed-off-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1322295
Reviewed-by: Randall Spangler <rspangler@chromium.org>

[modify] https://crrev.com/50354b924b13e780a398c5d6b57c726992049d6e/futility/updater.c

Owner: hungte@chromium.org
Status: Fixed (was: Untriaged)

Sign in to add a comment