New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 923496 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Today
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug-Regression



Sign in to add a comment

reef: M72 and M73 mp-v2 recovery image fails at firmware updater step due to model name mismatch.

Project Member Reported by bleung@chromium.org, Jan 18 (4 days ago)

Issue description

Attempting to use canary-channel_reef_11606.0.0_chromeos_11606.0.0_reef_recovery_canary-channel_mp-v2.bin on a REEF based system. Confirmed to be an Electro variant.


Starting firmware updater (//usr/sbin/chromeos-firmwareupdate --mode=recovery)
Command: //usr/sbin/chromeos-firmwareupdate --mode=recovery

  Machine Model: 
  Write Protect: HW=1/1 SW=0
  Last Boot Version: RO=Google_Reef.9042.206.0 ACT/recovery=Google_Reef.9042.206.0
  Firmware Updater:  {
  "basking": {
    "host": { "versions": { "ro": "Google_Reef.9042.206.0", "rw": "Google_Reef.9042.206.0" },
      "keys": { "root": "2f8cf1b72a00cbd97f8154dbcf3352129fd8e885", "recovery": "6dcd7f2855597d15a2e44db14958a9ea8fdc6e14" },
      "image": "images/bios-reef.ro-9042-206-0.rw-9042-206-0.bin" },
    "ec": { "versions": { "ro": "reef_v1.1.5947-5e87552", "rw": "reef_v1.1.5947-5e87552" },
      "image": "images/ec-reef.ro-1-1-5947.rw-1-1-5947.bin" },
    "patches": { "rootkey": "keyset/rootkey.basking", "vblock_a": "keyset/vblock_A.basking", "vblock_b": "keyset/vblock_B.basking" },
    "signature_id": "basking"
  },
  "electro": {
    "host": { "versions": { "ro": "Google_Reef.9042.206.0", "rw": "Google_Reef.9042.206.0" },
      "keys": { "root": "ac7c01b1bea84da486f30a52bba5eb67ff45f50f", "recovery": "6dcd7f2855597d15a2e44db14958a9ea8fdc6e14" },
      "image": "images/bios-reef.ro-9042-206-0.rw-9042-206-0.bin" },
    "ec": { "versions": { "ro": "reef_v1.1.5947-5e87552", "rw": "reef_v1.1.5947-5e87552" },
      "image": "images/ec-reef.ro-1-1-5947.rw-1-1-5947.bin" },
    "patches": { "rootkey": "keyset/rootkey.electro", "vblock_a": "keyset/vblock_A.electro", "vblock_b": "keyset/vblock_B.electro" },
    "signature_id": "electro"
  }
}
  
Platform not supported
Application error: Platform not supported
ERROR: manifest_find_model: Cannot get model name.
You are probably running an image for wrong board, or a device in early stage that 'mosys' command is not ready, or image from old (or factory) branches that Unified Build config is not updated yet for 'mosys'.
Please check command 'mosys platform model', which should output one of the supported models below:
 electro basking
ERROR: Execution failed: (error code = 1)
Finished after 1 seconds.
Failed Command: //usr/sbin/chromeos-firmwareupdate --mode=recovery - Exit Code 1
Firmware update failed (error code: 1).
Rolling back update due to failure installing required firmware.
Successfully updated GPT with all settings to rollback.
PostInstall Failed
 
dmesg
43.9 KB View Download
recovery.log
237 KB View Download

Comment 1 by bleung@chromium.org, Jan 18 (4 days ago)

Labels: ReleaseBlock-Stable
Setting RBS to make sure this is looked at before 73 goes stable, with busted recovery images.

Comment 2 by bleung@chromium.org, Jan 18 (4 days ago)

Labels: Proj-Reef

Comment 3 by bleung@chromium.org, Jan 18 (4 days ago)

Today's dev channel dev-channel_reef_11578.0.0_chromeos_11578.0.0_reef_recovery_dev-channel_mp-v2.bin  also fails.

I'm going to check beta, M72 beta-channel_reef_11316.82.0_chromeos_11316.82.0_reef_recovery_beta-channel_mp-v2.bin and try to bisect when this failed.
recovery.log
241 KB View Download

Comment 4 by bleung@chromium.org, Jan 18 (4 days ago)

Owner: bleung@chromium.org
Status: Started (was: Unconfirmed)

Comment 5 by bleung@chromium.org, Jan 18 (4 days ago)

Labels: M-72
Summary: reef: M72 and M73 mp-v2 recovery image fails at firmware updater step due to model name mismatch. (was: reef: 11606.0 mp-v2 recovery image fails at firmware updater step due to model name mismatch.)
Not good. Looks like Beta channel is also affected by this.

My bisecting shows that M71 beta-channel_reef_11151.59.0_chromeos_11151.59.0_reef_recovery_beta-channel_mp-v2.bin successfully recovers, but something changed in M72 that broke this. 
recovery.log
223 KB View Download

Comment 6 by bleung@chromium.org, Jan 18 (4 days ago)

Cc: djmm@chromium.org

Comment 7 by bleung@google.com, Jan 18 (4 days ago)

Components: Infra>Client>ChromeOS

Comment 8 by bleung@google.com, Jan 19 (4 days ago)

Cc: shapiroc@google.com shapiroc@chromium.org yueherngl@chromium.org mruthven@chromium.org
Labels: -Type-Bug Type-Bug-Regression
Owner: shapiroc@chromium.org
Status: Assigned (was: Started)
Bisect completed. First recovery image that failed was canary-channel_reef_11233.0.0_chromeos_11233.0.0_reef_recovery_canary-channel_mp-v2.bin

Looking at the changes between 11232 and 11234, this one looks suspicious:
https://chrome-internal.googlesource.com/chromeos/overlays/overlay-reef-private/+/f324ac19fdf6227822ecc238918f179e6860a662

Charles, can you take a look at this? Reef recovery images fail at the firmware update step on my Electro system since early November on M72 and M73.

Comment 9 by bleung@google.com, Jan 19 (4 days ago)

Labels: Proj-Electro

Comment 10 by bleung@google.com, Jan 19 (4 days ago)

Before that build:
localhost ~ # mosys platform model
electro

Starting on 11233, after the change above:
localhost ~ # mosys platform model
cros_config_fdt_err: find mapping: FDT_ERR_NOTFOUND
internal_cros_config_read_sku_info: Failed to read master configuration
Platform not supported
Application error: Platform not supported

Comment 11 by shapiroc@chromium.org, Jan 20 (2 days ago)

this is indicative of having an old version of mosys in the fw shellball.

i'll take a look at this.

Comment 12 by shapiroc@chromium.org, Jan 20 (2 days ago)

Owner: hungte@chromium.org
comment #8-#10 looks like a red herring.  the output from mosys is from a very old version of mosys where FDT has since been fully removed

based on the original log, this looks more closely aligned to the changes around futility and the fw updater scripts

hungte@ ... can you take a look at this

Comment 13 by hungte@chromium.org, Jan 20 (2 days ago)

Owner: shapiroc@chromium.org
I don't see anything obious related to firmware updater / futility. It should be the problem in mosys. Firmware updater needs $(mosys platform model) to tell it which firmware to select in unified build. From all logs here, mosys failed to report right model.

> this is indicative of having an old version of mosys in the fw shellball.

No, fw shellball does not contain programs anymore.

As bleung stated, "Starting on 11233, after the change above:
localhost ~ # mosys platform model"

He executed mosys directly, so it's not "old version of mosys".

Sounds like the reef you're testing has problem detecting model. Maybe lack of VPD? Also check b/120241063.

Comment 14 by bleung@google.com, Jan 20 (2 days ago)

Cc: hungte@chromium.org
I believe that my Electro is a DVT phase model, and that I didn't mess with the VPD. I will try to find my PVT or MP model and give this a try too on Tuesday.

Comment 15 by hungte@google.com, Jan 21 (2 days ago)

@bleung, you can do "vpd -l" to see if customization_id was there.

Comment 16 by shapiroc@chromium.org, Yesterday (44 hours ago)

Owner: bleung@chromium.org
thanks hungte@ for the direction here

Comment 17 by bleung@chromium.org, Today (12 hours ago)

Hey hungte@

It looks like my Electro DVT did not have the customization_id set to anything. The field is blank.

I'll check my PVT device.

Comment 18 by bleung@chromium.org, Today (11 hours ago)

Status: WontFix (was: Assigned)
Thanks for clearing this up hungte@. My PVT system has a customization_id of "PARMA_ELECTRO" and correctly works.

I guess DVT systems are considered deprecated at this point because they came with the wrong customization_id?


Comment 19 by shapiroc@chromium.org, Today (10 hours ago)

this VPD value is required to select the correct signing key

Comment 20 by hungte@chromium.org, Today (4 hours ago)

@bleung you can also provision that value by yourself.
Also, devices may have VPD blew away if people reflash firmware without using standard firmware updater (i.e., if they simply run servo or flashrom directly).

I believe we can solve this by changing mosys/unibuild config to identify device by SKU ID instead of VPD values anyway.

Comment 21 by hungte@chromium.org, Today (4 hours ago)

In fact, non-uni reef mosys does identify device properly. It's unibuild that used VPD based approach...

Comment 22 by hungte@chromium.org, Today (4 hours ago)

I found this caused by  https://chrome-internal-review.googlesource.com/c/709777 , which was going to fix b/119032930, and exactly M72 issue.

Before that, we can identify electro and basking correctly by sku-id even if VPD was not properly provisioned.

And after that landed, all reef units without VPD are failing.

@Charles, are we sure if that CL is really needed? Maybe that's not the right fix?

Sign in to add a comment