New issue
Advanced search Search tips

Issue 734485 link

Starred by 0 users

Issue metadata

Status: Verified
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 717702



Sign in to add a comment

coral: signing broken due to broken firmware install

Project Member Reported by vapier@chromium.org, Jun 19 2017

Issue description

going to guess it's related to unibuild related changes as it started around June 16th/17th

190617 04:16 DEBUG: RunCommand: /cros/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /tmp/signer.uF6qBz/recovery_image.bin /cros/keys/CoralPreMPKeys /tmp/signer.uF6qBz/chromeos_9664.0.0_coral_recovery_canary-channel_premp.bin /cros/keys/CoralPreMPKeys/key.versions
190617 04:16 DEBUG: (stdout):
Extracting to: /tmp/tmp.jXpWLAK3bO
No dev firmware keyblock/datakey found. Reusing normal keys.

190617 04:16 DEBUG: (stderr):
sign_official_build.sh: INFO   : Using firmware version: 1
sign_official_build.sh: INFO   : Using kernel version: 1
sign_official_build.sh: INFO   : Preparing recovery image...
sign_official_build.sh: INFO   : Found a valid firmware update shellball.
Can't open /tmp/tmp.jXpWLAK3bO/bios*.bin: No such file or directory
Use --help for usage instructions

 

Comment 1 by sjg@google.com, Jun 19 2017

Yes, with unified builds the bios.bin file is in a subdirectory:

./models/pyro/bios.bin
./models/snappy/bios.bin
./models/reef/bios.bin

From a quick play it seems like this might involve non-trivial changes.

I'm not sure where the tests are for this 900-line shell script?

Comment 2 by vapier@chromium.org, Jun 19 2017

there are no unittests

Comment 3 by sjg@google.com, Jun 19 2017

OK ta. Should we rewrite it in Python and add some?

Comment 4 by vapier@chromium.org, Jun 19 2017

that's a pretty tall order.  i'm not against it ... in fact, i've proposed doing something similar for the keygen scripts.  i just want to be clear up front that this is going to be a bit of a yak shave that'll take more than a few days.

if you've got someone interested, then it might be pretty interesting.

although for this particular issue, i think the shell script as-is needs to be updated since coral is on fire now.

Comment 5 by sjg@google.com, Jun 19 2017

OK I'll see if I can make sense of it.

Comment 6 by shapiroc@google.com, Jun 19 2017

I'll take a look at fixing the shell script for now.

Comment 7 by sjg@google.com, Jun 19 2017

~/cosarm/chroot/build/reef-uni/usr/sbin/chromeos-firmwareupdate --sb_extract /tmp/x

 ./sign_official_build.sh firmware /build/reef-uni/usr/sbin/chromeos-firmwareupdate ../../tests/devkeys x



Cc: shapiroc@chromium.org

Comment 9 by sjg@google.com, Jun 19 2017

Owner: shapiroc@chromium.org
Is this bug urgent?

If you like you could do a temporary revert to get the signer running.

Comment 10 by sjg@google.com, Jun 19 2017

./sign_official_build.sh firmware /build/reef-uni/firmware/image-pyro.bin ../../tests/devkeys x

i'll leave it to you how to proceed, but as for the question "is it urgent", that depends on "is anyone using coral".  if they are, then yes, this is urgent because nothing in ToT is being signed, and nothing will be until this is resolved.

Comment 12 by sjg@google.com, Jun 19 2017

./build_image --board=reef-uni dev
./mod_image_for_recovery.sh --board=reef-uni
~/trunk/src/platform/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /mnt/host/source/src/build/images/reef-uni/R61-9666.0.2017_06_19_1511-a1/recovery_image.bin ../../tests/devkeys x

Comment 13 by sjg@google.com, Jun 19 2017

~/trunk/src/platform/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /mnt/host/source/src/build/images/reef-uni/R61-9666.0.2017_06_19_1511-a1/recovery_image.bin ~/trunk/src/platform/vboot_reference/tests/devkeys x

Comment 14 by sjg@google.com, Jun 19 2017

Just in case the local error helps:

~/trunk/src/platform/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /mnt/host/source/src/build/images/reef-uni/R61-9666.0.2017_06_19_1511-a1/recovery_image.bin ~/trunk/src/platform/vboot_reference/tests/devkeys x
sign_official_build.sh: INFO   : Using firmware version: 1
sign_official_build.sh: INFO   : Using kernel version: 1
sign_official_build.sh: INFO   : Preparing recovery image...
Extracting to: /tmp/tmp.BDLM1wS05a
sign_official_build.sh: INFO   : Found a valid firmware update shellball.
/home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/sign_firmware.sh /tmp/tmp.BDLM1wS05a/bios*.bin /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys /tmp/tmp.BDLM1wS05a/bios*.bin 1 
++ dirname /home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/sign_firmware.sh
+ SCRIPT_DIR=/home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing
+ . /home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/common_minimal.sh
+++ dirname /home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/sign_firmware.sh
++ SCRIPT_DIR=/home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing
+++ basename /home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/sign_firmware.sh
++ PROG=sign_firmware.sh
++ GPT=cgpt
++ TAG_NEEDS_TO_BE_SIGNED=/root/.need_to_be_signed
+++ mktemp
++ TEMP_FILE_LIST=/tmp/tmp.e3omW4h0Hp
+++ mktemp
++ TEMP_DIR_LIST=/tmp/tmp.5UWbaQjTld
++ trap cleanup_temps_and_mounts EXIT
++ trap cleanup_temps_and_mounts EXIT
+ set -e
+ main '/tmp/tmp.BDLM1wS05a/bios*.bin' /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys '/tmp/tmp.BDLM1wS05a/bios*.bin' 1 ''
+ [[ 5 -lt 3 ]]
+ [[ 5 -gt 5 ]]
+ local 'in_firmware=/tmp/tmp.BDLM1wS05a/bios*.bin'
+ local key_dir=/home/sjg/trunk/src/platform/vboot_reference/tests/devkeys
+ local 'out_firmware=/tmp/tmp.BDLM1wS05a/bios*.bin'
+ local firmware_version=1
+ local loem_output_dir=
++ make_temp_file
+++ mktemp
++ local tempfile=/tmp/tmp.8688pxakZg
++ echo /tmp/tmp.8688pxakZg
++ echo /tmp/tmp.8688pxakZg
+ local temp_fw=/tmp/tmp.8688pxakZg
+ [[ -e /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/loem.ini ]]
+ sign_one
+ local loem_key=
+ local loemid=
+ /home/sjg/trunk/src/platform/vboot_reference/scripts/image_signing/resign_firmwarefd.sh '/tmp/tmp.BDLM1wS05a/bios*.bin' /tmp/tmp.8688pxakZg /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/firmware_data_key.vbprivk /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/firmware.keyblock /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/dev_firmware_data_key.vbprivk /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/dev_firmware.keyblock /home/sjg/trunk/src/platform/vboot_reference/tests/devkeys/kernel_subkey.vbpubk 1 '' '' ''
No dev firmware keyblock/datakey found. Reusing normal keys.
Can't open /tmp/tmp.BDLM1wS05a/bios*.bin: No such file or directory
Use --help for usage instructions
+ cleanup_temps_and_mounts
++ cat /tmp/tmp.e3omW4h0Hp
+ for i in '$(cat $TEMP_FILE_LIST)'
+ rm -f /tmp/tmp.8688pxakZg
+ set +e
++ cat /tmp/tmp.5UWbaQjTld
+ set -e
+ rm -rf /tmp/tmp.5UWbaQjTld /tmp/tmp.e3omW4h0Hp

RE: #11

Yes, coral images are being used by ODM for various activities currently.
https://chromium-review.googlesource.com/c/540131/ should fix this once it makes it through CQ
I submitted a trybot build (w/ CL:540131) which completes successfully.

https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/release/builds/12280

Thanks.

Comment 18 by sjg@google.com, Jun 20 2017

Blocking: 717702
Project Member

Comment 19 by bugdroid1@chromium.org, Jun 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/vboot_reference/+/4007d6ff218110d55830c6dc2ca9822825afa0da

commit 4007d6ff218110d55830c6dc2ca9822825afa0da
Author: C Shapiro <shapiroc@google.com>
Date: Tue Jun 20 20:38:10 2017

Unified build support for multi-firmware signing

Unified builds break down multiple firmware images for each model;
however, the signing script didn't have support for this.

This updates the signing script to iterate over all models in a unified
build and sign each firmware image separately.

BUG= chromium:734485 
TEST=sign_official_build.sh recovery for reef and reef-uni
BRANCH=none

Change-Id: Ia2b5b8bd36ac77aeb7944362186d1d5739e6ff3d
Reviewed-on: https://chromium-review.googlesource.com/540131
Commit-Ready: C Shapiro <shapiroc@google.com>
Tested-by: C Shapiro <shapiroc@google.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/4007d6ff218110d55830c6dc2ca9822825afa0da/scripts/image_signing/sign_official_build.sh

Comment 20 by sjg@google.com, Jun 21 2017

Next problem is here:

https://uberchromegw.corp.google.com/i/chromeos/builders/coral-release/builds/74/steps/steps/logs/stdio


        "details": "Traceback (most recent call last):\n  File \"//cros/signer/signing_poller.py\", line 944, in SignArtifacts\n    work_dir, insn_cfg, metadata)\n  File \"//cros/signer/signing_poller.py\", line 676, in SignLocalArtifact\n    output_names)\n  File \"//cros/signer/signing_poller.py\", line 522, in SignImage\n    self.signer_if.SignImage(input_image, itype, keyset, output)\n  File \"/cros/signer/image_signer.py\", line 450, in SignImage\n    self.ExecVbootScript(signing_cmd, extra_env=extra_env)\n  File \"/cros/signer/image_signer.py\", line 403, in ExecVbootScript\n    **kwargs)\n  File \"/cros/signer/image_signer.py\", line 373, in ExecCrosScript\n    return cros_build_lib.RunCommand(cmd, **kwargs)\n  File \"/cros/signer/lib/chromite_init.py\", line 35, in RunCommand\n    return OrigRunCommand(cmd, **kwds)\n  File \"/cros/signer/chromite/lib/cros_build_lib.py\", line 624, in RunCommand\n    raise RunCommandError(msg, cmd_result)\nRunCommandError: return code: 1; command: /cros/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /tmp/signer.yN0G4x/recovery_image.bin /cros/keys/CoralPreMPKeys /tmp/signer.yN0G4x/chromeos_9670.0.0_coral_recovery_canary-channel_premp.bin /cros/keys/CoralPreMPKeys/key.versions\n\u001b[1;32msign_official_build.sh: INFO   : Using firmware version: 1\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Using kernel version: 1\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Preparing recovery image...\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Found a valid firmware update shellball.\u001b[0m\nCan't open /tmp/tmp.TiTDlPIsup/bios*.bin: No such file or directory\nUse --help for usage instructions\n\nExtracting to: /tmp/tmp.TiTDlPIsup\nNo dev firmware keyblock/datakey found. Reusing normal keys.\n\ncwd=None\n", 
        "summary": "return code: 1; command: /cros/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /tmp/signer.yN0G4x/recovery_image.bin /cros/keys/CoralPreMPKeys /tmp/signer.yN0G4x/chromeos_9670.0.0_coral_recovery_canary-channel_premp.bin /cros/keys/CoralPreMPKeys/key.versions\n\u001b[1;32msign_official_build.sh: INFO   : Using firmware version: 1\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Using kernel version: 1\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Preparing recovery image...\u001b[0m\n\u001b[1;32msign_official_build.sh: INFO   : Found a valid firmware update shellball.\u001b[0m\nCan't open /tmp/tmp.TiTDlPIsup/bios*.bin: No such file or directory\nUse --help for usage instructions\n\nExtracting to: /tmp/tmp.TiTDlPIsup\nNo dev firmware keyblock/datakey found. Reusing normal keys.\n\ncwd=None"
    }, 


it's the same problem ;).  you'll need to update the vboot hash in the signer repo.  like this:
https://chrome-internal-review.googlesource.com/382528
Project Member

Comment 22 by bugdroid1@chromium.org, Jun 21 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/cros-signing/+/dd3ee0116a09f5ce918398264b3b212d02d2a5f5

commit dd3ee0116a09f5ce918398264b3b212d02d2a5f5
Author: C Shapiro <shapiroc@google.com>
Date: Wed Jun 21 21:15:30 2017

Comment 24 by sjg@google.com, Jul 5 2017

Labels: Team-BLD

Sign in to add a comment