New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 887982 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 21
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Nautilus failed signing (and signertest)

Project Member Reported by bhthompson@google.com, Sep 21

Issue description

https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8934797435741259712

https://luci-logdog.appspot.com/v/?s=chromeos/buildbucket/cr-buildbucket.appspot.com/8934797435741259712/+/steps/SignerTest/0/stdout

...
03:38:15: INFO: RunCommand: /b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpLkofHE' -- ./security_test_image '--board=nautilus' in /b/swarming/w/ir/cache/cbuild/repository
INFO    security_test_image: Loading baselines from /mnt/host/source/cros-signing/security_test_baselines
INFO    security_test_image: Using /mnt/host/source/src/build/images/nautilus/R70-11021.23.0/recovery_image.bin
INFO    security_test_image: Using vboot_reference.git rev 661cca6bf64a92a8a4d72196c0b69e0da7a423b5
INFO    security_test_image: Running ensure_no_nonrelease_files.sh
INFO    security_test_image: Running ensure_sane_lsb-release.sh
ensure_sane_lsb-release.sh: INFO   : Loading config from /mnt/host/source/cros-signing/security_test_baselines/ensure_sane_lsb-release.config
INFO    security_test_image: Running ensure_secure_kernelparams.sh
sed: -e expression #1, char 75: Invalid back reference
Unexpected kernel parameters found:
  usbcore.quirks=2cb7:0007:k 
Debug output:
required_kparams=(
	'cros_secure'
	'dm_verity.error_behavior=3'
	'dm_verity.max_bios=-1'
	'dm_verity.dev_wait=1'
	'init=/sbin/init'
	'ro'
	'rootwait'
	'add_efi_memmap'
	'boot=local'
	'i915.modeset=1'
	'nmi_watchdog=panic,lapic'
	'noresume'
	'noswap'
	'tpm_tis.force=1'
	'tpm_tis.interrupts=0'
)
required_kparams_regex=(
	'root=/dev/dm-[0-1]'
	'kern_guid=\(PARTUUID=\)\?%U'
)
optional_kparams=(
	'console='
	'console=tty2'
	'cros_recovery'
	'disablevmx=off'
	'earlyprintk'
	'i8042.nomux=1'
	'idle=halt'
	'noinitrd'
	'oops=panic'
	'panic=-1'
	'panic=60'
	'quiet'
	'vt.global_cursor_default=0'
)
optional_kparams_regex=(
	'earlycon=[a-z0-9,]*'
	'gsmi.s0ix_logging_enable=[01]'
	'i915.alpha_support=[01]'
	'i915.enable_dbc=[01]'
	'i915.enable_dpcd_backlight=[01]'
	'i915.enable_execlists=[01]'
	'i915.enable_psr=[01]'
	'i915.preliminary_hw_support=[01]'
	'intel_idle.max_cstate=[0-9]'
	'intel_idle.slp_s0_check=[01YN]'
	'intel_idle.slp_s0_seed=[0-9]\+'
	'iwlwifi.remove_when_gone=[01]'
	'iTCO_vendor_support.vendorsupport=[0-9]'
	'kern_b_hash=[a-f0-9]*'
	'loglevel=[0-9]'
	'maxcpus=[0-9]\+'
	'ramoops.ecc=[0-9]\+'
)
required_dmparams=(
	''
)
required_dmparams_regex=(
	'vroot none ro,0 [1-9][0-9]\{6\} verity %U+1 %U+1  0 sha1 MAGIC_HASH'
	'vroot none ro,0 [1-9][0-9]\{6\} verity payload=%U+1 hashtree=%U+1 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH'
	'vroot none ro,0 [1-9][0-9]\{6\} verity payload=%U+1 hashtree=%U+1 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
	'1 vroot none ro 1,0 [1-9][0-9]\{6\} verity payload=%U+1 hashtree=%U+1 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
	'2 vboot none ro 1,0 \(1768000\|2129920\|2545920\|2579200\) bootcache %U+1  MAGIC_HASH [0-9]\+ [0-9]\+ [0-9]\+, vroot none ro 1,0 [1-9][0-9]\{6\} verity payload=254:0 hashtree=254:0 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
	'1 vroot none ro 1,0 [1-9][0-9]\{6\} verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
	'2 vboot none ro 1,0 \(1768000\|2129920\|2545920\|2579200\) bootcache PARTUUID=%U/PARTNROFF=1  MAGIC_HASH [0-9]\+ [0-9]\+ [0-9]\+, vroot none ro 1,0 [1-9][0-9]\{6\} verity payload=254:0 hashtree=254:0 hashstart= alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
)
kparams='console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 4194304 verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart=4194304 alg=sha1 root_hexdigest=6abbbc5eb2fa3a3048ce1f1295f87c2d8609b2ea salt=ab1b2e37ad9492aecc453a57a6ca9381e5603079f294c6479b42cc7d1e4a8e94" noinitrd vt.global_cursor_default=0 kern_guid=%U add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic intel_idle.slp_s0_check=1 intel_idle.slp_s0_seed=5 gsmi.s0ix_logging_enable=1 ramoops.ecc=1 disablevmx=off usbcore.quirks=2cb7:0007:k '
dmparams='1 vroot none ro 1,0 4194304 verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart=4194304 alg=sha1 root_hexdigest=6abbbc5eb2fa3a3048ce1f1295f87c2d8609b2ea salt=ab1b2e37ad9492aecc453a57a6ca9381e5603079f294c6479b42cc7d1e4a8e94'
kparams_nodm='console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1  noinitrd vt.global_cursor_default=0 kern_guid=%U add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic intel_idle.slp_s0_check=1 intel_idle.slp_s0_seed=5 gsmi.s0ix_logging_enable=1 ramoops.ecc=1 disablevmx=off usbcore.quirks=2cb7:0007:k '
mangled_dmparams='1 vroot none ro 1,0 4194304 verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart=4194304 alg=sha1 root_hexdigest=MAGIC_HASH salt=MAGIC_SALT'
(actual error will be at the top of output)
ERROR   security_test_image: secure_kernelparams: test failed
INFO    security_test_image: Running ensure_not_ASAN.sh
ERROR   security_test_image: 1 tests failed
03:38:35: ERROR: 
return code: 1; command: /b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpLkofHE' -- ./security_test_image '--board=nautilus'
cmd=['/b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_sdk', 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpLkofHE', '--', './security_test_image', u'--board=nautilus'], cwd=/b/swarming/w/ir/cache/cbuild/repository, extra env={'PARALLEL_EMERGE_STATUS_FILE': '/tmp/tmpLkofHE'}
03:38:35: ERROR: ./security_test_image failed (code=1)
03:38:35: INFO: Translating result ./security_test_image failed (code=1) to fail.
...

Failure started around https://crosland.corp.google.com/log/11076.0.0..11077.0.0

I suspect https://chromium-review.googlesource.com/c/chromiumos/overlays/board-overlays/+/1220287

This appears to have been picked to 70 as well and has caused the build to fail there also. 

Please put in a fix so the test passes or revert the change ASAP, this blocks all builds from Nautilus.
 
Owner: rajatja@chromium.org
Passing to Rajat as the reviewer of the CL that broke the build.
Cc: alexpau@chromium.org
Owner: rajatja@google.com
Ah, the new module parameter needs to be added to security_test_baselines/ensure_secure_kernelparams.config. I'll send a CL. Question: Now that the build is broken, would I need to chump it in, or go through the CQ?
The CQ will still pass as it does not do any signing, I am also ok with chumping it for the same reason (and I am sheriff this week).

Project Member

Comment 6 by bugdroid1@chromium.org, Sep 21

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/cros-signing/+/5b42d3a43d21f044c52855b910446fbd6e2281da

commit 5b42d3a43d21f044c52855b910446fbd6e2281da
Author: Rajat Jain <rajatja@google.com>
Date: Fri Sep 21 17:36:36 2018

Labels: Merge-Request-70
Project Member

Comment 8 by sheriffbot@chromium.org, Sep 21

Labels: -Merge-Request-70 Merge-Review-70 Hotlist-Merge-Review
This bug requires manual review: M70 has already been promoted to the beta branch, so this requires manual review
Please contact the milestone owner if you have questions.
Owners: benmason@(Android), kariahda@(iOS), geohsu@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Merge-Review-70 Merge-Approved-70
Project Member

Comment 10 by bugdroid1@chromium.org, Sep 21

Labels: merge-merged-release-R70-11021.B
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/cros-signing/+/6ac4edc8c856f71da7d7057c5f6adb55eb22d413

commit 6ac4edc8c856f71da7d7057c5f6adb55eb22d413
Author: Rajat Jain <rajatja@google.com>
Date: Fri Sep 21 18:13:32 2018

Status: Fixed (was: Untriaged)
I requeued all the failing Nautilus jobs and they all passed now
Project Member

Comment 13 by sheriffbot@chromium.org, Sep 25

This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 14 by sheriffbot@chromium.org, Sep 28

This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible!

If all merges have been completed, please remove any remaining Merge-Approved labels from this issue.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Sign in to add a comment