New issue
Advanced search Search tips

Issue 910705 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Dec 27
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Provisioning failed on veyron_minnie-chrome-pfq with "Filesystem hash verification failed"

Project Member Reported by derat@chromium.org, Nov 30

Issue description

Last night's Chrome PFQ run failed due to a provisioning error in the veyron_minnie-chrome-pfq run at http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8928452740594651024:

"provision: FAIL: Download and install failed from chromeos4-devserver9.cros.corp.google.com onto chromeos4-row9-rack9-host21: command execution error"

In the logs at http://stainless/browse/chromeos-autotest-results/262496262-chromeos-test/, I see the following in https://storage.cloud.google.com/chromeos-autotest-results/262496262-chromeos-test/chromeos4-row9-rack9-host21/debug/autoserv.ERROR:

...
Set boot target to /dev/mmcblk0p3: Partition 3, Slot A
SetImage
KERNEL_CONFIG: console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 3334144 verity payload=PARTUUID=%U/PARTNROFF=1 hashtree=PARTUUID=%U/PARTNROFF=1 hashstart=3334144 alg=sha1 root_hexdigest=b29b7b27188d6bc42b4cfa6ca3d0376c2e5858af salt=97cadae5fc23783744fd49aef618c4696bdf82316c723475fe4e264376951896" noinitrd cros_debug vt.global_cursor_default=0 kern_guid=%U 
Setting up verity.
Finished after 53 seconds.
SetImage failed.
PostInstall Failed
Downloaded hashes prior to postinst failure:
d4597ea7803d2893e574f0e1371bcfa3  -
8bbba3999f9a245071d40410f2a40a4f  -
5d99a1239331daf75b5e9dd57bc4681a  -
2018-11-30 01:13:56-08:00 ERROR: FATAL: postinst failed.
2018-11-30 01:13:56-08:00 INFO: Updated status: FATAL: postinst failed.
11/30 01:16:09.823 ERROR|       autoupdater:1006| Failure during download and install.
...
Setting up verity.
Finished after 65 seconds.
SetImage failed.
PostInstall Failed
...

I think that this may be the relevant error (from the same file):

2018-11-30 01:13:02-08:00 INFO: Update next kernel to try (via postinst)
dm:dm bht[DEBUG] Setting block_count 416768
dm:dm bht[DEBUG] Setting depth to 3.
dm:dm bht[DEBUG] depth: 0 entries: 1
dm:dm bht[DEBUG] depth: 1 entries: 26
dm:dm bht[DEBUG] depth: 2 entries: 3256
[1130/011356:ERROR:chromeos_verity.cc(280)] Filesystem hash verification failed
[1130/011356:ERROR:chromeos_verity.cc(281)] Expected b29b7b27188d6bc42b4cfa6ca3d0376c2e5858af != actual c6e624b23c2816295f318ca25080c67f8dee8319
PostInstall Configured: (A, /dev/mmcblk0p3, /dev/mmcblk0p2, /dev/mmcblk0p12)

 Issue 899583  looks like it was an occurrence of the same error on tricky-chrome-pfq from a month ago, but it was archived without any resolution being stated.

Allen (current deputy), please investigate this to figure out what happened and how it can be prevented. Would a retry have helped? Do we need to verify checksums at some earlier step?
 
Labels: Hotlist-Deputy
If I had to guess, I'd guess bad/failing disk.

I'd suggest the following:
- verify the image on another DUT to rule out a bad image
- check the logs/SMART on this DUT to see evidence of any write I/O failures
- check historic logs of this device and see if there is a history of failures

Assuming all of those simple cases don't show up anything, I'd trying the following to collect more information
- retry the same image on this DUT a few times via both quick provision and legacy flow
- try a different image on this same DUT

There should not be any need to add a retry.  That would just be papering up over legitimate issues.
Owner: pprabhu@chromium.org
Status: ExternalDependency (was: Assigned)
That DUT hasn't passed a single provision attempt in the last 24 hours

pprabhu@pprabhu:chromiumos-overlay$ dut-status -f chromeos4-row9-rack9-host21
chromeos4-row9-rack9-host21
    2018-12-03 15:57:50  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2874612-repair/
    2018-12-03 15:56:37  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2874599-verify/
    2018-12-03 15:00:47  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2874356-repair/
    2018-12-03 14:51:24  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2874299-provision/
    2018-12-03 09:51:10  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2873254-repair/
    2018-12-03 09:49:53  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2873239-verify/
    2018-12-03 09:10:50  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2873106-repair/
    2018-12-03 09:02:05  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2873063-provision/
    2018-12-03 08:52:18  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872977-repair/
    2018-12-03 08:51:02  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872965-verify/
    2018-12-03 08:10:54  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872671-repair/
    2018-12-03 08:01:59  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872606-provision/
    2018-12-03 07:51:53  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872539-repair/
    2018-12-03 07:50:36  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872531-verify/
    2018-12-03 06:56:35  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872194-repair/
    2018-12-03 06:47:05  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872151-provision/
    2018-12-03 06:37:15  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872098-repair/
    2018-12-03 06:35:57  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2872089-verify/
    2018-12-03 05:34:53  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871846-repair/
    2018-12-03 05:26:09  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871815-provision/
    2018-12-03 05:25:41  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/263216125-chromeos-test/
    2018-12-03 05:24:53  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871809-reset/
    2018-12-03 05:18:26  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/263211954-chromeos-test/
    2018-12-03 05:17:44  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871766-reset/
    2018-12-03 05:16:21  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871757-repair/
    2018-12-03 05:14:37  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871735-reset/
    2018-12-03 05:10:09  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/263211938-chromeos-test/
    2018-12-03 05:09:28  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871692-reset/
    2018-12-03 05:07:57  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871689-repair/
    2018-12-03 05:06:12  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871683-reset/
    2018-12-03 05:01:22  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/263211924-chromeos-test/
    2018-12-03 05:00:39  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871656-reset/
    2018-12-03 04:59:09  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871650-repair/
    2018-12-03 04:57:27  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871645-reset/
    2018-12-03 04:52:24  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/263211915-chromeos-test/
    2018-12-03 04:51:42  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871611-reset/
    2018-12-03 04:41:39  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871561-repair/
    2018-12-03 04:40:22  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2871547-verify/
    2018-12-03 03:15:47  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870981-repair/
    2018-12-03 03:06:25  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870940-provision/
    2018-12-03 02:56:53  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870885-repair/
    2018-12-03 02:55:38  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870870-verify/
    2018-12-03 02:15:41  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870532-repair/
    2018-12-03 02:06:34  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870465-provision/
    2018-12-03 01:56:31  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870362-repair/
    2018-12-03 01:55:15  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870326-verify/
    2018-12-03 01:29:23  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870134-repair/
    2018-12-03 01:20:39  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2870069-provision/
    2018-12-03 01:10:50  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869977-repair/
    2018-12-03 01:09:33  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869964-verify/
    2018-12-03 00:28:18  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869626-repair/
    2018-12-03 00:19:20  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869562-provision/
    2018-12-02 22:59:32  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869429-repair/
    2018-12-02 22:58:11  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869422-verify/
    2018-12-02 22:37:47  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869366-repair/
    2018-12-02 22:27:00  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869329-provision/
    2018-12-02 22:07:33  OK https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869227-repair/
    2018-12-02 22:06:19  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2869214-verify/
    2018-12-02 21:24:29  NO https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2868963-repair/
    2018-12-02 21:14:39  -- https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host21/2868916-provision/



The apparent repair successes are beacause the prior failing repair wipes stateful before failing (so next repair thinks AU succeeded and doesn't even try).

We should simply remove the DUT: 
- I've locked the DUT so it doesn't get used.
- Filed b/120445488 to remove the DUT physically.
- Once physically removed, deputy should remove references from our inventory.
Components: -Infra>Client>ChromeOS>Test Infra>Client>ChromeOS>Test>Platform
Status: Verified (was: ExternalDependency)
bad dut is gone.

Sign in to add a comment