New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 817955 link

Starred by 1 user
Status: Fixed
Owner:
davidri...@chromium.org
Closed: Mar 2018
Cc:
█ gu...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Quick provisioning reboot can erroneously fail provision which has otherwise completed.

Project Member Reported by davidri...@chromium.org, Mar 1 2018 
At the end of the quick provision script it does a "(sleep 2; reboot) &" right before exiting.  The intention is that the SSH call should return successfully, and then the device should reboot.  It appears as if the script can finish, but the SSH call doesn't actually return successfully, and then when it's terminated by the reboot, the entire provisioned is failed.

From:
https://storage.cloud.google.com/chromeos-autotest-results/hosts/chromeos4-row6-rack9-host12/431248-provision/20182802083741/autoupdate_logs/CrOS_update_chromeos4-row6-rack9-host12_6975.log?_ga=2.155353147.-1609996729.1510708542

2018-02-28 08:42:55-08:00 INFO: Reboot (into cyan-paladin/R66-10446.0.0-rc2)
2018-02-28 08:42:55-08:00 INFO: Updated status: DUT: Reboot
KEYVAL: QUICK_PROVISION_start=1519835976
KEYVAL: QUICK_PROVISION_end=1519836175
KEYVAL: QUICK_PROVISION_elapsed=199

2018/02/28 08:43:34.879 DEBUG|    cros_build_lib:0648| (stderr):
Warning: Permanently added 'chromeos4-row6-rack9-host12,100.115.197.86' (ED25519) to the list of known hosts.
Warning: Permanently added 'chromeos4-row6-rack9-host12,100.115.197.86' (ED25519) to the list of known hosts.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  9663  100  9663    0     0  3145k      0 --:--:-- --:--:-- --:--:-- 4718k
Timeout, server chromeos4-row6-rack9-host12 not responding.
Project Member

Comment 1 by bugdroid1@chromium.org, Mar 2 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/dev-util/+/f5941190dcc2a367fc8fa0f86d2594c2197cf12d

commit f5941190dcc2a367fc8fa0f86d2594c2197cf12d
Author: David Riley <davidriley@chromium.org>
Date: Fri Mar 02 08:11:59 2018

quick-provision: Handle ssh errors more gracefully.

There is a race upon reboot of the quick provision pass where the reboot
can occur and terminate the ssh connection resulting in an ssh failure.
Capture that failure and use the COMPLETED keyval to determine if the
error should be ignored.

BUG= chromium:817955 
TEST=~autotest/contrib/loadtest.py DEVSERVER config-cyan.json -t 1 -s 1 --ping

Change-Id: I8f633b37581580483dce9970525857d1e828c229
Reviewed-on: https://chromium-review.googlesource.com/944564
Commit-Ready: David Riley <davidriley@chromium.org>
Tested-by: David Riley <davidriley@chromium.org>
Reviewed-by: Congbin Guo <guocb@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/f5941190dcc2a367fc8fa0f86d2594c2197cf12d/quick-provision/quick-provision
[modify] https://crrev.com/f5941190dcc2a367fc8fa0f86d2594c2197cf12d/cros_update.py

Comment 2 by ayatane@chromium.org, Mar 5 2018

Status: Assigned (was: Untriaged)

Comment 3 by davidri...@chromium.org, Mar 6 2018

Status: Fixed (was: Assigned)
►

Sign in to add a comment