New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 644877 link

Starred by 1 user
Status: Verified
Owner:
█ jongkyuj@chromium.org
Email to this user bounced
Closed: Sep 2016
Cc:
█ pho...@chromium.org
aashuto...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocked on:
issue 644287
issue 644561
Blocking:
issue 644866



Sign in to add a comment

DUTs went into Repair Failed state over the weekend

Project Member Reported by tienchang@chromium.org, Sep 7 2016 
Roughly 25 DUTs in wificells went into Repair Failed over the weekend due to powerwashes. It looks like most of these are due to 

Example host:
https://ubercautotest.corp.google.com/afe/#tab_id=view_host&object_id=5633

Example logs:
https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/hosts/android1758-cros-rack1-host2/58306870-repair

"FAIL	----	verify.python	timestamp=1473278536	localtime=Sep 07 13:02:16	Python is missing; may be caused by powerwash"

Not sure why this happened or if we can prevent this in the future, hence CC'ing infra team.

Paul, please manually reimage these DUTs via USB stick and verify these hosts are returned to Ready state.

Comment 1 by jongkyuj@google.com, Sep 7 2016

Status: Started (was: Assigned)
android1758-row4-rack1-host3 : Ultima USB stick is bad and needs to replace with new build.

android1758-row4-rack3-host1: OS is reinstalled from USB stick and provisioing.

Comment 2 by jongkyuj@google.com, Sep 7 2016 

android1758-row4-rack2-host1 : Reinstalled OS from USB stick and repairing.

Comment 3 by jongkyuj@google.com, Sep 7 2016 

android1758-row4-rack2-host1 : Ready

Comment 4 by jongkyuj@google.com, Sep 7 2016 

android1758-row4-rack2-host4 Ready
Reinstalled OS from USB stick

Comment 5 by jongkyuj@google.com, Sep 7 2016 

android1758-row4-rack2-host2 : OS reinstalled from USB stick and provisioning

Comment 6 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack5-host1: OS reinstalled from USB stick repairning

Comment 7 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack8-host1 : OS reinstalled from USB stick repairing

Comment 8 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack1-host2 : OS reinstalled from USB stick repairing

Comment 9 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack1-host4 : OS reinstalled from USB stick provisioning

Comment 10 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack5-host2 : OS reinstalled from USB stick repairing

Comment 11 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack5-host4 : OS reinstalled from USB stick repairing

Comment 12 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack8-host1 : OS reinstalled from USB stick repairing

Comment 13 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack8-host3 : OS reinstalled from USB stick verifying

Comment 14 by jongkyuj@google.com, Sep 8 2016 

android1758-cros-rack1-host2 : OS reinstalled from USB stick repairing

Comment 15 by jongkyuj@google.com, Sep 8 2016 

android1758-cros-rack14-host1 : OS reinstalled from USB stick repairing

Comment 16 by jongkyuj@google.com, Sep 8 2016 

android1758-cros-rack4-host3 : repairing

Comment 17 by jongkyuj@google.com, Sep 8 2016 

android1758-row4-rack7-host4 : Repairing

Comment 18 by jongkyuj@google.com, Sep 8 2016 

chromeos1-grover-host5 : TOT OS reinstalled from USB stick repairing

Comment 19 by tienchang@chromium.org, Sep 8 2016

Blockedon: 644561 644287

Comment 20 by ka...@chromium.org, Sep 8 2016

Blocking: 644866

Comment 21 by tienchang@chromium.org, Sep 15 2016

Status: Verified (was: Started)
This should be resolved now.
►

Sign in to add a comment