New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 856177 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Task



Sign in to add a comment

quawks BVT pool is dead

Reported by jrbarnette@chromium.org, Jun 25 2018

Issue description

$ dut-status -b quawks -p bvt
hostname                       S   last checked         URL
chromeos4-row10-rack8-host19   NO  2018-06-25 05:36:23  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack8-host19/831296-repair/
chromeos4-row10-rack10-host13  NO  2018-06-25 06:32:16  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack10-host13/831494-repair/
chromeos4-row10-rack10-host21  NO  2018-06-25 06:32:16  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack10-host21/831496-repair/
chromeos4-row4-rack8-host1     NO  2018-06-25 06:32:18  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row4-rack8-host1/831497-repair/
chromeos4-row4-rack8-host2     NO  2018-06-25 06:32:16  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row4-rack8-host2/831495-repair/

The most likely cause of this outage is the crashes reported
in bug 855139.

We need to balance the quawks BVT pool so that testing can resume.  When
we do that, we should also increase the pool size to the standard minimum
of 6 DUTs.

After balancing, we need to monitor the quawks BVT pool to see whether
devices continue to fail repair.  If there's another outage, we should
escalate bug 855139.

 
Cc: reinauer@chromium.org bhthompson@chromium.org namyoon@chromium.org gu...@chromium.org yllin@chromium.org
Labels: -Type-Bug -Pri-3 Pri-1 Type-Task
Owner: xixuan@chromium.org
Status: Assigned (was: Untriaged)
+sheriffs, because there's canary redness
+bhthompson because this will be breaking release builders

Lab deputy owns the actual work here.

Comment 2 by xixuan@chromium.org, Jun 25 2018

Owner: reinauer@chromium.org
Pool bvt for quawks are is fixed.

xixuan@xixuan0:~/chromiumos/src/third_party/autotest/files$ dut-status -b quawks -p bvt
hostname                       S   last checked         URL
chromeos4-row10-rack9-host1    OK  2018-06-25 09:25:53  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack9-host1/832153-reset/
chromeos4-row10-rack9-host5    OK  2018-06-25 09:25:54  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack9-host5/832154-reset/
chromeos4-row10-rack10-host1   OK  2018-06-25 09:25:57  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack10-host1/832156-reset/
chromeos4-row10-rack10-host3   OK  2018-06-25 09:25:56  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack10-host3/832155-reset/
chromeos4-row10-rack10-host5   OK  2018-06-25 09:25:57  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack10-host5/832157-reset/
chromeos4-row10-rack9-host7    OK  2018-06-25 08:54:57  http://cautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/hosts/chromeos4-row10-rack9-host7/832040-reset/

Re-assign to sheriff. If quawks-release fail again due to DUT shortage, please comment.

Sign in to add a comment