New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 598224 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Mar 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 595763



Sign in to add a comment

Several CQ/paladin builders offline

Project Member Reported by llozano@chromium.org, Mar 27 2016

Issue description

since friday night the commit queue is not allowing any changes to go through.

https://uberchromegw.corp.google.com/i/chromeos/builders/master-paladin/builds/10615

the failure seems to be very similar all the time:

Ex: CommitQueueCompletion guado_moblab-paladin did not start. auron-paladin did not start. daisy-paladin did not start. 


 
Blocking: 595763
Cc: akes...@chromium.org
Labels: -Build Infra-Labs Infra-ChromeOS
Summary: Several CQ/paladin builders offline (was: commit queue is not allowing any chnages to go through)

Comment 3 by sosa@chromium.org, Mar 27 2016

Many builders seem down on the CrOS waterfall. For example daisy-paladin has been not completing regularly and it looks like it's using it's backup builder:

https://uberchromegw.corp.google.com/i/chromeos/builders/daisy-paladin

The backup builder looks to be overwhelmed as it's building for many build types which it shouldn't really be. See  https://uberchromegw.corp.google.com/i/chromeos/buildslaves/build111-m2 . Not sure what's going on here.

Comment 4 by sosa@google.com, Mar 27 2016

Looking at the build111-m2 it looks to be a backup builder. So it seems like at least 5 of the CQ builders are offline (based on unique configs running on build111-m2).

I see:
build139-m2
build116-m2
build146-m2
build329-m2
build109-m2

all offline
Cc: h...@chromium.org

Comment 6 by pschm...@google.com, Mar 28 2016

build{139,116,146,109}-m2 are back up.    Somebody connected up build329-m2

build189-m2 will be up in a bit (Noticed it was down too.)

Comment 7 by d...@chromium.org, Mar 28 2016

Re: build329-m2, that was me, i just started the buildbot process.
Cc: xixuan@chromium.org gwendal@chromium.org d...@chromium.org kevcheng@chromium.org
 Issue 598323  has been merged into this issue.

Comment 9 by vhang@google.com, Mar 28 2016

Owner: pschmidt@chromium.org
Status: Assigned (was: Untriaged)
Status: Started (was: Assigned)
build189-m2 is now back up.

I think this bug is now fixed?

Comment 11 by d...@chromium.org, Mar 28 2016

Status: Fixed (was: Started)
I think so. Thanks!
Labels: VerifyIn-51
Components: Infra>Client>ChromeOS
Labels: -Infra-ChromeOS
Components: Infra>Labs
Labels: -Infra-Labs
Status: Verified (was: Fixed)
Bulk verified

Sign in to add a comment