New issue
Advanced search Search tips

Issue 651133 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

bugdroid is broken

Project Member Reported by martiniss@chromium.org, Sep 28 2016

Issue description

https://uberchromegw.corp.google.com/i/internal.infra.cron/builders/bugdroid/builds/19837/steps/bugdroid/logs/stdio

:( Not sure what happened, but it needs to be fixed.

cc-ing current trooper FYI
 

Comment 1 by agable@google.com, Sep 28 2016

Status: Started (was: Assigned)
Ah crap, I added a new repo for it to poll, but the machine it is on doesn't have creds for that googlesource host.

Short term: reverting swiftshader config so bugdroid will come back to life

Long term: deploy swiftshader creds to bugdroid host (and everywhere else?), make bugdroid not die if it can't poll a single repo, reland swiftshader config.

Comment 3 by agable@google.com, Sep 28 2016

Also apparently, bugdroid doesn't do the "schedule every minute, but run for 10 minutes and coalesce build requests" thing that gnumbd does. This means that it's currently running for ~3 minutes, dying, and then sitting idle for 7 minutes. That should be fixed as well.

Comment 4 by agable@google.com, Sep 28 2016

Revert and restart worked, bugdroid is now updating bugs. Now working on long-term fixes.

Comment 5 by agable@google.com, Sep 28 2016

Labels: -Pri-1 -OS-Chrome OS-All Pri-2

Comment 6 by agable@google.com, Sep 28 2016

scheduler fix: https://chromereviews.googleplex.com/518677013
bugdroid fix: https://chromium-review.googlesource.com/390319

deploying credentials is probably not strictly necessary with that fix, but should still be done because bugdroid is a bot and therefore shouldn't be using anonymous quota.

swiftshader config reland: https://chromereviews.googleplex.com/515197013
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chrome/tools/build/+/c69876f9ebbfb20717f12b2e67dd7420e4daaf8f

commit c69876f9ebbfb20717f12b2e67dd7420e4daaf8f
Author: agable <agable@google.com>
Date: Wed Sep 28 18:16:14 2016

Comment 8 by agable@google.com, Sep 28 2016

And bugdroid creds fix: https://chromereviews.googleplex.com/516967014
Project Member

Comment 9 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/b5f3b14a7804c04a8376cfcb194d7d8b0ede8420

commit b5f3b14a7804c04a8376cfcb194d7d8b0ede8420
Author: Aaron Gable <agable@chromium.org>
Date: Wed Sep 28 18:12:46 2016

Prevent bugdroid from dying when missing credentials

Bugdroid *should* have a .netrc entry for all hosts it is polling
(for policy and quota reasons, even if those hosts allow anonymous access),
but it also shouldn't die in a fire if it is missing those creds.

BUG= 651133 

Change-Id: Ifd638426423245cee63c44bec3f5294c34c22e12
Reviewed-on: https://chromium-review.googlesource.com/390319
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>

[modify] https://crrev.com/b5f3b14a7804c04a8376cfcb194d7d8b0ede8420/infra/services/bugdroid/gob_helper.py

Project Member

Comment 10 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infra/puppet/+/4cc928d4d64bd6c874421f4620900c845b3806a3

commit 4cc928d4d64bd6c874421f4620900c845b3806a3
Author: Aaron Gable <agable@chromium.org>
Date: Wed Sep 28 20:14:51 2016

Comment 11 by capn@chromium.org, Sep 28 2016

https://chromereviews.googleplex.com/515197013 LGTM, but for some reason I can't comment on the review... Feel free to TBR as far as I'm concerned.

Comment 12 by agable@google.com, Sep 28 2016

I'm waiting for puppet to sync the new credentials to the bugdroid machine. Then I'll reland the swiftshader config.
Project Member

Comment 13 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/0f7b0c0346a0bb2989f38a394a22d05d9f983e18

commit 0f7b0c0346a0bb2989f38a394a22d05d9f983e18
Author: Aaron Gable <agable@chromium.org>
Date: Wed Sep 28 20:42:06 2016

Project Member

Comment 14 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/0f7b0c0346a0bb2989f38a394a22d05d9f983e18

commit 0f7b0c0346a0bb2989f38a394a22d05d9f983e18
Author: Aaron Gable <agable@chromium.org>
Date: Wed Sep 28 20:42:06 2016

Project Member

Comment 15 by bugdroid1@chromium.org, Sep 28 2016

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config.git/+/0f7b0c0346a0bb2989f38a394a22d05d9f983e18

commit 0f7b0c0346a0bb2989f38a394a22d05d9f983e18
Author: Aaron Gable <agable@chromium.org>
Date: Wed Sep 28 20:42:06 2016

Comment 16 by capn@chromium.org, Sep 28 2016

It's working great (https://bugs.chromium.org/p/swiftshader/issues/detail?id=10)! Thanks Aaron.

Comment 17 by agable@google.com, Sep 29 2016

Status: Fixed (was: Started)

Sign in to add a comment