Dirk, do you have clue what could have broken this ? It has been a week since we had any new builds for fuzzing.

Offhand, I don't know. Maybe LKGR hasn't advanced?

I'll look into it more tomorrow.

aneesh's recipe changes weren't landed, so I don't think anything interesting has changed, builder-wise.

The CL to change LKGR to recipes hasn't landed, so I don't know what could be causing this.

Dirk, did you fix anything. Looks like builders started picking up builds this morning. Thanks a lot for whatever you did :)

Nope, didn't touch anything. Strange ...

Really weird, i can close as Fixed. Btw, what was the link you gave some time back to see which is the next green build it will pick from. Maybe we just need to monitor that or hopefully after you change these builders to do their own tests, that will be awesome too.

I think maybe LKGR stopped updating, and then started again. I don't know why.

Looking at https://build.chromium.org/p/chromium.infra.cron/builders/lkgr-tag-pusher?numbuilds=200

There's a suspicious lack of "refs/heads/lkgr" comments prior to build 48890.

cc'ing agable in case he has any ideas. However, given that we want to move off of LKGR as soon as we can, I wouldn't want to spend much time on this unless it happens again.

Sounds good, thanks Dirk!

Looks like we haven't gotten any new builds since April 15: https://build.chromium.org/p/chromium.lkgr/builders/ASAN%20Release

Hm. Assigned to me, huh? :)

Looking at the logs on https://build.chromium.org/p/chromium.infra.cron/builders/lkgr-tag-pusher?numbuilds=200 again, it looks like LKGR didn't advance over the weekend.

I'm not actually even sure how we're calculating LKGR these days, so I'm not sure if this is right or not; looking at the current sheriff-o-matic data, I don't see any failures that I would've guessed would block it, but maybe I'm wrong.

At this point, I'm kinda inclined to just move the builders to tip-of-tree and see what happens, since the tree is usually pretty green these days.

The Android Tests (dbg) bot was broken all weekend:
https://build.chromium.org/p/chromium/lkgr-status/
https://build.chromium.org/p/chromium.linux/builders/Android%20Tests%20%28dbg%29

So, everything WAI here :(

agable@ reminded me that you can see what lkgr is looking at here:

https://build.chromium.org/p/chromium/lkgr-status/

where it looks like "Android Tests (dbg)" was in fact broken alll weekend and the last actual lkgr was on Friday.

downgrading to Pri-2 but leaving this open and assigned to me to help me remember to get these things off of lkgr altogether.

Thanks!

Looks like all of the builders except for SyzyASan are red now. Would you mind taking another look?

Looks like none of the bots are finding goma, which I believe was moved yesterday but they probably didn't update the non-recipe configs for it. Or, it's possible I got something messed up when I converted the bots to MB, but I didn't see any signs of that at first look.

At any rate, it's a different issue. I'm closing this bug, since LKGR has been rolling forward okay, and filed a new bug ( bug 606082 ) for the new issue.