Thanks for reporting.

These flakes were detected by jmadill@ in  Issue 916544  and tracked down to https://crrev.com/c/1283010 . A revert was submitted and the test suites recovered.

In this case the GPU tests and bots were the primary canary which detected this legitimate bug in the Chrome browser. In my opinion it would have been a mistake to institute any more retries which would have made these failures less visible.

Findit should have caught this automatically.  Issue 916762  has been filed to figure out why it didn't. Please help chanli@ and stgao@ on that.

Duplicating into  Issue 916544 .

Thanks for the update, Ken. For now, let's keep the current behavior [no retries].

> In my opinion it would have been a mistake to institute any more retries which would have made these failures less visible.

If we reach a state where the failures are equally visible [e.g. FindIt catches these errors even when we retry], then I think we should reconsider instituting retries. In that case, we'll still be filing bugs on the errors, but they won't cause a significant amount of false-rejects for the rest of Chrome devs.

I didn't want to interject into the prior discussion but would like to offer my vote against flaky retries.

I often diagnoses and report upstream errors that are not in my code but affect me. I'd much rather have those errors be visible and immediately problematic than have any kind of mode that allows people to ignore them. It makes more convincing to revert problematic CLs when you can say "this breaks my CQ" than "this makes my CQ take an extra hour to detect flaky tests and sometimes makes it fail".

> I'd much rather have those errors be visible and immediately problematic than have any kind of mode that allows people to ignore them.

These should be immediately visible to the teams that care [e.g. your team, GPU team, etc.]

> that allows people to ignore them

There are hundreds of Chrome devs. Most of them would like to not be impacted when flakiness is introduced into a test suite that they know nothing about.

> It makes more convincing to revert problematic CLs when you can say "this breaks my CQ" than "this makes my CQ take an extra hour to detect flaky tests and sometimes makes it fail".

That's a policy change we can make. 


Please see https://docs.google.com/document/d/1O9nzVMA6rEe2-rhjsni_8wS9Lw4OnFpK2NE0wWB5VaY/edit#heading=h.ef0u5ezii5be for more details.

> That's a policy change we can make. 

It is, but I expect it won't be the popular option. Most devs don't seem to want flakes to block the CQ.

It would probably be more popular if we could show that the flakes are new and that people will actively triage and resolve them. Both things are probably true for the GPU test suites, but not true generally for test suites elsewhere.

(This is, of course, something that we're hoping to change, and eventually get to the point where we are dealing w/ flakes better everywhere).