The "recently resolved" list used by CrOS might useful for this particular case (alert would still be visible after a passing build), but it really looks like we need to deal with the flakiness at the root of it.

Yea: I think that, if a failure only happens ~33% of the time (pretty common among our tests), it's not reasonable to have that alert go away after 2 passing runs in a row, for example. This is actually a pretty similar problem to what we deal with when doing bisects for flakiness: after how many repetitions are you reasonably confident that a test is passing at a given CL? It might be worth reaching out to dtu@ (who's the TL of bisect) in order to get some ideas. I think that anything you do will have to be probabilistic, but to get rid of the alert on the first passing run probably isn't the right behavior.

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot