Hit me this morning as well.

https://bugs.chromium.org/p/chromium/issues/detail?id=542435

cdavid: Could you take a look at this soon?

Note that  issue 627121  (at least partially) addresses the flaky/disabled test concerns.

It doesn't help with eg things like bugs that are for "turn on new warning" and then have a cl to disable the new warning for now.

This case should be covered by 627121.

@Nico - Are there any patterns (e.g. labels/ components), which would make those easy to skip?

We have landed a cl to ignore the following bugs.
1. Components (Test, Tests)
2. Any blocked or blockedon issues.
3. Any issues with label autofiled,tracking_work or performance
4. Flaky tests from http://chromium-try-flakes.appspot.com

FWIW, I get an average of a dozen of these a day, and so far I think it's caught 2 bugs total, over the lifetime of this rule (the last couple weeks), where the CL in question really fixed the bug.  This is because I file or am CCed on a LOT of bugs, but I also pretty aggressively clean up those bugs, so it's rare for one to be outstandingly should-be-closed.

I cannot think of a rule that would help exclude these.  They don't generally fit into the categories in comment 8.

The cost is definitely much, much higher than the benefit here for me personally.

Bugs tracked by chromium-try-flakes may not be obviously detectable as my app follows links for bugs marked as Duplicate of another bug. Furthermore, users may manually override bug ID number in the app and bug may have no information that would identify it as a chromium-try-flakes bug.

Re #8, how exactly do you detect bugs for flaky tests from chromium-try-flakes?

Please teach it to ignore CLs which only touch files used to disable tests such as:

- //tools/valgrind/gtest_exclude/*
- //tools/valgrind/drmemory/suppressions*
- //third_party/WebKit/LayoutTests/*Expectations

This rule is triggering on bugs tracking disabled tests and is decidedly unhelpful in that case.

Re #10 we check comments for http://chromium-try-flakes.appspot.com and ignore it.

Would it make sense to change the bug status to Untriaged when adding this label? It would draw attention to them during each team's triage process before they are closed without review (in case the owner is no longer active or busy).

I can't think of any patterns in particular. Many bugs I'm cc'd on have the component "Build", so ignoring that component would fix most of the problem for me, but that's just because many of my bugs are in that category, not because that category has more flaky tests. (I guess most "clean up warning X" bugs are in that category.)

I think it's best to just unconditionally disable this, but if that's not an option then disabling it for Component Build would be a start.

This rule is now suspended. We will re-enable once all concerns are resolved.

 Issue 626030  has been merged into this issue.