New issue
Advanced search Search tips

Issue 875055 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 837855



Sign in to add a comment

Change sheriff-o-matic to use Flaky-Test customized field when filing flaky test bugs

Project Member Reported by liaoyuke@chromium.org, Aug 16

Issue description

So that FindIt can better search Monorail for flakiness bugs filed by try-flakes and sheriff-o-matic and reuse them to avoid filing duplicate ones.
 
Summary: Change chromium-try-flakes and sheriff-o-matic to use Test-Flaky label when filing flaky test bugs (was: Fix chromium-try-flakes and sheriff-o-matic to use Test-Flaky label)
Cc: seanmccullough@chromium.org
+seanmccullough, do you have any objections at the sheriff-o-matic's side?
Not Test-Flaky label, but it is the Test-Flaky customized field as shown in https://screenshot.googleplex.com/6H9tTgU2huw.png
Summary: Change chromium-try-flakes and sheriff-o-matic to use Flaky-Test customized field when filing flaky test bugs (was: Change chromium-try-flakes and sheriff-o-matic to use Test-Flaky label when filing flaky test bugs)
Thanks! That's even better.
Another question is that:
from sheriff-o-matic's workflow, can we tell whether the developers are trying to file a bug for flaky tests or something else?
From SoM's workflow all we know is text from the alert description. We don't know if it's due to a flake or not.
Thanks for the reply! Then, it seems that this is not doable for sheriff-o-matic.
Is there any chance we could change SoM UI for flake bugs, e.g. add a "File-Flake-Bug" button after each test?
Duplicate bugs will cause unexpected burden for sheriffs and developers.
re #8 sure we can change it, but we'd have to decide how to prioritize that vs other feature requests (specifically TA/DA improvements). "File Flake Bug" sounds useful, and in lieu of TA/DA working for tests outside of webkit that's probably a workflow improvement for many sheriffs. 

Do we want sheriffs to be able to disable flaky tests directly from SoM, or is just filing flake-tagged bugs enough of an improvement over status quo?

Work on SoM side would be some extra UI (which could be quite noisy if there are lots of tests, or complex to implement if we want to reduce noise by making multi-select etc), and changes to Monorail API calls to support custom fields.
We may not need a new button, instead allow sheriffs to add Test-Flake when file a bug?

Though from our current logic of looking for existing bug for a flaky test, if the test is not specifically mentioned, I don't think we can find a matched SoM-filed bug.
Components: -Tools>Test>FindIt>Flakiness Infra>Sheriffing>SheriffOMatic
Labels: -Pri-1 Pri-3
Status: Available (was: Started)
Summary: Change sheriff-o-matic to use Flaky-Test customized field when filing flaky test bugs (was: Change chromium-try-flakes and sheriff-o-matic to use Flaky-Test customized field when filing flaky test bugs)
try-flakes is deprecated, so I'm unassigning myself and mark it as available.
Owner: ----

Sign in to add a comment