Change sheriff-o-matic to use Flaky-Test customized field when filing flaky test bugs |
|||||
Issue descriptionSo that FindIt can better search Monorail for flakiness bugs filed by try-flakes and sheriff-o-matic and reuse them to avoid filing duplicate ones.
,
Aug 16
+seanmccullough, do you have any objections at the sheriff-o-matic's side?
,
Aug 16
Not Test-Flaky label, but it is the Test-Flaky customized field as shown in https://screenshot.googleplex.com/6H9tTgU2huw.png
,
Aug 16
Thanks! That's even better.
,
Aug 16
Another question is that: from sheriff-o-matic's workflow, can we tell whether the developers are trying to file a bug for flaky tests or something else?
,
Aug 16
From SoM's workflow all we know is text from the alert description. We don't know if it's due to a flake or not.
,
Aug 16
Thanks for the reply! Then, it seems that this is not doable for sheriff-o-matic.
,
Aug 16
Is there any chance we could change SoM UI for flake bugs, e.g. add a "File-Flake-Bug" button after each test? Duplicate bugs will cause unexpected burden for sheriffs and developers.
,
Aug 16
re #8 sure we can change it, but we'd have to decide how to prioritize that vs other feature requests (specifically TA/DA improvements). "File Flake Bug" sounds useful, and in lieu of TA/DA working for tests outside of webkit that's probably a workflow improvement for many sheriffs. Do we want sheriffs to be able to disable flaky tests directly from SoM, or is just filing flake-tagged bugs enough of an improvement over status quo? Work on SoM side would be some extra UI (which could be quite noisy if there are lots of tests, or complex to implement if we want to reduce noise by making multi-select etc), and changes to Monorail API calls to support custom fields.
,
Dec 27
We may not need a new button, instead allow sheriffs to add Test-Flake when file a bug? Though from our current logic of looking for existing bug for a flaky test, if the test is not specifically mentioned, I don't think we can find a matched SoM-filed bug.
,
Jan 8
try-flakes is deprecated, so I'm unassigning myself and mark it as available.
,
Jan 8
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by liaoyuke@chromium.org
, Aug 16