New issue
Advanced search Search tips

Issue 748188 link

Starred by 0 users

Issue metadata

Status: Archived
Owner: ----
Closed: Dec 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

consistent failure criteria ignores which tests failed?

Project Member Reported by ojan@chromium.org, Jul 24 2017

Issue description

webkit_tests failed twice in a row, but the first time no tests failed. The second time a specific test failed.

This should be treated as a new failure instead of a consistent failure since the same failure didn't happen twice I think? At least as the UI is today it looks like the test failed two builds in a row when in fact it only failed one build. Although, it's a bit tricky because this test flaked in the previous builds (failed and passed on retry in the same run). Maybe that's confusing the alerting logic.

See screenshots.
 
consistentfailure.png
81.8 KB View Download
consistentfailure2.png
67.2 KB View Download
consistentfailure3.png
59.7 KB View Download
consistentfailure4.png
38.3 KB View Download

Comment 1 by ojan@chromium.org, Jul 24 2017

Labels: -Pri-2 Pri-1
I actually see this for a case with different stepnames as well (I think?). I'm marking this as P1. It makes it so I can't trust anything in sheriff-o-matic and need to click through to the buildbot pages to verify that it's actually a consistent failure.
Labels: Milestone-Workflow
Status: Available (was: Untriaged)
This sounds like a bug in the step failure grouping logic. 
Is this still happening?  And if so, is it really a P1?  Our P1 turn around time shouldn't be >4 months ...
Labels: -Pri-1 Pri-2
This is probably P2 instead of P1 though we do have some old P1 in Sheriff-o-Matic. The SoM Triage Rotation ( go/som-bug-triage ) tends to mostly focus on processing new bugs, and we don't tend to assign owners. 

Sean, what do you think about potentially amending our triage process a bit to help us have a better turnaround time on bugs? ie: maybe P1/P0 bugs should either get assigned an owner or deprioritized. 

Another thing I've thought is that it's a bit too easy for us to put milestones on bugs then forget they exist. I'm not sure how to best solve this, but it's something I think we can improve. I try to randomly look at go/som-issues every now and then but with almost 200 open SoM bugs going through the list can be overhwelming sometimes. 
Labels: Type-Bug
Project Member

Comment 6 by sheriffbot@chromium.org, Dec 17

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Archived (was: Untriaged)

Sign in to add a comment