New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 671874 link

Starred by 0 users

Issue metadata

Status: Duplicate
Merged: issue 670326
Owner: ----
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug



Sign in to add a comment

miscategorized "flaky test failures" for webkit_tests that we should deal with.

Project Member Reported by dpranke@chromium.org, Dec 7 2016

Issue description

 bug #637478  is a long-standing open bug that theoretically is tracking webkit_tests hanging when run on Windows.

Unfortunately, it looks like chromium-try-flakes is perhaps incorrectly assigning *all* webkit_tests flakes to it (or at least some of them).

For example, in https://bugs.chromium.org/p/chromium/issues/detail?id=637478#c132 , 29 new flakes are reported. However, 24 of them are actually failures on mac or linux that appear to have been due to r436518 landing without running try jobs (which is "as expected" since it was a re-land).

So, there are several issues here that we should probably split out into separate bugs:

1) these failures are being classified as "flakes" even though there were failures on the waterfall builders at the same time.

2) the failures are being assigned to the "wrong" bug

3) there's no actual way to edit the flakiness assignments in 

https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJAsSBUZsYWtlIhl3ZWJraXRfdGVzdHMgKHdpdGggcGF0Y2gpDA

to reassociate them with a different bug, or to annotate them that someone has actually triaged the flakes.

I think I've filed bugs related to (1) before. (2) might be a new issue that we could track here. (3) probably deserves a separate bug or bugs on its own.

I'm not sure who the best people to look at this are or what the right components are, but I'm guessing it's estaab and sergiyb, and Infra>Flakiness or Infra>Flakiness>Pipeline :).
 
Thanks for filing such a detailed bug report.

1) We are running steps with and without patch and only report them flaky if "with patch" fails while "without patch" passes. This should exclude any failures on the HEAD (waterfall). If you see examples proving otherwise, please post links to specific builds reported as flaky and corresponding waterfall failures.

2) I assume you suggest that we should group flakes to different bugs based on more than just the step name. This is similar to issue 670326.

3) Annotations were requested in issue 588522.

Unfortunately due to current focus on Flakiness Surface project I have no time to improve chromium-try-flakes and only keep it in maintenance mode. AFAIK there are also no immediate plans to increase headcount for flakiness work, so it may take a while before I'll be able to address these issues.
Cc: -andyb...@chromium.org
Mergedinto: 670326
Status: Duplicate (was: Untriaged)
Looks like issue described in this bug are already tracked elsewhere. I am going to mark it as a duplicate of 670326, but please see comment #1 for other issues as well.

Sign in to add a comment