New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 706444 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Make "Flaky Failures" on ChromiumDash page show the total number of flaky failures in all tests in a given directory or owned by a given team

Project Member Reported by ojan@chromium.org, Mar 29 2017

Issue description

https://chromiumdash.appspot.com/flakiness/input-dev shows 0 flakes. 

https://test-results.appspot.com/flakiness/team/input-dev shows a bunch of flakes.

These two should show consistent numbers.
 
Cc: ojan@chromium.org
Most of the tests are not so flaky (~3% probability of failure) and there are plenty of non-flaky tests thus averaging total to 0% flakiness for the whole team and for each of the directories. I didn't verify this in detail yet, but it's a plausible explanation.

Also it may be somewhat confusing, but Flaky Failures shown for directories in https://chromiumdash.appspot.com/flakiness/input-dev are not the same as Flaky Failures shown for individual tests in https://test-results.appspot.com/flakiness/team/input-dev. Please hover the column titles to see the description and let me know if you intuition about these columns was different... then perhaps we should use something simpler, e.g. just summing the number of flakes of tests that happen in a given directory or whole team (for the first line).

Comment 2 by ojan@chromium.org, Mar 30 2017

I think it's too confusing for the two dashboards to show different data. I don't think we we gain much from the more complicated flakiness metrics in chromiumdash.

A simple sum of number of flakes seems like it would work well, be easily understood and match the numbers on test-results. Is that right?
Summary: Make "Flaky Failures" on ChromiumDash page show the total number of flaky failures in all tests in a given directory or owned by a given team (was: flakiness surfaces data doesn't match test-results)
Sure, will change that. Thanks for the feedback.
Components: -Infra>Flakiness>Dashboard Infra>Flakiness
Labels: Flakiness-Surface Milestone-Dogfood
Cc: seanmccullough@chromium.org
Owner: ----
Status: Available (was: Assigned)
Removing ownership as I'm transitioning to another team and Sean is taking over the flakiness effort.

Comment 6 by ojan@chromium.org, May 8 2018

Cc: -ojan@chromium.org

Sign in to add a comment