New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 601857 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner: ----
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Feature



Sign in to add a comment

Support finding culprit for flaky tests in Findit

Project Member Reported by st...@chromium.org, Apr 8 2016

Issue description

Now Findit supports only finding culprits for compile failures and reliable test failures on Chromium waterfall.

This culprit-finding infra could be tweaked to support finding culprit for flaky tests.
The high-level solution is described in https://docs.google.com/a/google.com/document/d/14N2FRl_qH6HqvbvdmKo66VLhTId-SqJmhpspV28DnT0/edit?usp=sharing

Main tasks will include:
1. Within Findit app, trigger Swarming rerun to narrow down the regression range of a flaky test into a single Chromium build cycle.
   Isolate Server has 2-month old data, thus we could rerun Swarming without recompile for over 10k+ revisions back.
2. For recipe, make Gtest run a single test multiple times in a try job, and collect results of each iteration for the computation of failure rate.
3. Use step-change detection api from perf dashboard to find the jump of failure rate for a test.

There will be other smaller tasks to support these main ones.
 

Comment 1 by st...@chromium.org, Apr 22 2016

Components: Tools>Test>FindIt
Project Member

Comment 2 by sheriffbot@chromium.org, Apr 24 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been available for more than 365 days, and should be re-evaluated. Please re-triage this issue.
The Hotlist-Recharge-Cold label is applied for tracking purposes, and should not be removed after re-triaging the issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 3 by st...@chromium.org, Apr 24 2017

Labels: -Hotlist-Recharge-Cold -Findit
Status: Fixed (was: Untriaged)
This was done as the effort in Flake Analyzer. https://findit-for-me.appspot.com/waterfall/list-flakes

Sign in to add a comment