Now Findit supports only finding culprits for compile failures and reliable test failures on Chromium waterfall.
This culprit-finding infra could be tweaked to support finding culprit for flaky tests.
The high-level solution is described in https://docs.google.com/a/google.com/document/d/14N2FRl_qH6HqvbvdmKo66VLhTId-SqJmhpspV28DnT0/edit?usp=sharing
Main tasks will include:
1. Within Findit app, trigger Swarming rerun to narrow down the regression range of a flaky test into a single Chromium build cycle.
Isolate Server has 2-month old data, thus we could rerun Swarming without recompile for over 10k+ revisions back.
2. For recipe, make Gtest run a single test multiple times in a try job, and collect results of each iteration for the computation of failure rate.
3. Use step-change detection api from perf dashboard to find the jump of failure rate for a test.
There will be other smaller tasks to support these main ones.
Comment 1 by st...@chromium.org
, Apr 22 2016