If a test is already flaky, there is no need to determine convergence. This can greatly improve performance times by allowing analyses of tests that are obviously flaky to move on sooner
This is only true given that if all the remaining iterations would fail, the test would still be flaky.
Issue 819698 has been merged into this issue.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b62154d9197ba69084037a3e747d23acfb8d75dd commit b62154d9197ba69084037a3e747d23acfb8d75dd Author: Jeffrey Li <lijeffrey@chromium.org> Date: Tue Mar 13 04:51:13 2018 [Findit] Flake Analyzer - Speed up analyses by moving on if tests are flaky Tests that are obviously flaky don't need a minimum number of iterations or convergence to tell that they're flaky, so the analysis can continue much sooner. Pass rates for data points that are stable however will retain the need for further testing at the same commit position before moving on. Bug: 819830 Change-Id: I8e365b1dd8b4ea8a4a861f7913d3ad659d8b248e Reviewed-on: https://chromium-review.googlesource.com/954062 Reviewed-by: Brandon Wylie <wylieb@chromium.org> Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/test/lookback_algorithm_test.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/pass_rate_util.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/lookback_algorithm.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/event_reporting.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/test/pass_rate_util_test.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/pipelines/flake_failure/determine_approximate_pass_rate_pipeline.py [modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/pipelines/flake_failure/test/determine_approximate_pass_rate_pipeline_test.py
Comment 1 by wylieb@chromium.org
, Mar 8 2018