New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 819830 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 0
Type: Bug

Blocking:
issue 826087



Sign in to add a comment

[Findit] Flake Analyzer - Move on in DetermineApproximatePassRate if test already flaky

Project Member Reported by lijeffrey@chromium.org, Mar 7 2018

Issue description

If a test is already flaky, there is no need to determine convergence. This can greatly improve performance times by allowing analyses of tests that are obviously flaky to move on sooner
 
This is only true given that if all the remaining iterations would fail, the test would still be flaky.
Cc: wylieb@chromium.org
 Issue 819698  has been merged into this issue.
Project Member

Comment 3 by bugdroid1@chromium.org, Mar 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/b62154d9197ba69084037a3e747d23acfb8d75dd

commit b62154d9197ba69084037a3e747d23acfb8d75dd
Author: Jeffrey Li <lijeffrey@chromium.org>
Date: Tue Mar 13 04:51:13 2018

[Findit] Flake Analyzer - Speed up analyses by moving on if tests are flaky

Tests that are obviously flaky don't need a minimum number of iterations
or convergence to tell that they're flaky, so the analysis can continue
much sooner. Pass rates for data points that are stable however will retain
the need for further testing at the same commit position before moving on.

Bug:  819830 

Change-Id: I8e365b1dd8b4ea8a4a861f7913d3ad659d8b248e
Reviewed-on: https://chromium-review.googlesource.com/954062
Reviewed-by: Brandon Wylie <wylieb@chromium.org>
Commit-Queue: Jeffrey Li <lijeffrey@chromium.org>

[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/test/lookback_algorithm_test.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/pass_rate_util.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/lookback_algorithm.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/event_reporting.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/services/flake_failure/test/pass_rate_util_test.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/pipelines/flake_failure/determine_approximate_pass_rate_pipeline.py
[modify] https://crrev.com/b62154d9197ba69084037a3e747d23acfb8d75dd/appengine/findit/pipelines/flake_failure/test/determine_approximate_pass_rate_pipeline_test.py

Status: Fixed (was: Assigned)

Comment 5 by wylieb@chromium.org, Mar 27 2018

Labels: -Pri-3 Pri-0

Comment 6 by wylieb@chromium.org, Mar 27 2018

Blocking: 826087

Sign in to add a comment