[Findit] Flake Analyzer - Bail out of try jobs to avoid false positives |
||||
Issue descriptionRefactor logic to bail out of try jobs: 1. No suspected build 2. Build before suspected build is not 100% passing 2. If build before suspected build is 100% passing, suspected build itself must have > 60% confidence These should be put into a function and made extendable based on other criteria As we would like to re enable pushing results to code review, these criteria should prevent as many false positives as possible
,
Sep 25 2017
,
Sep 25 2017
,
Sep 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80 commit 38d97bc4e8eeaa2f1387bf868f0908ad1e410b80 Author: Jeffrey Li <lijeffrey@chromium.org> Date: Wed Sep 27 18:49:51 2017 [Findit] Flake Analyzer - Introduce function to determine whether to run try jobs Try jobs should or should not be run in the following scenarios: 1. No suspected build number - Do not run try jobs. 2. The user specified a range leading to a suspected build number - always run try jobs regardless of confidence, not truly 100% passing, etc. 3. The data point before the suspect is not a solid 100% or 0% passing - Do not run try jobs. 4. The previous data point is 100% passing, but the confidence score is < 60% for the suspected point. Do not run try jobs. This change makes it possible to extend or reduce this list of criteria as needed and in isolation so other parts of the code are unaffected if it is updated. Bug: 768044 Change-Id: Ia76e649ed16ba0d6dd401334cbf0d3f60f514bad Reviewed-on: https://chromium-review.googlesource.com/685796 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> Reviewed-by: Brandon Wylie <wylieb@chromium.org> [modify] https://crrev.com/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80/appengine/findit/waterfall/flake/test/initialize_flake_try_job_pipeline_test.py [modify] https://crrev.com/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80/appengine/findit/waterfall/flake/test/lookback_algorithm_test.py [modify] https://crrev.com/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80/appengine/findit/waterfall/flake/flake_constants.py [modify] https://crrev.com/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80/appengine/findit/waterfall/flake/initialize_flake_try_job_pipeline.py [modify] https://crrev.com/38d97bc4e8eeaa2f1387bf868f0908ad1e410b80/appengine/findit/waterfall/flake/lookback_algorithm.py
,
Sep 29 2017
|
||||
►
Sign in to add a comment |
||||
Comment 1 by lijeffrey@chromium.org
, Sep 22 2017