[Findit] Flake Analyzer - Smarter confidence score |
||||
Issue descriptionUsing pure steppiness for confidence score (for regressions) yields a lot of false negatives. 1. Boundaries that are clearly 100% passing -> clearly flaky should have higher confidence and not rely on steppiness 2. Series of definitely stable points -> series of definitely flaky points should have higher confidence
,
Mar 26 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/9994eca5ed8412c3e00e4ee5b6489e48c74369db commit 9994eca5ed8412c3e00e4ee5b6489e48c74369db Author: Brandon Wylie <wylieb@chromium.org> Date: Mon Mar 26 17:06:11 2018 [Findit] Flake Analyzer - Heuristics for culprit confidence Bug: 807947 Change-Id: I3a2eddeede7d62a9e023180740bf9c2a76c60380 Reviewed-on: https://chromium-review.googlesource.com/972361 Reviewed-by: Jeffrey Li <lijeffrey@chromium.org> Commit-Queue: Brandon Wylie <wylieb@chromium.org> [modify] https://crrev.com/9994eca5ed8412c3e00e4ee5b6489e48c74369db/appengine/findit/services/flake_failure/confidence_score_util.py [modify] https://crrev.com/9994eca5ed8412c3e00e4ee5b6489e48c74369db/appengine/findit/services/flake_failure/test/confidence_score_util_test.py [modify] https://crrev.com/9994eca5ed8412c3e00e4ee5b6489e48c74369db/appengine/findit/services/flake_failure/pass_rate_util.py [modify] https://crrev.com/9994eca5ed8412c3e00e4ee5b6489e48c74369db/appengine/findit/services/flake_failure/test/pass_rate_util_test.py
,
Mar 27 2018
,
Mar 27 2018
,
Mar 30 2018
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 13 2018
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 26 2018
|
||||
►
Sign in to add a comment |
||||
Comment 1 by wylieb@chromium.org
, Mar 8 2018