[Findit] Flake Analyzer - Dives may introduce false positives for culprits |
||||
Issue descriptionhttps://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVy9AELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCK9AWNocm9taXVtLndpbi9XaW4xMCBUZXN0cyB4NjQvMTAxMzkvYnJvd3Nlcl9zaWRlX25hdmlnYXRpb25fY29udGVudF91bml0dGVzdHMgb24gV2luZG93cy0xMC0xMDU4Ni9VbVZ1WkdWeVpYSkJkV1JwYjA5MWRIQjFkRk4wY21WaGJVWmhZM1J2Y25sSmJuUmxaM0poZEdsdmJsUmxjM1F1VTNSeVpXRnRTVzUwWldkeVlYUnBiMjVVWlhOMAwLEhNNYXN0ZXJGbGFrZUFuYWx5c2lzGAEM In this case, the test seems as though it was always flaky. However since a dive occurred a suspected build was identified with high confidence, leading to try jobs. However the lower bound build point's pass rate is only 91%, meaning the test was already flaky to begin with, leading try jobs to identify a false positive as the culprit. Possible solutions: 1. In addition to running try jobs only with high confidence, ensure the lower-bound of the regression range is within the stable threshold (0.02 or 0.98), bailing out otherwise 2. Disable dive analysis, since the test was already flaky to begin with it's unlikely the exact culprit is within the dive region
,
Apr 25 2017
,
Apr 26 2018
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Apr 26 2018
The new analysis pipelines and lookback algorithm no longer consider dives as potential suspects (and no longer support dive checking due to false positives). |
||||
►
Sign in to add a comment |
||||
Comment 1 by st...@chromium.org
, Apr 25 2017