[Findit] Flake Analyzer - 99.99% passing upper flake threshold may be ignored |
||
Issue descriptionhttps://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVyvQELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKGAWNocm9taXVtLmNocm9taXVtb3MvbGludXgtY2hyb21lb3MtZGJnLzcwNDcvYnJvd3Nlcl90ZXN0cy9URzluYVc1UWNtOXRjSFJDY205M2MyVnlWR1Z6ZEM1RFlXNWpaV3hNYjJkcGJrbHVkR1Z5YzNScGRHbGhiRTl1VW1Wa2FYSmxZM1E9DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw In this analysis it seems 98% is still being used as the upper flake threshold despite config specifying 99.99%+
,
Aug 3
The root cause is the config values are being read "incorrectly," with upper/lower_flake_threshold being under "swarming_rerun" within flake settings. The code that reads these values reads them at the root level (as it should), causing the values not to be found and thus using fallback values defined in flake_constants.py which still use the old 0.02, 0.98 figure. The full fix will be to clean up the config definition so the code can read it from the correct place.
,
Aug 6
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/e3deba1486febffd947515d92816a2970b66df37 commit e3deba1486febffd947515d92816a2970b66df37 Author: Jeffrey Li <lijeffrey@chromium.org> Date: Mon Aug 06 21:33:49 2018 [Findit] Flake Analyzer - Fixing check for stability to use proper thresholds This is part 1 of a 2-part fix. The root cause is the config values are being read "incorrectly," with upper/lower_flake_threshold being under "swarming_rerun" within flake settings. The code that reads these values reads them at the root level (as it should), causing the values not to be found and thus using fallback values defined in flake_constants.py which still use the old 0.02, 0.98 figure. In Part 1 here, the fix is to use pass_rate_util.IsStableDefaultThresholds which will always read from config and to update the fallback values to 1e-7 and 0.9999999. Part 2 will be to clean up config so it is in the expected format, captured in issue 799832 . Bug: 870376 Change-Id: Ie7ddd85d558b22c94b2991e8808bc7936115d28c Reviewed-on: https://chromium-review.googlesource.com/1162665 Commit-Queue: Jeffrey Li <lijeffrey@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> Reviewed-by: Yuke Liao <liaoyuke@chromium.org> [modify] https://crrev.com/e3deba1486febffd947515d92816a2970b66df37/appengine/findit/services/flake_failure/test/pass_rate_util_test.py [modify] https://crrev.com/e3deba1486febffd947515d92816a2970b66df37/appengine/findit/services/flake_failure/pass_rate_util.py [modify] https://crrev.com/e3deba1486febffd947515d92816a2970b66df37/appengine/findit/services/flake_failure/flake_constants.py
,
Aug 7
|
||
►
Sign in to add a comment |
||
Comment 1 by st...@chromium.org
, Aug 2