[Findit] Flake Analyzer - Improve confidence score algorithm |
||
Issue descriptionStepiness alone is not smart enough to determine a culprit can be found in a range. In many cases, such as https://fa2a1f56aa-dot-waterfall-frontend-dot-findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVytAELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCJ-Y2hyb21pdW0ubWVtb3J5L0xpbnV4IENocm9taXVtIE9TIEFTYW4gTFNhbiBUZXN0cyAoMSkvMjEzOTQvYnJvd3Nlcl90ZXN0cy9VR0ZuWlV4dllXUk5aWFJ5YVdOelFuSnZkM05sY2xSbGMzUXVRbUZrV0doMGJXd3ZNUT09DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw, the existence of a culprit is obvious to the human eye but not so much by following stepiness alone. Some initial possible improvements: 1. Newly added tests (such as the case above) already detected to be flaky should always force try jobs with 100% confidence 2. Anywhere there's a 100% stable -> flaky (if suspected build) followed by only flaky points should have high confidence
,
Aug 7 2017
|
||
►
Sign in to add a comment |
||
Comment 1 by bugdroid1@chromium.org
, Aug 2 2017