[Findit] Flake Analyzer - Waterfall may have already notified a code review of being a reliable failure |
|
Issue descriptionhttps://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVysgELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCJ8Y2hyb21pdW0ubGludXgvTGludXggVGVzdHMgKGRiZykoMSkoMzIpLzQ0NzMxL2Jyb3dzZXJfdGVzdHMvUVd4c1VISnZZMlZ6YzJWekwwMWxiV3h2WjBKeWIzZHpaWEpVWlhOMExrVnVaRlJ2Ulc1a1ZISmhZMmx1Wnk4dwwLEhNNYXN0ZXJGbGFrZUFuYWx5c2lzGAEM In this analysis, the test was flaky, however waterfall thought it was a reliable failure and notified the code review first. It is possible flake analyzer will send a subsequent notification that the test is flaky. Waterfall got there first (mistaking a highly-flaky test for a reliable failure and posting a notification as such), but flake analyzer is technically correct (the test is flaky, not a reliable failure). Newly-added tests are more susceptible to this double analysis, since the main waterfall analysis looks for culprits in builds that just failed, whereas flake analyzer can look back in time much further. Ideas: 1. Merge the culprit page with the flake culprit page to be a generic "failure" page that links all available information that Findit was able to gather 2. Post a single more general notification (can be either flaky or reliable) that links to that page, with the page's contents updated based on completed analyses OR Post both notifications for now. This is a placeholder bug to keep this topic open for discussion. |
|
►
Sign in to add a comment |
|
Comment 1 by st...@chromium.org
, Oct 5 2017