New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 771743 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[Findit] Flake Analyzer - Waterfall may have already notified a code review of being a reliable failure

Project Member Reported by lijeffrey@chromium.org, Oct 4 2017

Issue description

https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVysgELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCJ8Y2hyb21pdW0ubGludXgvTGludXggVGVzdHMgKGRiZykoMSkoMzIpLzQ0NzMxL2Jyb3dzZXJfdGVzdHMvUVd4c1VISnZZMlZ6YzJWekwwMWxiV3h2WjBKeWIzZHpaWEpVWlhOMExrVnVaRlJ2Ulc1a1ZISmhZMmx1Wnk4dwwLEhNNYXN0ZXJGbGFrZUFuYWx5c2lzGAEM

In this analysis, the test was flaky, however waterfall thought it was a reliable failure and notified the code review first.

It is possible flake analyzer will send a subsequent notification that the test is flaky.

Waterfall got there first (mistaking a highly-flaky test for a reliable failure and posting a notification as such), but flake analyzer is technically correct (the test is flaky, not a reliable failure).

Newly-added tests are more susceptible to this double analysis, since the main waterfall analysis looks for culprits in builds that just failed, whereas flake analyzer can look back in time much further.

Ideas:
1. Merge the culprit page with the flake culprit page to be a generic "failure" page that links all available information that Findit was able to gather
2. Post a single more general notification (can be either flaky or reliable) that links to that page, with the page's contents updated based on completed analyses

OR

Post both notifications for now.

This is a placeholder bug to keep this topic open for discussion.

 

Comment 1 by st...@chromium.org, Oct 5 2017

Status: Assigned (was: Available)
chanli@, any thought on this?

Sign in to add a comment