New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 769383 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Sep 10
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: ----



Sign in to add a comment

Wrong culprit for flaky test

Project Member Reported by st...@chromium.org, Sep 27 2017

Issue description

https://findit-for-me.appspot.com/waterfall/flake?key=ag9zfmZpbmRpdC1mb3ItbWVyvQELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCKGAWNocm9taXVtLm1lbW9yeS9MaW51eCBDaHJvbWl1bSBPUyBBU2FuIExTYW4gVGVzdHMgKDEpLzIzODY0L2FzaF91bml0dGVzdHMvVTJOeVpXVnVUR0Y1YjNWMFQySnpaWEoyWlhKVVpYTjBMa1JwYzNCc1lYbE9iM1JwWm1sallYUnBiMjV6DAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw

In this case, the first data point (build-level Swarming task) has 1800 rerun to confirm it is a stable one, the second data point (commit-level try-job) has 100 rerun with 99% pass rate, and the third data point (commit-level try-job) has 100 rerun with 64% pass rate.

The regression range between the first two data points are irrelevant to the test: https://chromium.googlesource.com/chromium/src/+log/f1b84062d58fd80832fccfb828b2d5c25b6b7388..ee07711fafdfd5d785eed3b07797714ae55d8ca6?pretty=fuller
two commits are for layout tests and the other is for perf test framework.

The failure message is:
16918:16924:0927/094440.499581:4860965953:FATAL:resource_provider.cc(1354)] Check failed: sync_token.HasData() || gl->GetGraphicsResetStatusKHR() != GL_NO_ERROR. 


Two possibilities:
1. This test is hard to repro in the first two data points. However it is pretty easy to repro in all following data points.
2. Our algorithm needs some improvement.
 
Status: WontFix (was: Assigned)
Upper flake threshold has been set to 0.999999, and we now have smarter confidence score for determining whether to notify code reviews/comment on bugs about findings. Closing.

Sign in to add a comment