Flakiness detection should not run on experimental bot steps |
|||
Issue descriptionContext: https://chromium-review.googlesource.com/c/chromium/src/+/1179665#message-e3213afa35c0101b5ee7cdec7bf056d0f339beee In the above CL, tests were disabled because a flake was detected in the following bugs: https://bugs.chromium.org/p/chromium/issues/detail?id=875092 https://bugs.chromium.org/p/chromium/issues/detail?id=875129 The problem is the flakes occurred in the step: chrome_public_test_apk on Android device Nexus 5X (with patch, experimental) Since the step is experimental, it does not block the CQ and thus is unclear sheriffs should be operating on it at all. In particular for these tests, it was a underlying failure from another CL: https://chromium-review.googlesource.com/c/chromium/src/+/1174709 that likely triggered them. But again, in a way that can't be tested reliably. While I want to unrevert the test disabling patch, I can't do that if the flake detector will get it disabled again.
,
Aug 20
,
Aug 29
This is already taken care of in the new Flake Detection because in the sql query: https://cs.chromium.org/chromium/infra/appengine/findit/services/flake_detection/flaky_tests.cq_false_rejection.sql?l=192, we select with_patch steps by '%(with patch)%', and an experimental step will never match because it always looks like 'test target (with patch, experimental)'. Will wait for the new flake detection to be online and verify the behavior before marking this as fixed.
,
Jan 14
|
|||
►
Sign in to add a comment |
|||
Comment 1 by st...@chromium.org
, Aug 20