"analyze" is flaky |
||
Issue description"analyze" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is not infrastructure-related (e.g. flaky test), please add Sheriff-Chromium label and change issue status to Untriaged. When done, please remove the issue from Trooper Bug Queue by removing the Infra-Troopers label. We have detected 22 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyEgsSBUZsYWtlIgdhbmFseXplDA. The chromium-try-flakes app is able to file bugs for individual tests when the test launcher is uploading results to the Test Results Server. If recent flakes above are caused by failing tests and you would like to have them filed as invidual bugs, please read more at https://goo.gl/QJKXV4. This flaky test/step was previously tracked in issue 708893 .
,
May 26 2017
that one patchset, fwiw, is https://codereview.chromium.org/2898303002/#ps20001
,
May 27 2017
This case is similar to patch failure flakiness. We explicitly blacklist it because patches often fails to apply on a given patchset and applies just later on due to changes in the tree. Is it common that analyze step flakiness are caused by changes in the repository? I wonder if we can somehow fix that - it is sad to see that we've had 4 false rejections on this CL. If it's hard or impossible to fix, I can add "analyze" to the blacklist. OTH, I am not sure this warrants blacklisting all flakes that happen on a single patchset. Although flakiness may be introduced by the code change in the patchset, there is high likelihood that that code has already committed by CQ because the builder that ran the flaky test must have passed for the flake to be detected.
,
Jun 8 2017
In one sense, I think that it depends on what you think a false negative is. If you think a false negative is a failed tryjob *for any reason at all* when a subsequent tryjob on the same patchset passes, then this isn't a false negative (the try job failed, because the tree was broken). On the other hand, if you don't want to count patches that fail because the tree is broken as a false negative, or you wanted to separate that out from a real flaky failure, we should still do something. However, in neither situation is this a "flake" on analyze's part. I can't remember the last time I saw a real analyze flake, and there have only been a couple bugs at all in the past year. Ideally we'd filter out cases where a step is failing on the main waterfall at the same time it's failing on the trybots, and keep retrying the patch, but this is complicated by analyze not actually being used on the waterfall. So, yeah, maybe we should blacklist it.
,
Jun 9 2017
,
Jun 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/7291c451c5a08fdb79ab38f2bcd175b97e871bfc commit 7291c451c5a08fdb79ab38f2bcd175b97e871bfc Author: Sergiy Byelozyorov <sergiyb@chromium.org> Date: Fri Jun 09 11:24:15 2017 Add analyze to the list of ignored flakes R=tandrii@chromium.org Bug: 725802 Change-Id: Ie17ef440f7d05685e9eeff56dc6b9c35c6df64a6 Reviewed-on: https://chromium-review.googlesource.com/528140 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org> [modify] https://crrev.com/7291c451c5a08fdb79ab38f2bcd175b97e871bfc/appengine/chromium_try_flakes/handlers/flake_issues.py
,
Jun 9 2017
Deployed to the app. |
||
►
Sign in to add a comment |
||
Comment 1 by jbudorick@chromium.org
, May 26 2017Components: -Tests>Flaky Infra>Flakiness>Pipeline
Status: WontFix (was: Untriaged)