Buildbots ignoring failing tests. |
|||||||
Issue descriptionlinux-chromeos-rel buildbot is green even though many unit tests fail. https://build.chromium.org/p/chromium.chromiumos/builders/linux-chromeos-rel/builds/2791 Failures, look for FileSystemProviderService failures https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium.chromiumos%2Flinux-chromeos-rel%2F2791%2F%2B%2Frecipes%2Fsteps%2Funit_tests%2F0%2Fstdout First build with these tests failures is here: https://build.chromium.org/p/chromium.chromiumos/builders/linux-chromeos-rel/builds/2600 and it's green :O. In this particular case the regression started with this CL: https://chromium-review.googlesource.com/c/chromium/src/+/759680 which for some strange reason passed CQ smoothly.
,
Nov 21 2017
+cc Chromium sheriffs.
,
Nov 21 2017
I guess it's passing in the retry window specified for the tests. It's interesting that the test is passing on the second trial for most of builds. https://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=unit_tests&tests=eSystemProviderServiceTest.RestoreFileSystem_OnExtensionLoa
,
Dec 11 2017
Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable? If a fix is in active development, please set the status to Started. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 11 2017
Adding the sheriffing label and CC-ing current sheriffs.
,
Dec 12 2017
?? Is this still the case?
,
Dec 12 2017
,
Dec 12 2017
Thanks to mtomasz@, this is temporarily fixed, but I'm not sure this is completely fixed. +dpranke, could you take a look at this issue? Thanks,
,
Dec 12 2017
,
Dec 12 2017
As noted in #c3, the tests are failing, then passing on the retry. See, e.g.: https://ci.chromium.org/buildbot/chromium.chromiumos/linux-chromeos-rel/2791 https://chromium-swarm.appspot.com/task?id=39f5eef866adf210&refresh=10&show_raw=1 In the latest build (on the waterfall), at least, it looks like the test is passing: https://ci.chromium.org/buildbot/chromium.chromiumos/linux-chromeos-rel/3435 https://chromium-swarm.appspot.com/task?id=3a648a08d3681d10&refresh=10&show_raw=1 I think the system worked the way it was supposed to. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by mtomasz@chromium.org
, Nov 21 2017