win7_chromium_rel_ng false positive success |
|||
Issue descriptionLook at this: https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/163821 The build says success, while webkit_layout_tests failed. Not sure what's happening.
,
May 15 2018
Could it be because the failing test is marked experimental? webkit_layout_tests (with patch, experimental)
,
May 15 2018
Indeed, this is the point of experimental tests fserb, you're not the only person to get confused by this -- anything in particular that you think could help make this clearer? Adding a brief explanation that gets linked from the build page? Something else?
,
May 15 2018
hmmmm. But the problem is those tests WERE broken on linux,mac and they should have been broken on win7 too. So I was confused by win7 being green, not by the test being red. I had noticed the "experimental" on the title, but I assumed that, since it was replacing the normal one, that were the real deal. Is it possible to run both the experimental and the non-experimental one?
,
May 15 2018
It's the same suite; it's not replacing the normal one, the builder is simply treating it differently. We marked it as experimental a month or so ago because the suite was problematically flaky (or flakier) on windows.
,
May 15 2018
Ahn. Can you call it something else that makes is explicit that is being ignored, like "experimental - results ignored" And wouldn't that mean that if I have a LayoutTest that only breaks on win7, that I'll be able to submit my change?
,
May 15 2018
Yes, "experimental" means that if tests fail, your changes is not blocked from landing. As jbudorick@ says, in a sense this isn't worse than not running them at all. (In another sense, it might be worse, because you might not understand what "experimental" means but you understand what it means for the test to not be there). |
|||
►
Sign in to add a comment |
|||
Comment 1 by fs...@chromium.org
, May 15 2018