mojo-loading ontimeout-event-override-after-failure.html got flaky recently |
||||
Issue descriptionvirtual/mojo-loading/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html recently got flaky: https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=virtual%2Fmojo-loading%2Fhttp%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html&testType=webkit_tests This flakiness is much worse than the baseline flakiness of the non-virtual test. The error: FAIL: Timeout override reactivated the timer FAIL: error event not dispatched
,
Nov 15 2016
Thanks for the report. > This flakiness is much worse than the baseline flakiness of the non-virtual test. Can you tell me what you mean by "much worse"? https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=ontimeout-event-override-after-failure.html shows both are flaky.
,
Nov 15 2016
I think you are right to question "much worse". Looking at it again, I don't think the data says either way. The recent failures could be a result of the underlying flakiness.
,
Nov 15 2016
Thank you, then I will add a TestExpectation entry.
,
Nov 15 2016
If the pass/fail result of the test is not important, can we just remove the test entirely? Alternatively, the test depends on setTimeout which is known to be a flaky approach to testing. Can we adjust how the test works to reduce flakiness? Maybe we could increase the xhr timeout value of 100?
,
Nov 15 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1a8d96b6418a9dda53ef147a95ef721ad5daaa6e commit 1a8d96b6418a9dda53ef147a95ef721ad5daaa6e Author: yhirano <yhirano@chromium.org> Date: Tue Nov 15 09:13:18 2016 Add expectations on virtual/mojo-loading tests whose base tests have expectations If a test X has a TestExpecation entry, virtual/mojo-loading/X should have an entry as well. BUG= 659917 , 520194 , 518987, 446385, 665165 Review-Url: https://codereview.chromium.org/2502913002 Cr-Commit-Position: refs/heads/master@{#432150} [modify] https://crrev.com/1a8d96b6418a9dda53ef147a95ef721ad5daaa6e/third_party/WebKit/LayoutTests/TestExpectations
,
Nov 15 2016
,
Nov 16 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/613135a62748f6d2f875f11b9096c368a3873714 commit 613135a62748f6d2f875f11b9096c368a3873714 Author: pdr <pdr@chromium.org> Date: Wed Nov 16 06:45:55 2016 Deflake ontimeout-event-override-after-failure by waiting longer for http This test flakes when the http server does not respond quick enough. I suspect this happens on bots that are busy running other tests in parallel. I was able to simulate this flakiness by lowering the XHR timeout to 13ms. This patch increases the XHR timeout from 100ms to 180ms which should give more time for the test http server to respond. This has no effect on overall test time because the pass condition is behind a 200ms timer. After this patch lands, we need to monitor the following: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html If the flakiness disappears, the lines in TestExpectations should be removed. BUG= 464736 , 665165 Review-Url: https://codereview.chromium.org/2504713003 Cr-Commit-Position: refs/heads/master@{#432409} [modify] https://crrev.com/613135a62748f6d2f875f11b9096c368a3873714/third_party/WebKit/LayoutTests/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html
,
Nov 24 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4 commit ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4 Author: pdr <pdr@chromium.org> Date: Thu Nov 24 21:43:44 2016 Rewrite ontimeout-event-override-after-failure for less flakes and better errors This patch rewrites ontimeout-event-override-after-failure with these changes: 1) The test no longer depends on setTimeout racing the network to pass. Instead of having an async setTimeout check for success, we rely on async callbacks in xhr's onerror and ontimeout. This should make the test faster in the common case, and still pass if the test server is slow. 2) The xhr timeout has been increased to 500ms. If this test is flaky due to the test server not responding fast enough, this change should deflake it. 3) The two failure modes of this test now print different errors which should help in debugging this if it is still flaky. 4) The test has been reformated to 80cols. This test is flaky but only very rarely: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html A previous attempt at deflaking this test by increasing the timeout failed: https://crrev.com/613135a62748f6d2f875f11b9096c368a3873714 After this patch lands, we need to monitor the following: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html If the flakiness disappears, the lines in TestExpectations should be removed. BUG= 464736 , 665165 Review-Url: https://codereview.chromium.org/2518383003 Cr-Commit-Position: refs/heads/master@{#434390} [modify] https://crrev.com/ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4/third_party/WebKit/LayoutTests/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html
,
Dec 7 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f26356128e7e04ff3793f93ffc98a89e7d25b710 commit f26356128e7e04ff3793f93ffc98a89e7d25b710 Author: pdr <pdr@chromium.org> Date: Wed Dec 07 06:57:08 2016 Mark ontimeout-event-override-after-failure.html as not flaky This test is no longer flaky: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html BUG= 464736 , 665165 Review-Url: https://codereview.chromium.org/2560723002 Cr-Commit-Position: refs/heads/master@{#436881} [modify] https://crrev.com/f26356128e7e04ff3793f93ffc98a89e7d25b710/third_party/WebKit/LayoutTests/TestExpectations |
||||
►
Sign in to add a comment |
||||
Comment 1 by moh...@chromium.org
, Nov 14 2016