New issue
Advanced search Search tips

Issue 665165 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

mojo-loading ontimeout-event-override-after-failure.html got flaky recently

Project Member Reported by pdr@chromium.org, Nov 14 2016

Issue description

virtual/mojo-loading/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html recently got flaky:
https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=virtual%2Fmojo-loading%2Fhttp%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html&testType=webkit_tests

This flakiness is much worse than the baseline flakiness of the non-virtual test.

The error:
FAIL: Timeout override reactivated the timer
FAIL: error event not dispatched
 

Comment 1 by moh...@chromium.org, Nov 14 2016

Not sure why I'm cc'd on this bug. Maybe, you wanted to cc someone else?
Cc: yhirano@chromium.org
Thanks for the report.

> This flakiness is much worse than the baseline flakiness of the non-virtual test.

Can you tell me what you mean by "much worse"? https://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=ontimeout-event-override-after-failure.html shows both are flaky.

Comment 3 by pdr@chromium.org, Nov 15 2016

Cc: -moh...@chromium.org
I think you are right to question "much worse". Looking at it again, I don't think the data says either way. The recent failures could be a result of the underlying flakiness.
Cc: -yhirano@chromium.org
Owner: yhirano@chromium.org
Thank you, then I will add a TestExpectation entry.

Comment 5 by pdr@chromium.org, Nov 15 2016

If the pass/fail result of the test is not important, can we just remove the test entirely?

Alternatively, the test depends on setTimeout which is known to be a flaky approach to testing. Can we adjust how the test works to reduce flakiness? Maybe we could increase the xhr timeout value of 100?
Project Member

Comment 6 by bugdroid1@chromium.org, Nov 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1a8d96b6418a9dda53ef147a95ef721ad5daaa6e

commit 1a8d96b6418a9dda53ef147a95ef721ad5daaa6e
Author: yhirano <yhirano@chromium.org>
Date: Tue Nov 15 09:13:18 2016

Add expectations on virtual/mojo-loading tests whose base tests have expectations

If a test X has a TestExpecation entry, virtual/mojo-loading/X should have an
entry as well.

BUG= 659917 ,  520194 , 518987, 446385,  665165 

Review-Url: https://codereview.chromium.org/2502913002
Cr-Commit-Position: refs/heads/master@{#432150}

[modify] https://crrev.com/1a8d96b6418a9dda53ef147a95ef721ad5daaa6e/third_party/WebKit/LayoutTests/TestExpectations

Status: Fixed (was: Assigned)
#5: Sounds reasonable, but it should be addressed in  issue 464736 .
Project Member

Comment 8 by bugdroid1@chromium.org, Nov 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/613135a62748f6d2f875f11b9096c368a3873714

commit 613135a62748f6d2f875f11b9096c368a3873714
Author: pdr <pdr@chromium.org>
Date: Wed Nov 16 06:45:55 2016

Deflake ontimeout-event-override-after-failure by waiting longer for http

This test flakes when the http server does not respond quick enough. I
suspect this happens on bots that are busy running other tests in
parallel. I was able to simulate this flakiness by lowering the XHR
timeout to 13ms.

This patch increases the XHR timeout from 100ms to 180ms which should
give more time for the test http server to respond. This has no effect
on overall test time because the pass condition is behind a 200ms timer.

After this patch lands, we need to monitor the following:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html
If the flakiness disappears, the lines in TestExpectations should be removed.

BUG= 464736 , 665165 

Review-Url: https://codereview.chromium.org/2504713003
Cr-Commit-Position: refs/heads/master@{#432409}

[modify] https://crrev.com/613135a62748f6d2f875f11b9096c368a3873714/third_party/WebKit/LayoutTests/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 24 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4

commit ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4
Author: pdr <pdr@chromium.org>
Date: Thu Nov 24 21:43:44 2016

Rewrite ontimeout-event-override-after-failure for less flakes and better errors

This patch rewrites ontimeout-event-override-after-failure with these changes:
1) The test no longer depends on setTimeout racing the network to pass. Instead
of having an async setTimeout check for success, we rely on async callbacks in
xhr's onerror and ontimeout. This should make the test faster in the common case,
and still pass if the test server is slow.
2) The xhr timeout has been increased to 500ms. If this test is flaky due to the
test server not responding fast enough, this change should deflake it.
3) The two failure modes of this test now print different errors which should
help in debugging this if it is still flaky.
4) The test has been reformated to 80cols.

This test is flaky but only very rarely:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html

A previous attempt at deflaking this test by increasing the timeout failed:
https://crrev.com/613135a62748f6d2f875f11b9096c368a3873714

After this patch lands, we need to monitor the following:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#tests=http%2Ftests%2Fxmlhttprequest%2Fontimeout-event-override-after-failure.html
If the flakiness disappears, the lines in TestExpectations should be removed.

BUG= 464736 , 665165 

Review-Url: https://codereview.chromium.org/2518383003
Cr-Commit-Position: refs/heads/master@{#434390}

[modify] https://crrev.com/ff4bdcb669f08b5119a63d8eb60c40bb5c4b89c4/third_party/WebKit/LayoutTests/http/tests/xmlhttprequest/ontimeout-event-override-after-failure.html

Sign in to add a comment