4 tests in http/tests/appcache/ are flaky when run in random order |
|||
Issue descriptionThe tests: http/tests/appcache/main-resource-redirect.html http/tests/appcache/non-html.xhtml http/tests/appcache/obsolete-error-events.html http/tests/appcache/remove-cache.html appear to be flaky when run in random order. Steps to reproduce: This should pass consistently: Tools/Scripts/run-webkit-tests http/tests/appcache/ This should fail intermittently: Tools/Scripts/run-webkit-tests http/tests/appcache/ --order=random We're working on removing order dependency from Blink layout tests. More context here: https://docs.google.com/document/d/11-628Ny6hY__kfIv__zVD0m3rA0OB9tQLLC_gJI_PNg/edit
,
Nov 29 2016
Able to repro. Output locally and on the bots is just a spurious extra (or sometimes missing): CONSOLE WARNING: line 22: Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/. I see that when running w/o random order, so it appears to just be general flakiness. http/tests/appcache/obsolete-error-events.html on the dashboard shows as generally flaky anyway.
,
Nov 29 2016
Many of the appcache tests use a trick where the manifest is a PHP file which can be accessed with ?command=XXX to control server behavior (e.g. "connect" or "disconnect" to simulate online/offline behavior). These 4 in particular have code like: // If the manifest script happened to be in a wrong state, reset it. var timeoutId = setTimeout(resetManifest, 100); (where resetManifest does a sync XHR) ... which explains the random order flakiness: if a previous run left the server in a bad state, the test needs to do an extra XHR to reset it, causing the extra logging.
,
Nov 29 2016
,
Nov 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8 commit 1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8 Author: jsbell <jsbell@chromium.org> Date: Wed Nov 30 22:31:15 2016 Deflake tests in http/tests/appcache/ when run in random order The tests use sync XHR to poke the server to tweak the AppCache manifest state. Since the manifest is PHP server is re-used across tests the initial state could be unexpected. The tests handled this as necessary, but meant that a variable number of sync XHRs would be issued. Since sync XHRs log a deprecation warning the output was flaky. Fix this by switching server control to async XHR. Additionally, obsolete-error-events.html would time out if no update was needed, so handle the 'noupdate' event too. BUG= 663860 R=jeffcarp@chromium.org Review-Url: https://codereview.chromium.org/2540833002 Cr-Commit-Position: refs/heads/master@{#435450} [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/RandomOrderExpectations [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/main-resource-redirect-expected.txt [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/main-resource-redirect.html [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/non-html-expected.txt [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/non-html.xhtml [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/obsolete-error-events-expected.txt [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/obsolete-error-events.html [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/remove-cache-expected.txt [modify] https://crrev.com/1b6aabf1422f519dfe93fde0ba65b04c3a7a60a8/third_party/WebKit/LayoutTests/http/tests/appcache/remove-cache.html
,
Nov 30 2016
|
|||
►
Sign in to add a comment |
|||
Comment 1 by jeffcarp@chromium.org
, Nov 14 2016