Fix all the failures on the Win RandomOrder bot |
|||||||||
Issue descriptionTo deploy LayoutTests on Swarming for Win, we need to fix all the tests which are currently failing. https://luci-milo.appspot.com/buildbot/chromium.fyi/WebKit%20Win%20-%20RandomOrder/ Run locally; ------------ Total tests: 69795 * Passed: 57296 (57205 expected, 91 unexpected) * Skipped: 11640 (11640 expected, 0 unexpected) * Failed: 792 (792 expected, 0 unexpected) * Flaky: 67 (67 expected, 0 unexpected) ------------ Run on swarming; ------------ Run on OS: 'Windows-7-SP1' Total tests: 69795 * Passed: 31987 (31920 expected, 67 unexpected) * Skipped: 36679 (36679 expected, 0 unexpected) * Failed: 1129 (668 expected, >>>461 unexpected<<<) * Flaky: 0 (0 expected, 0 unexpected) ------------
,
May 2 2017
Unexpected Failures: * external/wpt/IndexedDB/interleaved-cursors.html * http/tests/accessibility/slow-document-load.html * http/tests/cache/history-only-cached-subresource-loads-max-age-https.html * http/tests/cache/history-only-cached-subresource-loads.html * http/tests/cache/loaded-from-cache-after-reload-within-iframe.html * http/tests/cache/loaded-from-cache-after-reload.html * http/tests/cache/location-reload.html * http/tests/cache/preload-cleared-after-parsing-canceled-by-js.html * http/tests/cache/subresource-expiration-1.html * http/tests/cache/subresource-expiration-2.html * http/tests/cache/subresource-failover-to-network.html * http/tests/cache/subresource-multiple-instances.html * http/tests/cookies/double-quoted-value-with-semi-colon.html * http/tests/cookies/http-get-cookie-set-in-js.html * http/tests/cookies/js-get-and-set-http-only-cookie.html * http/tests/cookies/js-set-null.html * http/tests/cookies/multiple-cookies.html * http/tests/cookies/simple-cookies-expired.html * http/tests/cookies/simple-cookies-max-age.html * http/tests/cookies/single-quoted-value.html * http/tests/encoding/meta-switch-mid-parse-with-title.html * http/tests/encoding/meta-switch-mid-parse.html * http/tests/fetch/serviceworker/body-mixin-base-https-other-https.html * http/tests/fetch/serviceworker/body-mixin.html * http/tests/fetch/serviceworker/stream-reader-base-https-other-https.html * http/tests/fetch/serviceworker/stream-reader.html * http/tests/fetch/window/body-mixin-base-https-other-https.html * http/tests/fetch/window/body-mixin.html * http/tests/fetch/window/stream-reader-base-https-other-https.html * http/tests/fetch/window/stream-reader.html * ... 431 more ...
,
May 2 2017
,
May 2 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4e68a5f95ca744d875773e3bb12b79ec180a5bd9 commit 4e68a5f95ca744d875773e3bb12b79ec180a5bd9 Author: tansell <tansell@chromium.org> Date: Tue May 02 13:07:59 2017 buildbot: Force OS on the Win WebKit RandomOrder bot. The local VM is Windows 10 and the LayoutTests are passing, so force swarming to also run on Windows 10. BUG= 717344 TBR=dpranke@chromium.com,mcgreevy@chromium.org Review-Url: https://codereview.chromium.org/2850423002 Cr-Commit-Position: refs/heads/master@{#468613} [modify] https://crrev.com/4e68a5f95ca744d875773e3bb12b79ec180a5bd9/testing/buildbot/chromium.fyi.json
,
May 2 2017
Running on Windows 10 reduced the errors down a little bit; ---------------------------------- Run on OS: 'Windows-10-10240' swarming pending 57s Total tests: 69840 * Passed: 30651 (30594 expected, 57 unexpected) * Skipped: 38420 (38420 expected, 0 unexpected) * Failed: 769 (384 expected, >>>385 unexpected<<<) * Flaky: 0 (0 expected, 0 unexpected) Unexpected Failures: * css3/filters/blur-filter-page-scroll-parents.html * external/wpt/IndexedDB/interleaved-cursors.html * http/tests/accessibility/slow-document-load.html * http/tests/cache/history-only-cached-subresource-loads-max-age-https.html * http/tests/cache/history-only-cached-subresource-loads.html * http/tests/cache/loaded-from-cache-after-reload-within-iframe.html * http/tests/cache/loaded-from-cache-after-reload.html * http/tests/cache/location-reload.html * http/tests/cache/preload-cleared-after-parsing-canceled-by-js.html * http/tests/cache/subresource-expiration-1.html * http/tests/cache/subresource-expiration-2.html * http/tests/cache/subresource-failover-to-network.html * http/tests/cache/subresource-multiple-instances.html * http/tests/cookies/double-quoted-value-with-semi-colon.html * http/tests/cookies/http-get-cookie-set-in-js.html * http/tests/cookies/js-get-and-set-http-only-cookie.html * http/tests/cookies/js-set-null.html * http/tests/cookies/multiple-cookies.html * http/tests/cookies/simple-cookies-expired.html * http/tests/cookies/simple-cookies-max-age.html * http/tests/cookies/single-quoted-value.html * http/tests/encoding/meta-switch-mid-parse-with-title.html * http/tests/encoding/meta-switch-mid-parse.html * http/tests/fetch/serviceworker/body-mixin-base-https-other-https.html * http/tests/fetch/serviceworker/body-mixin.html * http/tests/fetch/serviceworker/stream-reader-base-https-other-https.html * http/tests/fetch/serviceworker/stream-reader.html * http/tests/fetch/window/body-mixin-base-https-other-https.html * http/tests/fetch/window/body-mixin.html * http/tests/fetch/window/stream-reader-base-https-other-https.html * ... 355 more ... --------------------------------------------------
,
May 4 2017
The HTTP failures appear to be caused by apache's inability to execute CGI programs; --------------- [Wed May 03 22:57:42 2017] [error] [client 127.0.0.1] (OS 2)The system cannot find the file specified. : couldn't create child process: 720002: subresource-failover-to-network.cgi, referer: http://127.0.0.1:8000/cache/subresource-failover-to-network.html [Wed May 03 22:57:42 2017] [error] [client 127.0.0.1] (OS 2)The system cannot find the file specified. : couldn't spawn child process: E:/b/swarm_slave/w/ir/third_party/WebKit/LayoutTests/http/tests/cache/resources/subresource-failover-to-network.cgi, referer: http://127.0.0.1:8000/cache/subresource-failover-to-network.html [Wed May 03 22:57:42 2017] [error] [client 127.0.0.1] (OS 2)The system cannot find the file specified. : couldn't create child process: 720002: random-cached.cgi, referer: http://127.0.0.1:8000/cache/loaded-from-cache-after-reload.html [Wed May 03 22:57:42 2017] [error] [client 127.0.0.1] (OS 2)The system cannot find the file specified. : couldn't spawn child process: E:/b/swarm_slave/w/ir/third_party/WebKit/LayoutTests/http/tests/cache/resources/random-cached.cgi, referer: http://127.0.0.1:8000/cache/loaded-from-cache-after-reload.html --------------- What is unclear to me is why it seems to be unable to find the file specified. It is included in the isolate and other files which are side-by-side are being loaded successfully. These cgi files are working when run on the local on the buildbot.
,
May 4 2017
Adding smut@ as he seems to understand the swarming image. smut@ do you have any idea why the Windows Apache server when run on swarming would be unable to find these .cgi file?
,
May 4 2017
It might be because on Windows we need //third_party/perl to be isolated...
,
May 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1b05bbd2767a86f2a54b8dbf091aaf3c389842c3 commit 1b05bbd2767a86f2a54b8dbf091aaf3c389842c3 Author: tansell <tansell@chromium.org> Date: Thu May 04 09:30:32 2017 Adding more deps to webkit-layout-tests on Windows. * Adding perl (needed for CGI scripts). * cygwin (possibly needed?) BUG= 717347 TBR=mcgreevy@chromium.org,dpranke@chromium.org Review-Url: https://codereview.chromium.org/2856283003 Cr-Commit-Position: refs/heads/master@{#469297} [modify] https://crrev.com/1b05bbd2767a86f2a54b8dbf091aaf3c389842c3/BUILD.gn
,
May 4 2017
After adding Perl + cygwin as a dependency, the tests now pass on swarming! -------------- webkit_layout_tests on Windows-10-10240 ( 4 mins 24 secs ) webkit_layout_tests on Windows-10-10240 Run on OS: 'Windows-10-10240' swarming pending 16s Total tests: 70040 * Passed: 57520 (57425 expected, 95 unexpected) * Skipped: 11680 (11680 expected, 0 unexpected) * Failed: 791 (791 expected, 0 unexpected) * Flaky: 49 (49 expected, 0 unexpected) -------------- Next would be to run the tests sharded -- however, there only seems to be one bot in the swarming pool that can do so, see https://crbug.com/718379
,
May 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/9fa32783bc56eb6874848a993aaca39ec5dc2591 commit 9fa32783bc56eb6874848a993aaca39ec5dc2591 Author: tansell <tansell@chromium.org> Date: Thu May 04 11:04:34 2017 buildbot: Use Win10-10586 rather Win10-10240 on Random Order bot. Swarming has only a single Win10-10240 bot but about 30-40 Win10-10586 bots. TBR=dpranke@chromium.org,mcgreevy@chromium.org BUG= 718379 , 717347 NOTRY=true Review-Url: https://codereview.chromium.org/2857213004 Cr-Commit-Position: refs/heads/master@{#469307} [modify] https://crrev.com/9fa32783bc56eb6874848a993aaca39ec5dc2591/testing/buildbot/chromium.fyi.json
,
May 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7560c4090d4fce5c16632560f76660ee9201873b commit 7560c4090d4fce5c16632560f76660ee9201873b Author: tansell <tansell@chromium.org> Date: Thu May 04 13:10:57 2017 buildbot: Also run on Random Order Layout Tests on Win7-SP1 TBR=dpranke@chromium.org,mcgreevy@chromium.org BUG= 718379 , 717347 NOTRY=true Review-Url: https://codereview.chromium.org/2857343003 Cr-Commit-Position: refs/heads/master@{#469318} [modify] https://crrev.com/7560c4090d4fce5c16632560f76660ee9201873b/testing/buildbot/chromium.fyi.json
,
May 4 2017
So they seem to work on Windows-10-10586 -------------- webkit_layout_tests on Windows-10-10586 webkit_layout_tests on Windows-10-10586 Run on OS: 'Windows-10-10586' Total tests: 70042 * Passed: 57519 (57425 expected, 94 unexpected) * Skipped: 11680 (11680 expected, 0 unexpected) * Failed: 785 (785 expected, 0 unexpected) * Flaky: 58 (58 expected, 0 unexpected) -------------- And Windows-7-SP1 -------------- webkit_layout_tests webkit_layout_tests Run on OS: 'Windows-7-SP1' Total tests: 70042 * Passed: 57537 (57455 expected, 82 unexpected) * Skipped: 11667 (11667 expected, 0 unexpected) * Failed: 788 (788 expected, 0 unexpected) * Flaky: 50 (50 expected, 0 unexpected) -------------- So, we need to figure out which Windows we should be running on...
,
May 4 2017
There are a lot more Windows-7-SP1 bots... ------------ 319 bots can run this task (0 dead, 0 quarantined) ------------
,
May 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e154b4bb6f3396b3f33d66946d1f7bf54b32e963 commit e154b4bb6f3396b3f33d66946d1f7bf54b32e963 Author: tansell <tansell@chromium.org> Date: Thu May 04 15:14:58 2017 buildbot: Use multiple shards on Win Random Order bot. TBR=dpranke@chromium.org,mcgreevy@chromium.org BUG= 718379 , 717347 NOTRY=true Review-Url: https://codereview.chromium.org/2859173002 Cr-Commit-Position: refs/heads/master@{#469340} [modify] https://crrev.com/e154b4bb6f3396b3f33d66946d1f7bf54b32e963/testing/buildbot/chromium.fyi.json
,
May 4 2017
Turning on multiple sharding seems to have started to cause a whole bunch of failures. Currently investigating.
,
May 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/49345d02f5e5a10cf2789d1d45b4eeec4656b542 commit 49345d02f5e5a10cf2789d1d45b4eeec4656b542 Author: tansell <tansell@chromium.org> Date: Fri May 05 03:40:14 2017 webkitpy: Merge script fix windows directory removing. Attempt to remove a directory tree multiple times as on Windows a process which is currently closing could still have a file open in the directory. Also continue going even if we fail to remove the directory. The merge script currently sometimes fails on Windows with the following error; ------------------------- Traceback (most recent call last): File "E:\b\c\b\WebKit_Win___RandomOrder\src\third_party\WebKit\Tools\Scripts\merge-layout-test-results", line 185, in <module> main(sys.argv[1:]) File "E:\b\c\b\WebKit_Win___RandomOrder\src\third_party\WebKit\Tools\Scripts\merge-layout-test-results", line 161, in main shutil.rmtree(args.output_directory) File "E:\b\depot_tools\python276_bin\lib\shutil.py", line 256, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "E:\b\depot_tools\python276_bin\lib\shutil.py", line 254, in rmtree os.rmdir(path) WindowsError: [Error 32] The process cannot access the file because it is being used by another process: 'E:\\b\\rr\\tmpxu2pfv\\w' WARNING:root:merge_cmd had non-zero return code: 1 ------------------------- BUG= 717347 , 524758 Review-Url: https://codereview.chromium.org/2862173002 Cr-Commit-Position: refs/heads/master@{#469588} [modify] https://crrev.com/49345d02f5e5a10cf2789d1d45b4eeec4656b542/third_party/WebKit/Tools/Scripts/merge-layout-test-results
,
May 5 2017
,
May 5 2017
,
May 8 2017
,
May 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7f6264df1bd3a73c0f8af49adfc345126eebb1cd commit 7f6264df1bd3a73c0f8af49adfc345126eebb1cd Author: tansell <tansell@chromium.org> Date: Mon May 08 04:56:11 2017 LayoutTests: Disable flaky tests on Windows. These tests are failing a small amount of the time on the Windows Random Order bots now they are running sharded on swarming. The tests affected are; * fast/css-grid-layout/grid-self-baseline-two-dimensional.html * fast/css-grid-layout/grid-align-justify-stretch-with-orthogonal-flows.html * fast/dnd/dropEffect-for-effectAllowed.html * external/wpt/css/css-ui-3/outline-004.html Bugs have been filed for each of the failing tests. The most likely cause is that they depend on another test being run and the sharding makes this no longer happening. BUG= 717347 , 718717 , 719298 , 719299 NOTRY=true Review-Url: https://codereview.chromium.org/2872463002 Cr-Commit-Position: refs/heads/master@{#469907} [modify] https://crrev.com/7f6264df1bd3a73c0f8af49adfc345126eebb1cd/third_party/WebKit/LayoutTests/TestExpectations
,
May 19 2017
,
May 19 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a5c323b49440e551e321155f7a9bdb80fec5f80a commit a5c323b49440e551e321155f7a9bdb80fec5f80a Author: Manuel Rego Casasnovas <rego@igalia.com> Date: Fri May 19 08:37:31 2017 LayoutTests: Disable flaky test on Windows The following test is flacky on Windows Random Order bots: * fast/css/font-family-case-insensitive.html The issue is that it uses "ahem" instead of "Ahem" as font, and for some reason that's causing the flakiness. This patch flags it on TestExpectations. BUG= 717347 , 724392 TBR=qyearsley@chromium.org,tansell@chromium.org Change-Id: I7676bfcf4ab9f6759b7a705c159621a8bfc5e496 Reviewed-on: https://chromium-review.googlesource.com/509448 Reviewed-by: Manuel Rego Casasnovas <rego@igalia.com> Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org> Commit-Queue: Manuel Rego Casasnovas <rego@igalia.com> Cr-Commit-Position: refs/heads/master@{#473129} [modify] https://crrev.com/a5c323b49440e551e321155f7a9bdb80fec5f80a/third_party/WebKit/LayoutTests/TestExpectations
,
Sep 6 2017
This bot is green most of the time now! |
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by tansell@chromium.org
, May 2 2017