Support reftest-wait for wpt tests |
|||||
Issue descriptionThe web-platform-tests (wpt) test suites have a mechanism to allow tests to control when a screenshot is taken for reference based tests: "... the test may have a class="reftest-wait" attribute specified on the root element. This will cause the screenshot to be delayed until the load event has fired and the reftest-wait class has been removed from the root element." (Source: http://web-platform-tests.org/writing-tests/reftests.html#controlling-when-comparison-occurs) Currently we have no support for this, which is a problem for multiple reasons: 1. Many of the tests in external/wpt/webvtt depend on it and are probably failing just because of lack of support. 2. wpt will soon[0] import the csswg-test suite of tests, which rely heavily[1] on this feature. 3. Many of the Blink layout tests use the Blink-only testRunner waitUntilDone/notifyDone infrastructure[2]. Upstreaming these would require porting to reftest-wait, which would require support for it. As noted we already have a similar feature in Blink in the form of testRunner, but it is currently unknown how much of that code could be reused to support reftest-wait. The mechanisms are quite different, though perhaps we could just inject some javascript that uses MutationObserver to listen for class changes and then triggers testRunner internally. [0] 'soon' is ETA 2 weeks at time of this bug filing [1] https://github.com/w3c/csswg-test/search?utf8=%E2%9C%93&q=reftest-wait [2] $ grep -r "testRunner.waitUntilDone" third_party/WebKit/LayoutTests/ | wc -l 3418
,
Mar 3 2017
Proof of concept at https://codereview.chromium.org/2729863004
,
Mar 3 2017
Thanks for filing this and investigating a potential solution :-D
,
Mar 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/255d25e6d389106b75c90110b4e00b022e472cc5 commit 255d25e6d389106b75c90110b4e00b022e472cc5 Author: smcgruer <smcgruer@chromium.org> Date: Tue Mar 07 00:56:43 2017 Support reftest-wait in web-platform-tests The web-platform-tests (wpt) suite allows tests to control when the screenshot is taken for reftests, based on the presence (and then absence) of a 'reftest-wait' class on the root element. To support this mechanism a JavaScript shim is inserted into wpt tests, which uses existing TestRunner methods (waitUntilDone, notifyDone), combined with a MutationObserver on the root element. BUG= 698256 Review-Url: https://codereview.chromium.org/2729863004 Cr-Commit-Position: refs/heads/master@{#455009} [modify] https://crrev.com/255d25e6d389106b75c90110b4e00b022e472cc5/content/shell/test_runner/test_runner.cc [modify] https://crrev.com/255d25e6d389106b75c90110b4e00b022e472cc5/third_party/WebKit/LayoutTests/TestExpectations [add] https://crrev.com/255d25e6d389106b75c90110b4e00b022e472cc5/third_party/WebKit/LayoutTests/external/wpt/infrastructure/reftest-wait-ref.html [add] https://crrev.com/255d25e6d389106b75c90110b4e00b022e472cc5/third_party/WebKit/LayoutTests/external/wpt/infrastructure/reftest-wait.html
,
Mar 7 2017
,
Mar 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/19e4c274dd7efd8731cceb5f0c0ac0a2860ea064 commit 19e4c274dd7efd8731cceb5f0c0ac0a2860ea064 Author: smcgruer <smcgruer@chromium.org> Date: Mon Mar 13 18:11:02 2017 Import wpt infrastructure tests The infrastructure tests contain the LayoutTest for reftest-wait support (http://crrev.com/255d25e), so we should definitely have them in our copy. They also contain a deliberately failing test though, so for now just let the auto-importer pick them up and mark them as skipped in TestExpectations. A follow-up CL will enable the reftest-wait test. BUG= 698256 Review-Url: https://codereview.chromium.org/2738863003 Cr-Commit-Position: refs/heads/master@{#456427} [modify] https://crrev.com/19e4c274dd7efd8731cceb5f0c0ac0a2860ea064/third_party/WebKit/LayoutTests/W3CImportExpectations
,
Jul 3 2017
,
Jul 3 2017
,
Aug 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/21f578129b2b47b16460be10b2fe84670a826536 commit 21f578129b2b47b16460be10b2fe84670a826536 Author: Philip Jägenstedt <foolip@chromium.org> Date: Fri Aug 04 10:00:11 2017 Enable tests expected to fail/timeout but passing in wpt Based on the "Expected to timeout, but passed" and "Expected to fail, but passed" sections of a full run of LayoutTests/external/wpt on Linux Debug. Bug: 310004 , 679742, 698256 Change-Id: Ifc5966c6e8d611b2deb67c56bf2fb06fe35e71d9 Reviewed-on: https://chromium-review.googlesource.com/599728 Reviewed-by: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com> Commit-Queue: Philip Jägenstedt <foolip@chromium.org> Cr-Commit-Position: refs/heads/master@{#491979} [modify] https://crrev.com/21f578129b2b47b16460be10b2fe84670a826536/third_party/WebKit/LayoutTests/TestExpectations |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by smcgruer@chromium.org
, Mar 3 2017