WPT Import: platform-specific rebaselines do not always take baseline search paths into account |
||
Issue descriptionThis isn't strictly specific to WPT Import and may be more of a general webkit-patch rebaseline-cl problem. WPT imports have started failing in https://luci-milo.appspot.com/buildbot/chromium.infra.cron/wpt-importer/2576 because of css/css-shapes-1/shape-outside/values/shape-outside-computed-shape-000.html, which causes the following chain of events: * The win10_blink_rel and win7_blink_rel trybots fail one of the tests ('FAIL ex units assert_equals: expected "polygon(130.078px 130.078px)" but got "polygon(130.08px 130.08px)"'). Note that all tests pass on the Linux and Mac trybots. * webkit-patch rebaseline-cl adds LayoutTests/platform/win/external/wpt/css/css-shapes-1/shape-outside/values/shape-outside-computed-shape-000-expected.txt with the Windows test expectations. * wpt-import uploads updates the CL (https://chromium-review.googlesource.com/c/chromium/src/+/702160 in this case) and includes the new expectation file. The CQ is triggered. * The CQ fails because the layout tests fail on linux_chromium_rel_ng. That happens because the baseline search path for Linux is "linux -> win -> generic", so it ends up using that newly-generated Windows expectation that does not actually work on Linux.
,
Oct 6 2017
Ugh, that didn't work, wpt-import is removing that line from TestExpectations when rebaselining... Let's see if skipping works.
,
Oct 6 2017
After talking to Robert on IRC, we'll try dropping the [ Linux ] part from TestExpectations instead.
,
Oct 6 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/48db2ab0f5a4b446f630cbceca35f3a378494a99 commit 48db2ab0f5a4b446f630cbceca35f3a378494a99 Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Fri Oct 06 15:21:37 2017 Drop platform specifier in upcoming css-shapes-1 test. It's broken only on Linux, but having the platform specifier there causes wpt-import to remove the TestExpectations entry on a new import. The idea is to see if the line remains there when we drop the platform specifier and then manually add a proper expectation later. TBR=foolip@chromium.org,qyearsley@chromium.org,robertma@chromium.org Bug: 772327 Change-Id: I4638765e16eed97e70d8d4bd2040acf876e9a83f No-Try: True Reviewed-on: https://chromium-review.googlesource.com/704839 Reviewed-by: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com> Reviewed-by: Robert Ma <robertma@chromium.org> Commit-Queue: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#507076} [modify] https://crrev.com/48db2ab0f5a4b446f630cbceca35f3a378494a99/third_party/WebKit/LayoutTests/TestExpectations
,
Oct 6 2017
We still don't know if the temporary fix above works. The one with high hope timed out unfortunately. Reminder: after the import succeeds, we need to remove the temporary expectation and fix the baseline. As for the root cause of the issue, I'm suspecting it's related to https://crbug.com/767356#c6 , which will be the first thing after I come back from vacation. Thanks again, Raphael!
,
Oct 9 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f5d49a3e3f4834acc5191b656653be0d6ebecfb0 commit f5d49a3e3f4834acc5191b656653be0d6ebecfb0 Author: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com> Date: Mon Oct 09 12:27:37 2017 Stop marking shape-outside-computed-shape-000 as flaky. This was a temporary measure introduced in commit 48db2ab0f ("Drop platform specifier in upcoming css-shapes-1 test") so that the test could actually be imported from web-platform-tests. Due to bug 767356 , the rebaselining process was generating only an expectations file for Windows, which ended up being used on Linux due to the baseline search process and caused the test to be reported as failing on Linux. Manually add different expectations for Windows and Linux to fix the situation. TBR=foolip@chromium.org,qyearsley@chromium.org,robertma@chromium.org Bug: 772327 No-Try: True Change-Id: I2e4db4a3e6a5094d31d903871c74d512fcd0bceb Reviewed-on: https://chromium-review.googlesource.com/707060 Reviewed-by: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com> Commit-Queue: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com> Cr-Commit-Position: refs/heads/master@{#507360} [modify] https://crrev.com/f5d49a3e3f4834acc5191b656653be0d6ebecfb0/third_party/WebKit/LayoutTests/TestExpectations [add] https://crrev.com/f5d49a3e3f4834acc5191b656653be0d6ebecfb0/third_party/WebKit/LayoutTests/platform/linux/external/wpt/css/css-shapes-1/shape-outside/values/shape-outside-computed-shape-000-expected.txt [add] https://crrev.com/f5d49a3e3f4834acc5191b656653be0d6ebecfb0/third_party/WebKit/LayoutTests/platform/win/external/wpt/css/css-shapes-1/shape-outside/values/shape-outside-computed-shape-000-expected.txt
,
Oct 9 2017
Closing this one now that we have proper baselines and the test's no longer marked as flaky. The underlying fix will likely land as part of bug 767356 . |
||
►
Sign in to add a comment |
||
Comment 1 by bugdroid1@chromium.org
, Oct 6 2017