wpt-update-expectations tried to add lines with overlapping sets of specifiers |
||||||||
Issue descriptionIn https://build.chromium.org/p/chromium.infra.cron/builders/wpt-importer/builds/401, where there were many crashy mac jobs, wpt-update-expectations tried to add: crbug.com/626703 [ Android Retina ] external/wpt/encoding/legacy-mb-tchinese/big5/big5-encode-href.html [ Skip ] crbug.com/626703 [ Android Mac10.10 Mac10.12 ] external/wpt/encoding/legacy-mb-tchinese/big5/big5-encode-href.html [ Crash ] The thing that's overlapping is the Android expectation, which is only added because the tests are skipped on Android. The intention behind adding skipped platforms is so that if the result is cross-platform, the specifier list can be simplified. But in this case the result was not cross-platform. Proposal: skipped platforms should only be added to specifier lists in wpt-update-expectations if the result happened on most other platforms, or all other platforms.
,
Aug 8 2017
,
Aug 8 2017
The place where the extra specifier is being added to the TestExpectation line is at: https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_expectations_updater.py?l=337 Robert, are you interested in taking this one?
,
Aug 8 2017
Taking on this issue.
,
Aug 10 2017
This is happening today as well: https://build.chromium.org/p/chromium.infra.cron/builders/wpt-importer/builds/449
,
Aug 10 2017
Yes, we do expect this to happen quite often (and once it happens, it will likly stick around until manual intervention or the flaky tests in question are changed). Bumping to P1.
,
Aug 16 2017
,
Aug 28 2017
,
Sep 6 2017
,
Sep 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0a1d3e903a663a0c229e8d4aac1407c8219d6b92 commit 0a1d3e903a663a0c229e8d4aac1407c8219d6b92 Author: Robert Ma <robertma@chromium.org> Date: Thu Sep 07 16:02:49 2017 Avoid creating overlapping sets of specifiers in test expectations With this CL, skipped platforms are only added to a specifier list when the extended list can cover all platforms. Bug: 750684 Change-Id: I384be52b21a0770a8d41147d81c3269da90ccb14 Reviewed-on: https://chromium-review.googlesource.com/653740 Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Cr-Commit-Position: refs/heads/master@{#500310} [modify] https://crrev.com/0a1d3e903a663a0c229e8d4aac1407c8219d6b92/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_expectations_updater.py [modify] https://crrev.com/0a1d3e903a663a0c229e8d4aac1407c8219d6b92/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_expectations_updater_unittest.py
,
Sep 7 2017
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by qyears...@chromium.org
, Jul 31 2017