wpt-importer is incorrectly trying to delete baselines for .any.js tests |
|||
Issue descriptionThe CL uploaded automatically by wpt-importer: https://chromium-review.googlesource.com/c/590771/ Corresponding build of wpt-importer: https://build.chromium.org/p/chromium.infra.cron/builders/wpt-importer/builds/386 Looking at the CL closely, we can find that the initial upload is totally incorrect, apparently it is not containing any of today's diff in the mirror repo https://chromium.googlesource.com/external/w3c/web-platform-tests/. Detail failure log: Issue: Issue number: 590771 (https://chromium-review.googlesource.com/590771) Triggering try jobs for updating expectations. Waiting for try jobs (timeout: 10800 seconds). Waiting. 121 seconds passed. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Waiting. 7150 seconds passed. All jobs finished. Adding test expectations lines to LayoutTests/TestExpectations. Tests to rebaseline: external/wpt/WebIDL/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.html external/wpt/WebIDL/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.worker.html external/wpt/background-fetch/interfaces.worker.html external/wpt/dom/abort/event.any.html external/wpt/dom/abort/event.any.worker.html external/wpt/dom/events/EventTarget-constructible.any.html external/wpt/dom/events/EventTarget-constructible.any.worker.html external/wpt/fetch/api/basic/request-upload.any.html external/wpt/fetch/api/basic/request-upload.any.worker.html external/wpt/fetch/api/basic/scheme-about.any.html external/wpt/fetch/api/basic/scheme-about.any.worker.html external/wpt/fetch/api/cors/cors-preflight-redirect.any.html external/wpt/fetch/api/cors/cors-preflight-redirect.any.worker.html external/wpt/fetch/api/cors/cors-preflight-referrer.any.html external/wpt/fetch/api/cors/cors-preflight-referrer.any.worker.html external/wpt/fetch/api/cors/cors-preflight-star.any.html external/wpt/fetch/api/cors/cors-preflight-star.any.worker.html external/wpt/html/dom/interfaces.worker.html external/wpt/mediacapture-streams/MediaStreamTrack-getSettings.https.html external/wpt/url/historical.any.html external/wpt/url/historical.any.worker.html external/wpt/url/interfaces.any.html external/wpt/url/interfaces.any.worker.html external/wpt/url/toascii.window.html external/wpt/webrtc/RTCPeerConnection-addIceCandidate.html external/wpt/webrtc/RTCPeerConnection-createAnswer.html external/wpt/webrtc/RTCRtpReceiver-getCapabilities.html external/wpt/webrtc/RTCRtpSender-getCapabilities.html external/wpt/webrtc/RTCRtpTransceiver-setCodecPreferences.html external/wpt/workers/constructors/Worker/DedicatedWorkerGlobalScope-members.worker.html external/wpt/workers/constructors/Worker/expected-self-properties.worker.html external/wpt/workers/interfaces.worker.html external/wpt/workers/interfaces/WorkerUtils/importScripts/002.worker.html external/wpt/workers/semantics/interface-objects/001.worker.html external/wpt/workers/semantics/interface-objects/002.worker.html No lines to write to TestExpectations. Generating MANIFEST.json Triggering CQ try jobs. Waiting for try jobs (timeout: 10800 seconds). All jobs finished. No CQ try job results, aborting. step returned non-zero exit code: 1
,
Jul 28 2017
Is trying manual import at https://chromium-review.googlesource.com/c/590936/
,
Jul 28 2017
Thanks for reporting! The reason why it's deleting the baselines for these tests is related to the newly-added .any.js tests (related bugs: bug 703837 , bug 748723 ). P1 because this is blocking correct auto-imports. Details: One file "request-upload.any.js" corresponds to two tests: "request-upload.any.html" and "request-upload.any.worker.html". There's logic in the importer to delete "orphaned" baselines (baselines for tests that been deleted). The importer is deleting the baselines ("request-upload.any.worker-expected.txt" and "request-upload.any-expected.txt" because it can't find corresponding files. Specifically, after doing the import (removing all tests and copying over non-blacklisted tests from local wpt repo) it lists all -expected.txt files, and for each one it checks if there's a corresponding test. In this case, for "request-upload.any.worker-expected.txt" the corresponding file is actually "request-upload.any.js" but it's looking for "request-upload.any.js". Related code: https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py?l=319 There's a simple 1:1 mapping between baseline files and test names, but not between test names and file names, because in theory in wpt a file could have an arbitrary test name listed in the manifest file. Proposed solution: In the importer, list all tests in external/wpt (on the order of 10000 tests), and use that set to check if there's a corresponding existing test for each baseline. Other possible solutions: - We could make a special case for .any.js files and check specifically for those. - Instead of listing all tests at once we could list tests in each directory as needed.
,
Jul 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d154e953bc512dc27f84be61ee5b8cc1a24e304d commit d154e953bc512dc27f84be61ee5b8cc1a24e304d Author: Quinten Yearsley <qyearsley@google.com> Date: Sat Jul 29 01:00:46 2017 WPT Import: don't clear baselines of .any.js tests. Bug: 749980 Change-Id: I627260f31a7354b15ad94fd0a2b75f2e7372c375 Reviewed-on: https://chromium-review.googlesource.com/592234 Commit-Queue: Quinten Yearsley <qyearsley@chromium.org> Reviewed-by: Joshua Bell <jsbell@chromium.org> Cr-Commit-Position: refs/heads/master@{#490564} [modify] https://crrev.com/d154e953bc512dc27f84be61ee5b8cc1a24e304d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py [modify] https://crrev.com/d154e953bc512dc27f84be61ee5b8cc1a24e304d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py [modify] https://crrev.com/d154e953bc512dc27f84be61ee5b8cc1a24e304d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_manifest.py
,
Jul 31 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by leon....@intel.com
, Jul 28 2017