w3c-test-autoroller: Update or remove paths from TestExpectations for renamed or deleted tests. |
||||||
Issue descriptionIn the latest builds for w3c-test-autoroller (https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller), it is failing the PRESUBMIT with: webkitpy.common.system.executive.ScriptError: Failed to run "['git', 'cl', 'upload', '-f', '--rietveld', '-m', 'W3C auto test import CL.\n\nTBR=qyearsley@chromium.org', u'--cc=dom-dev@chromium.org', '--auth-refresh-token-json', '/creds/refresh_tokens/blink-w3c-test-autoroller']" exit_code: 1 Running presubmit upload checks ... ** Presubmit ERRORS ** LayoutTests/TestExpectations:818 Path does not exist. imported/wpt/pointerevents/pointerevent_suppress_compat_events_on_click.html LayoutTests/TestExpectations:819 Path does not exist. imported/wpt/pointerevents/pointerevent_suppress_compat_events_on_drag_mouse.html In this case, what's happening is that in the upstream repo, those two tests got renamed, and there were expectations for those tests. If we were to just run git cl upload with --bypass-hooks, then it could upload, but then it would be left in a state where the TestExpectations have the wrong test names, so the right solution should instead be to update TestExpectations for any renamed or removed tests. There should be a method added to DepsUpdater, called when doing an auto-update and before uploading a CL.
,
Sep 22 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6 commit 809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6 Author: qyearsley <qyearsley@chromium.org> Date: Thu Sep 22 16:12:36 2016 Update TestExpectations for removed/renamed tests when updating w3c tests. This change would make it so that when update-w3c-deps is run, lines in TestExpectations for deleted or renamed tests are updated. I believe that this wouldn't work for lines with directories instead of individual tests as-is, although I think that's a relatively rare case. BUG= 649010 Review-Url: https://codereview.chromium.org/2360893002 Cr-Commit-Position: refs/heads/master@{#420362} [modify] https://crrev.com/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py [modify] https://crrev.com/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater_unittest.py
,
Sep 22 2016
,
Oct 19 2016
It appears that this was fixed for the generic test expectations file (LayoutTests/TestExpectations) but not other test expectation files like FlagExpectations; the latest w3c-test-autoroller job failed due to this: https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/7756
,
Oct 20 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/01bcbb2eecb603397dc60604e4bc42c9582ea136 commit 01bcbb2eecb603397dc60604e4bc42c9582ea136 Author: qyearsley <qyearsley@chromium.org> Date: Thu Oct 20 17:04:54 2016 Update all non-generic test expectations files for removed tests when updating w3c tests. This is a follow up to http://crrev.com/2360893002, which made it so that lines in the generic test expectations file (LayoutTests/TestExpectations) are updated when tests are removed or renamed in the upstream repo. It turns out that this should be done for all test expectations files, not just that one. BUG= 649010 Review-Url: https://chromiumcodereview.appspot.com/2432883003 Cr-Commit-Position: refs/heads/master@{#426509} [modify] https://crrev.com/01bcbb2eecb603397dc60604e4bc42c9582ea136/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py [modify] https://crrev.com/01bcbb2eecb603397dc60604e4bc42c9582ea136/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater_unittest.py
,
Oct 21 2016
,
Oct 25 2016
w3c-test-autoroller hit an interesting edge case today: There is a recent change in upstream https://github.com/w3c/web-platform-tests/commit/144370536ea1bc0e2288d436a72add3e8fea1a23 which splits one test (Range-mutations.html) into several and extracts the common JS. When updating, since dom/ranges/Range-mutations.js is very similar to Range-mutations.html which was removed, so this was considered a rename. So, update-w3c-deps tried to update test expectations with Range-mutations.html, and change them to Range-mutations.js, which was incorrect. The actual best thing to try to do in this case might be to replace Range-mutations.html with all of the tests it was split into, but would be somewhat complex for the auto-importer to do. Proposed solution: change deps-updater.py to only consider changes to be renames if the similarity is 100%. With this change, Range-mutations.html would be considered to be deleted, and new expectations would have to be added for the individual sub-tests that it was split into.
,
Oct 26 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/15fc403e351f277a622ba802104f60a12c43f3c4 commit 15fc403e351f277a622ba802104f60a12c43f3c4 Author: qyearsley <qyearsley@chromium.org> Date: Wed Oct 26 21:43:14 2016 W3C test importer: only consider tests to be renamed with 100% similarity. BUG= 649010 Review-Url: https://codereview.chromium.org/2449303002 Cr-Commit-Position: refs/heads/master@{#427820} [modify] https://crrev.com/15fc403e351f277a622ba802104f60a12c43f3c4/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
,
Oct 31 2016
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by qyears...@chromium.org
, Sep 21 2016