New issue
Advanced search Search tips

Issue 649010 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature

Blocking:
issue 621599



Sign in to add a comment

w3c-test-autoroller: Update or remove paths from TestExpectations for renamed or deleted tests.

Project Member Reported by qyears...@chromium.org, Sep 21 2016

Issue description

In the latest builds for w3c-test-autoroller (https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller), it is failing the PRESUBMIT with:


webkitpy.common.system.executive.ScriptError: Failed to run "['git', 'cl', 'upload', '-f', '--rietveld', '-m', 'W3C auto test import CL.\n\nTBR=qyearsley@chromium.org', u'--cc=dom-dev@chromium.org', '--auth-refresh-token-json', '/creds/refresh_tokens/blink-w3c-test-autoroller']" exit_code: 1

Running presubmit upload checks ...

** Presubmit ERRORS **

LayoutTests/TestExpectations:818 Path does not exist. imported/wpt/pointerevents/pointerevent_suppress_compat_events_on_click.html
LayoutTests/TestExpectations:819 Path does not exist. imported/wpt/pointerevents/pointerevent_suppress_compat_events_on_drag_mouse.html


In this case, what's happening is that in the upstream repo, those two tests got renamed, and there were expectations for those tests.

If we were to just run git cl upload with --bypass-hooks, then it could upload, but then it would be left in a state where the TestExpectations have the wrong test names, so the right solution should instead be to update TestExpectations for any renamed or removed tests.

There should be a method added to DepsUpdater, called when doing an auto-update and before uploading a CL.
 
Issue 647726 has been merged into this issue.
Project Member

Comment 2 by bugdroid1@chromium.org, Sep 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6

commit 809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6
Author: qyearsley <qyearsley@chromium.org>
Date: Thu Sep 22 16:12:36 2016

Update TestExpectations for removed/renamed tests when updating w3c tests.

This change would make it so that when update-w3c-deps is run, lines in
TestExpectations for deleted or renamed tests are updated.

I believe that this wouldn't work for lines with directories instead of
individual tests as-is, although I think that's a relatively rare case.

BUG= 649010 

Review-Url: https://codereview.chromium.org/2360893002
Cr-Commit-Position: refs/heads/master@{#420362}

[modify] https://crrev.com/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
[modify] https://crrev.com/809c4145ce868eeb7bbbb6d65d08bc1eac16aaf6/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater_unittest.py

Status: Fixed (was: Assigned)
Cc: jeffcarp@chromium.org
Status: Started (was: Fixed)
It appears that this was fixed for the generic test expectations file (LayoutTests/TestExpectations) but not other test expectation files like FlagExpectations; the latest w3c-test-autoroller job failed due to this:

https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller/builds/7756
Project Member

Comment 5 by bugdroid1@chromium.org, Oct 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/01bcbb2eecb603397dc60604e4bc42c9582ea136

commit 01bcbb2eecb603397dc60604e4bc42c9582ea136
Author: qyearsley <qyearsley@chromium.org>
Date: Thu Oct 20 17:04:54 2016

Update all non-generic test expectations files for removed tests when updating w3c tests.

This is a follow up to http://crrev.com/2360893002, which made it so that
lines in the generic test expectations file (LayoutTests/TestExpectations)
are updated when tests are removed or renamed in the upstream repo.

It turns out that this should be done for all test expectations files,
not just that one.

BUG= 649010 

Review-Url: https://chromiumcodereview.appspot.com/2432883003
Cr-Commit-Position: refs/heads/master@{#426509}

[modify] https://crrev.com/01bcbb2eecb603397dc60604e4bc42c9582ea136/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py
[modify] https://crrev.com/01bcbb2eecb603397dc60604e4bc42c9582ea136/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater_unittest.py

Status: Fixed (was: Started)
Cc: tkent@chromium.org
Status: Started (was: Fixed)
w3c-test-autoroller hit an interesting edge case today:

There is a recent change in upstream https://github.com/w3c/web-platform-tests/commit/144370536ea1bc0e2288d436a72add3e8fea1a23 which splits one test (Range-mutations.html) into several and extracts the common JS.

When updating, since dom/ranges/Range-mutations.js is very similar to Range-mutations.html which was removed, so this was considered a rename. So, update-w3c-deps tried to update test expectations with Range-mutations.html, and change them to Range-mutations.js, which was incorrect.

The actual best thing to try to do in this case might be to replace Range-mutations.html with all of the tests it was split into, but would be somewhat complex for the auto-importer to do.

Proposed solution: change deps-updater.py to only consider changes to be renames if the similarity is 100%. With this change, Range-mutations.html would be considered to be deleted, and new expectations would have to be added for the individual sub-tests that it was split into.
Project Member

Comment 8 by bugdroid1@chromium.org, Oct 26 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/15fc403e351f277a622ba802104f60a12c43f3c4

commit 15fc403e351f277a622ba802104f60a12c43f3c4
Author: qyearsley <qyearsley@chromium.org>
Date: Wed Oct 26 21:43:14 2016

W3C test importer: only consider tests to be renamed with 100% similarity.

BUG= 649010 

Review-Url: https://codereview.chromium.org/2449303002
Cr-Commit-Position: refs/heads/master@{#427820}

[modify] https://crrev.com/15fc403e351f277a622ba802104f60a12c43f3c4/third_party/WebKit/Tools/Scripts/webkitpy/w3c/deps_updater.py

Status: Fixed (was: Started)

Sign in to add a comment