New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 676672 link

Starred by 6 users

Issue metadata

Status: Duplicate
Merged: issue 765334
Owner: ----
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

[W3C auto-import] Notify owners for new failures in imported wpt directories.

Project Member Reported by nzolghadr@chromium.org, Dec 22 2016

Issue description

It would be great if we had some sort of owners for each directory in wpt and whenever a change particularly a new failure happened as the result of automated import they would get notified so they can investigate and triage the bug in Blink.
 
Summary: Notifications for changes (particularly new failures) in imported wpt directories (was: Notifications for changed (particularly new failures) in imported wpt directories)

Comment 2 by rbyers@chromium.org, Dec 22 2016

We do have a convention of listing owners for imported WPT directories, eg. pointerevents is listed as "Owners: mustaq@chromium.org" today: https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/W3CImportExpectations?q=W3CImportExpectations&sq=package:chromium&dr&l=291

We've talked about auto-filing bugs for new failures in directories listed as "owned".  I think that (or at least sending a notification e-mail) would be a good idea.  qyearsley@ what's your current thinking here?

For generic notifications of changes, I think it's simplest if people just subscribe to the upstream OWNERS files.  That'll be more timely and give you a link to contribute directly to the PR conversation on GitHub.
Status: Available (was: Untriaged)
Summary: Notifications for changes (particularly new failures) in imported wpt directories. (was: Notifications for changes (particularly new failures) in imported wpt directories)
Currently the w3c-test-autoroller uses a JSON mapping of some test directories to owners (https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/directory_owners.json) to decide who to CC when auto-importing, and AFAIK that works now (for those directories with owners in that file). But that file isn't up to date, and isn't based on the "Owners" lines in W3CImportExpectations, so I think we need to improve our solution to make sure that the owners map is always kept up to date.

I still think that auto-filing bugs for new failures is likely be a good idea, although care should be taken to make sure it's not too noisy (e.g. we probably don't want a separate bug for every single individual test, and some for some newly-imported tests with -expected.txt files, we may not necessarily want to file a bug; in previous discussion (when the interns were here), we talked about filing one bug per directory upon importing, and only for directories that had opted-in to auto-bug-filing.

That said, w3c-test-autoroller still hasn't had a very long streak of successes, and there are probably other issues to fix first :-/

rbyers@' suggestion to subscribe to upstream OWNERS files sounds like a good idea for keeping up with changes in general. Although this won't tell you directly whether tests are now failing in Blink.

The related bug for auto-filing bugs for new failures is:  bug 626529 .

Comment 4 by rbyers@chromium.org, Dec 23 2016

Thanks Quinten, that's great!

Ideally we'd have just one place that lists the WPT directories we care about, and who in the project cares about them.  Perhaps we could eliminate W3CImportExpectations entirely and rely just on directory_owners.json to indicate what directories to import (after updating to match our current list of course)?


Labels: -Type-Bug Type-Feature
Summary: [WPT auto-import] Notify owners for new failures in imported wpt directories. (was: Notifications for changes (particularly new failures) in imported wpt directories.)
Summary: [W3C auto-import] Notify owners for new failures in imported wpt directories. (was: [WPT auto-import] Notify owners for new failures in imported wpt directories.)
Components: Blink>Infra>Predictability
Components: -Blink>Infra
Cc: morrita@chromium.org abarth@chromium.org meh@chromium.org eseidel@chromium.org dglazkov@chromium.org rafaelw@chromium.org
 Issue 360773  has been merged into this issue.
 Issue 626529  has been merged into this issue.
https://codereview.chromium.org/2830203003/#msg19 was a voice in favor of a bug auto-filer.
Cc: wolenetz@chromium.org
Components: Blink>Infra>Ecosystem
Components: -Blink>Infra>Predictability
Mergedinto: 765334
Status: Duplicate (was: Available)

Sign in to add a comment