Increasing the priority of this since it seems to have happened elsewhere as well:
https://codereview.chromium.org/2889683007
https://chromium-review.googlesource.com/518344

These files were added locally and reverted by import:
LayoutTests/external/wpt/css/css-position-3/position-sticky-table-tfoot-bottom-ref.html
LayoutTests/external/wpt/css/css-position-3/position-sticky-table-tfoot-bottom.html
LayoutTests/external/wpt/css/css-position-3/position-sticky-table-thead-top-ref.html
LayoutTests/external/wpt/css/css-position-3/position-sticky-table-thead-top.html

If I understand correctly: For the "fetch" test, we believe that the problem was that the importer should have aborted because there was an exportable but not yet exported test, and because it didn't abort, it ended up removing the newly-added-and-not-yet-exported tests.

One initial thought: Part of the definition of an "exportable" commit is that the patch needs to be apply cleanly on web-platform-tests HEAD; if for some reason the patch doesn't apply, then it's not considered exportable.
Relevant code: https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py?l=76

The second case (with the new "css-position-3" tests) seems different than the first, since it seems like those tests were first successfully landed in web-platform-tests and then the importer removed them, which is interesting and also incorrect.

Ah, the definition of is_exportable seems like a problem, although not necessarily the culprit here. IIRC, the point of trying to apply the patch is to see if it's already been upstreamed, to skip those. But if there are conflicts, that still needs to be resolved. Do you know what is_exportable returned for that commit in this case?

See also https://chromium-review.googlesource.com/523033 for another incident. Don't know if the definition of exportable was the problem there as well?

I'm planning to spend time investigating this this week!

Based on my comment in #2, my initial suspicion is that what should be changed should be that the importer should also abort if there are any locally-committed changes to wpt since the last export, regardless of whether the patch applies to wpt HEAD or not.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2bb44d71091f0595af103cdc2845ddaa85122a96

commit 2bb44d71091f0595af103cdc2845ddaa85122a96
Author: Quinten Yearsley <qyearsley@google.com>
Date: Tue Jun 13 20:23:40 2017

wpt sync: Log modified files when logging exportable Chromium commits.

The main motivation for this change is that later if there are
exportable changes and I want to make an import CL that doesn't undo
those changes, this will make it easy to quickly see what import changes
to revert after running `wpt-import --ignore-exportable-changes`.

This change includes:
 - Add a url method to ChromiumCommit
 - Add a MockChromiumCommit class.
 - Change LocalWPT to print URL and files when there's a patch conflict
 - Change TestImporter to print URL and files for commits pending export

Bug:  727923 
Change-Id: Ic7ca45e50062eb0d661d89727c32ba60e0df2bf9
Reviewed-on: https://chromium-review.googlesource.com/533915
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479128}
[modify] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit.py
[add] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit_mock.py
[modify] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit_unittest.py
[modify] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
[modify] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py
[modify] https://crrev.com/2bb44d71091f0595af103cdc2845ddaa85122a96/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6ad31af8c32c1a7bb67acb1d17469328ae7265b8

commit 6ad31af8c32c1a7bb67acb1d17469328ae7265b8
Author: Jeff Carpenter <jeffcarp@chromium.org>
Date: Wed Jun 14 19:59:48 2017

[WPT Sync] Make Importer use same exportable commit finder as Exporter

Also mark the previously used method as private to discourage use.

Bug:  727923 
Change-Id: I145d27aa5b51dbfe95a91f13978974d2294f08b2
Reviewed-on: https://chromium-review.googlesource.com/534775
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479476}
[modify] https://crrev.com/6ad31af8c32c1a7bb67acb1d17469328ae7265b8/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py
[modify] https://crrev.com/6ad31af8c32c1a7bb67acb1d17469328ae7265b8/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common_unittest.py
[modify] https://crrev.com/6ad31af8c32c1a7bb67acb1d17469328ae7265b8/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py
[modify] https://crrev.com/6ad31af8c32c1a7bb67acb1d17469328ae7265b8/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter_unittest.py
[modify] https://crrev.com/6ad31af8c32c1a7bb67acb1d17469328ae7265b8/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py

After the last change, we now have an interesting problem:

If an exportable change is landed in Chromium, but never landed upstream (because we don't want to land it), it can be considered "exportable but not exported" for a long time.

For example, my silly spelling CL from a while ago (https://chromium-review.googlesource.com/c/513527/) was landed in Chromium, a PR was created but closed without merging, and it is considered "exportable but not exported".

The most obvious fix here is: if there is a corresponding PR and that PR is *closed*, then it should not be considered exportable.

I think this might be done by changing the is_exportable function (at https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py?l=77) to also take a WPTGitHub instance, and then check something like `wpt_github.pr_with_position(chromium_commit.position).state == 'open'`.

But, this seems like it would make it so that wpt-import requires a GitHub user and token in order to start and check exportable commits... this could be set up on the importer bot, but should probably be optional if wpt-import is run manually.

Jeff, what do you think? Does that sound right? Might it be possible to check a PR status without GitHub credentials?

That's interesting. Treating a closed PR as an abandoned export makes sense. https://developer.github.com/v3/#authentication says "requests that require authentication", so presumably not all requests do require authentication?

That approach sounds good. Just checking a PR status shouldn't require credentials - I think you're limited to 320 unauthenticated requests to the GitHub API per hour, however the importer probably shares an egress IP with a lot of other things in the Golo, which would mean anything else making unauthenticated GitHub requests from the Golo would eat into that limit. Worst case we can add credentials to the importer if that becomes a problem, but this should be fine.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2745df54ac745edf2248dbb35e4944a3fb190c6a

commit 2745df54ac745edf2248dbb35e4944a3fb190c6a
Author: Quinten Yearsley <qyearsley@google.com>
Date: Fri Jun 23 20:38:31 2017

Consider commits with closed PRs to be not exportable.

In this CL:
 - In wpt_github, make credentials optional
 - In common, change is_exportable so that it takes a
   WPTGitHub and uses it to check PR status
 - Also change tests and add coverage for new behavior
 - Change importer and exporter to pass a WPTGitHub object

Bug:  727923 
Change-Id: I0603c1a9f15f4ae8d660a8c5ae7b48e2b84d348f
Reviewed-on: https://chromium-review.googlesource.com/541918
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Cr-Commit-Position: refs/heads/master@{#482013}
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit_mock.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common_unittest.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/local_wpt.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter_unittest.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py
[modify] https://crrev.com/2745df54ac745edf2248dbb35e4944a3fb190c6a/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/60b0c7c484dcdf25c80f4038cddd9120c4e1b970

commit 60b0c7c484dcdf25c80f4038cddd9120c4e1b970
Author: Quinten Yearsley <qyearsley@google.com>
Date: Mon Jun 26 22:40:51 2017

Put the default PR history window size in WPTGitHub.

This CL would remove the "number of PRs to fetch at once" constant
from test_exporter and put it in WPTGitHub, so that this number would
also be used by the importer when fetching PRs.

I believe that after this, the importer will be able to more accurately
decide whether a Chromium commit corresponds to an open or closed PR
since it will fetch more PRs.

Bug:  727923 
Change-Id: Idd1a04f9cb10c66f96260fc2cdef11e5ad0fbe94
Reviewed-on: https://chromium-review.googlesource.com/548996
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#482442}
[modify] https://crrev.com/60b0c7c484dcdf25c80f4038cddd9120c4e1b970/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py
[modify] https://crrev.com/60b0c7c484dcdf25c80f4038cddd9120c4e1b970/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/963fe26d0addd30ac2eac4ea07f49d7802d1dee1

commit 963fe26d0addd30ac2eac4ea07f49d7802d1dee1
Author: Quinten Yearsley <qyearsley@google.com>
Date: Tue Jun 27 20:26:06 2017

In is_exportable, use Change-Id to get corresponding PRs.

Relying on commit position doesn't work due to  http://crbug.com/737178 ;
relying on only change ID doesn't work all commits use Gerrit.

Bug:  727923 
Change-Id: Ia21d2f97088bc26ec33b2ed3b172c2cba270a487
Reviewed-on: https://chromium-review.googlesource.com/550558
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#482731}
[modify] https://crrev.com/963fe26d0addd30ac2eac4ea07f49d7802d1dee1/third_party/WebKit/Tools/Scripts/webkitpy/w3c/chromium_commit_mock.py
[modify] https://crrev.com/963fe26d0addd30ac2eac4ea07f49d7802d1dee1/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py
[modify] https://crrev.com/963fe26d0addd30ac2eac4ea07f49d7802d1dee1/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common_unittest.py
[modify] https://crrev.com/963fe26d0addd30ac2eac4ea07f49d7802d1dee1/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4317879feaebc3d9e36cc96612a5dd2b78b69f0c

commit 4317879feaebc3d9e36cc96612a5dd2b78b69f0c
Author: Quinten Yearsley <qyearsley@google.com>
Date: Thu Jun 29 15:47:04 2017

Add GitHub credentials to importer, if available.

Purpose: The importer is also going to be requesting lists of pull
requests from GitHub, so I think it makes sense for it to have the same
credentials as the exporter.

This CL extracts the related code into common.py and uses it in
test_importer.py.

Bug:  727923 
Change-Id: Iac88de469f1f52841faa434a7d9fdef49d5e1109
Reviewed-on: https://chromium-review.googlesource.com/548976
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Cr-Commit-Position: refs/heads/master@{#483373}
[modify] https://crrev.com/4317879feaebc3d9e36cc96612a5dd2b78b69f0c/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py
[modify] https://crrev.com/4317879feaebc3d9e36cc96612a5dd2b78b69f0c/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common_unittest.py
[modify] https://crrev.com/4317879feaebc3d9e36cc96612a5dd2b78b69f0c/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py
[modify] https://crrev.com/4317879feaebc3d9e36cc96612a5dd2b78b69f0c/third_party/WebKit/Tools/Scripts/wpt-export

After #7, the original bug here was resolved (since the importer would no longer clobber exportable commits). The rest of the changes are follow-ups to make sure that the importer doesn't abort when it doesn't have to abort.

In the latest import, there is still one case where the importer is aborting when it shouldn't be:
https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.infra.cron%2Fwpt-importer%2F247%2F%2B%2Frecipes%2Fsteps%2Fupdate_wpt%2F0%2Fstdout

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/e9f2277602b11980391041c7455fdcc915490115

commit e9f2277602b11980391041c7455fdcc915490115
Author: Quinten Yearsley <qyearsley@google.com>
Date: Fri Jun 30 23:02:24 2017

In the wpt_import recipe, also pass path to GitHub credentials.

After a recent change, the WPT importer now makes requests to GitHub
just like the exporter does. I copied the GitHub credentials from the
slave running the wpt export script to the slave running the wpt
import script.

The purpose of this CL is to make the importer use the same
credentials as the exporter when making requests to GitHub.

Bug:  727923 
Change-Id: Iecbdf38da7b13515de3e76233a6db96695868549
Reviewed-on: https://chromium-review.googlesource.com/555613
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/e9f2277602b11980391041c7455fdcc915490115/recipes/recipes/wpt_import.expected/wpt-import-with-issue.json
[modify] https://crrev.com/e9f2277602b11980391041c7455fdcc915490115/recipes/recipes/wpt_import.expected/wpt-import-without-issue.json
[modify] https://crrev.com/e9f2277602b11980391041c7455fdcc915490115/recipes/recipes/wpt_import.py

qyearsley@, this has been fully fixed now, right?

I believe so, yes -- assuming that the problem was that exportable commits before the last exported commit were not considered exportable and thus clobbered, which I believe is the case.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/995f3e6f520b491f9489a8842e1b3beb592a993d

commit 995f3e6f520b491f9489a8842e1b3beb592a993d
Author: Quinten Yearsley <qyearsley@google.com>
Date: Thu Jul 06 00:19:57 2017

Add method pr_for_chromium_commit to WPTGitHub, and use it in importer

Reason:
While the is_exportable function was using both Change Id and commit
position to find the PR, test_importer was using commit position
to find the PR. Because of  https://crbug.com/737178 , that sometimes
returns the wrong PR number, which caused some confusion.

This CL extracts the logic used to get the PR for a commit to avoid
repetition, and makes it so it's used in both is_exportable and the
importer.

Bug:  727923 
Change-Id: I9e5dfbdc2f8805693e1aa01760cb64951a6dd908
Reviewed-on: https://chromium-review.googlesource.com/558200
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Jeff Carpenter <jeffcarp@chromium.org>
Cr-Commit-Position: refs/heads/master@{#484402}
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/common.py
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter_unittest.py
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer.py
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_importer_unittest.py
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github.py
[modify] https://crrev.com/995f3e6f520b491f9489a8842e1b3beb592a993d/third_party/WebKit/Tools/Scripts/webkitpy/w3c/wpt_github_mock.py