[WPT Export] Check the patch and commit message against the final checked-in version before merging a PR |
||||||
Issue descriptionWe currently only update PRs (both the patch and the description) when processing in-flight gerrit CLs (https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py?l=75&rcl=c28a5164a85511aa3d85ec39868d639548d15d7c). There is an edge case where the PR might be merged with outdated patch and/or description: 1. A CL is created in Gerrit. 2. Exporter runs and creates a PR for the CL. 3. CL is changed and submitted before the exporter (successfully) runs again. 4. Exporter finally runs again and finds the commit in the history, so it automatically merges the corresponding PR (without checking their contents). This scenario rarely happens because the Chromium commit queue usually takes more than 10 minutes to finish, which is the interval the exporter runs. However, relying on such assumption is too fragile. For example, the exporter could fail, for whatever reason, a few times and miss the chance of updating PRs.
,
Aug 9 2017
,
Aug 14 2017
,
Aug 14 2017
Prioritizing as this might be more likely than we expected.
,
Aug 14 2017
https://github.com/w3c/web-platform-tests/pull/6624 is believed to be another case of this issue. In that case, there was a last-minute automatic rebase done by CQ missed by the exporter. See the comments for more details.
,
Aug 16 2017
,
Aug 16 2017
Instead of "check", what we really need here is an *update* -- when an in-flight CL lands, we need to update the corresponding provisional PR with the final version of the change that is checked into chromium master. We could first compare the PR with the commit before updating the PR, but we still need to talk to GitHub APIs and run git commands for the comparison and would not save any work than always updating provisional PRs before merging.
,
Aug 17 2017
Correction: as foolip@ kindly pointed out, my comment above missed an important aspect: Travis CI. By forcing an update every time, Travis will be triggered and the export latency will increase. Hence, ideally we do need to check the patch and skip this final update if the patches in the PR and chromium master are the same. However, we believe the tradeoff of reliable export vs increased export latency is worthwhile. We should fix the immediate "data loss" problem first, and then work on a follow-up to optimize.
,
Aug 24 2017
commit cd03f8ee05fca279fe59b6e143f966e2bd8cfec8 author Robert Ma <robertma@chromium.org> date Thu Aug 24 17:43:09 2017 Update provisional PRs with the checked-in change before merging Provisional PRs are created from Gerrit in-flight CLs. When they are landed, we update the PRs (both the patches and messages) one last time with the final changes checked in before merging the PRs to make sure not to miss any last-minute revisions. Bug: 753487 Change-Id: I14ca7c17188824a9be46b214587d0be7878225dc Reviewed-on: https://chromium-review.googlesource.com/618100 Commit-Queue: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Robert Ma <robertma@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Cr-Commit-Position: refs/heads/master@{#497116} ## Manually pasting the commit because monorail was down when the commit was merged.
,
Aug 25 2017
Now we have assured the exported patches are correct. The next step would be only updating the patch if the patch in PR is different from the patch checked in to avoid unnecessary Travis runs (the ideal solution described in comment 8). This is now a "good to have". Downgrading to P2.
,
Nov 3 2017
Ecosystem infra bug triage ping: Is this still on your roadmap Robert? How is export latency looking these days in practice? If it's OK, and we think this extra check is a bunch of work/complexity, then perhaps we should call this Pri-3 (just "nice to have someday")?
,
Nov 3 2017
Thanks for the reminder, Rick. It is on my roadmap, but you are right, P3 would be more appropriate. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by robertma@chromium.org
, Aug 8 2017