New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 753487 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

[WPT Export] Check the patch and commit message against the final checked-in version before merging a PR

Project Member Reported by robertma@chromium.org, Aug 8 2017

Issue description

We currently only update PRs (both the patch and the description) when processing in-flight gerrit CLs (https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/w3c/test_exporter.py?l=75&rcl=c28a5164a85511aa3d85ec39868d639548d15d7c). There is an edge case where the PR might be merged with outdated patch and/or description:

1. A CL is created in Gerrit.
2. Exporter runs and creates a PR for the CL.
3. CL is changed and submitted before the exporter (successfully) runs again.
4. Exporter finally runs again and finds the commit in the history, so it automatically merges the corresponding PR (without checking their contents).

This scenario rarely happens because the Chromium commit queue usually takes more than 10 minutes to finish, which is the interval the exporter runs. However, relying on such assumption is too fragile. For example, the exporter could fail, for whatever reason, a few times and miss the chance of updating PRs.
 
Components: Blink>Infra>Ecosystem
Owner: robertma@chromium.org
Status: Available (was: Untriaged)
Cc: robertma@chromium.org raphael....@intel.com
 Issue 754643  has been merged into this issue.
Labels: -Pri-2 Pri-1
Prioritizing as this might be more likely than we expected.
https://github.com/w3c/web-platform-tests/pull/6624 is believed to be another case of this issue. In that case, there was a last-minute automatic rebase done by CQ missed by the exporter. See the comments for more details.
Status: Started (was: Available)
Instead of "check", what we really need here is an *update* -- when an in-flight CL lands, we need to update the corresponding provisional PR with the final version of the change that is checked into chromium master.

We could first compare the PR with the commit before updating the PR, but we still need to talk to GitHub APIs and run git commands for the comparison and would not save any work than always updating provisional PRs before merging.
Correction: as foolip@ kindly pointed out, my comment above missed an important aspect: Travis CI. By forcing an update every time, Travis will be triggered and the export latency will increase.

Hence, ideally we do need to check the patch and skip this final update if the patches in the PR and chromium master are the same.

However, we believe the tradeoff of reliable export vs increased export latency is worthwhile. We should fix the immediate "data loss" problem first, and then work on a follow-up to optimize.
commit	cd03f8ee05fca279fe59b6e143f966e2bd8cfec8
author	Robert Ma <robertma@chromium.org>
date	Thu Aug 24 17:43:09 2017

Update provisional PRs with the checked-in change before merging

Provisional PRs are created from Gerrit in-flight CLs. When they are
landed, we update the PRs (both the patches and messages) one last
time with the final changes checked in before merging the PRs to make
sure not to miss any last-minute revisions.

Bug: 753487
Change-Id: I14ca7c17188824a9be46b214587d0be7878225dc
Reviewed-on: https://chromium-review.googlesource.com/618100
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Commit-Queue: Robert Ma <robertma@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#497116}


## Manually pasting the commit because monorail was down when the commit was merged.
Labels: -Pri-1 Pri-2
Now we have assured the exported patches are correct. The next step would be only updating the patch if the patch in PR is different from the patch checked in to avoid unnecessary Travis runs (the ideal solution described in comment 8).

This is now a "good to have". Downgrading to P2.
Ecosystem infra bug triage ping:
Is this still on your roadmap Robert?  How is export latency looking these days in practice?  If it's OK, and we think this extra check is a bunch of work/complexity, then perhaps we should call this Pri-3 (just "nice to have someday")?
Labels: -Pri-2 Pri-3
Status: Assigned (was: Started)
Thanks for the reminder, Rick.

It is on my roadmap, but you are right, P3 would be more appropriate.

Sign in to add a comment