jeffcarp@, would NOEXPORT=true / "No-Export: true" be honored for Gerrit reviews? If so that would be a workaround. (Forgetting to remove it before landing would lead to an eventual revert though.)

There might also be the option of not creating a PR but instead running Travis jobs from a branch. Bob, that is possible, right?

I believe it's possible, and that web-platform-tests used to run a build on every push to an in-repository branch--not just pull requests. 

Two months ago, this was limited to only building on the master branch (https://github.com/w3c/web-platform-tests/pull/5704/files), likely to cut down the length of the Travis queue (many PRs were adding two builds to the queue--one for the push to the branch and one for the PR to master).

Bob, would you happen to know if it's possible to run jobs for a branch, then create a PR for the branch, and have the previous job associated with it instead of re-running? That would mean that we could run the jobs before landing in Chromium, and the create a PR that can be merged immediately.

I don't believe so. The builds for the branch belong to that branch. A PR requires a build of the base (master) branch at a commit where the head branch is merged in. Any builds of the head branch would be ignored.

This makes sense, because the head could be built on an outdated master. There may not be merge conflicts, but the build result could be invalidated by changes in current master by the time a PR of that branch is opened. We would want a build to re-validate the head branch on current master.

The branches would always be created on the most recent master, but when it's time to merge that can of course have changed, yes. Somewhat off-topic, but when the master branch changes, do all Travis jobs for open PRs run again? If not, then such mishaps are not entirely prevented.

(One way of making this not matter would of course be to make the jobs runs very fast in the first place.)

They don't run again when master changes, so that risk is definitely there, but probably lower, assuming PRs are merged in a timely manner (and by someone with knowledge of current state of master). That said, we've had some instances in WPT where simultaneous PRs have created breakages. A somewhat timely build-able base branch with the PR changes is necessary, but not sufficient to ensure unbroken merges.

OK, so the takeaway is that in order to export changes quickly after they land in Chromium, already having PRs (not branches) in wpt is necessary. Since nobody is notified about those PRs, I guess they don't bother many people.

So... mkwst@, have you had people commenting on those WIP PRs, or why would you prefer them to not exist? (I was also surprised by them, but trying to pin down the biggest pain point.)

> So... mkwst@, have you had people commenting on those WIP PRs, or why would you prefer them to not exist? (I was also surprised by them, but trying to pin down the biggest pain point.)

I'd prefer for them not to exist. Basically, I'd like to be able to work in private until I have something with enough substance to put out in the world. Especially when experimenting with something new and weird, I'd prefer to be able to present a proposal to folks in a vaguely fleshed-out explainer, rather than folks being automatically CC'd on a PR full of barely-thought-out tests.

In these cases, I don't think running things through the stability checks has any real value: the tests probably barely work in `content_shell`, much less on Safari.

An alternative approach would be to add a new directory to `//LayoutTests/` that would let me write tests that are executed via `wptserve`, but that aren't meant to be part of the `web-platform-tests` repository. As-is, I only care about the export question we're discussing here because it's otherwise impossible to write tests that won't cause me more work later. :)

It is possible to mark a review as private in Gerrit, but that's not a feature I've used, and everyone would have to be annoyed once before learning that's how to work around this problem.

Just not creating the PRs and seeing if that creates other problems SGTM. jeffcarp@, would that be a simple change for now?

I guess I'd be fine with marking the review as private if that works today. It would make it more difficult to share with non-committers, though, which is a big drawback. *shrug*

Just not creating the PR seems like the simplest solution for the moment, and the least time investment, given how often this kind of switch is likely to be used.

This is similar to  issue 727914 .

Hi sorry I'm late to the party - we already had feedback on the "over eager" creation of PRs and I made  issue 727914  (only make PRs for CLs with reviewers) to fix it. I created and landed a CL for that issue (crrev.com/c/528453) however ended up not working as intended. I haven't gotten around to debugging the problem further.

Merging this into  issue 727914