I filed this under Infra>Git>Admin partly because in creating the harfbuzz repo the infra git admin suggested the whole refs/branch-heads/* thing, which isn't quite what we need. If there is nothing infra git admin can do to work around any of this, perhaps this is better for depot_tools (Infra>SDK?) or Build? I'm not entirely sure how to route this.

agable: I've been told you're probably the one who can evaluate this and state if this is resolvable and which way. I'm assigning this to you for the moment mostly for evaluation and input. I'm more than willing to poke at some code to make something work, but this isn't my area and I'd hate to work on something and have it be the wrong thing.

To summarize: we want to mirror a third party library, be able to DEPS in commits from this mirror, be able to push our own (low volume) commits into the mirror as needed, be able to roll DEPS to our own commits in this mirror, and be able to see our own commits in this mirror from trybots. With the current setup for the harfbuzz mirror we cannot see our own commits from the trybots, making testing difficult.

Biggest question: do you require up-to-date mirrors of any branches other than the master branch?

My suggestion would be to follow the model of the https://chromium.googlesource.com/external/github.com/grpc/grpc/+/master repo:
* Only mirror refs/heads/master from github to googlesource
* Maintain a separate refs/heads/chromium which integrates your patches on top of the upstream master via merge commits
  - You could also maintain a progressive series of refs/heads/chromium/<branchnumber> or something like that (like grpc does), so that you can maintain your patches via rebasing but not lose the old hashes when you do a rebase
* Point to sha1s that are on the chromium branch from DEPS files as necessary

Thank you so much for pointing this out, I think we are only interested in the master branch (and if we needed more we could add them to the mirror manually). It appears the difference is in the refspec to mirror

grpc has:
      refspec: <
        source: "refs/heads/master"
        destination: "refs/heads/master"
      >

where harfbuzz currently has:
      refspec: <
        source: "refs/heads/*"
        destination: "refs/heads/*"
      >

it appears that it might be even more flexible to use:
      refspec: <
        source: "refs/heads/*"
        destination: "refs/heads/upstream/*"
      >

so that all of the upstream branches are mirrored, but leaves us free to create any other branches we may need. This also works nicely with seeing all the commits in gitiles.

Yep, mirroring refs/heads/* into refs/heads/upstream/* is something that other repos have done as well, and would also be a great choice here.

So current git admin: please change harfbuzz's current mirroring scheme to push into refs/heads/upstream/* instead.

This is now a request to Infra>Git>Admin to update the chromium/external/github.com/harfbuzz/harfbuzz repo to change 'destination: "refs/heads/*" to 'destination: "refs/heads/upstream/*"' in the copy_inbound. The existing entries in refs/heads can all be removed (feel free to delete the whole repo and re-create, there's nothing pointing at it at the moment).

After this change the configuration at https://chromium-review.googlesource.com/admin/repos/external/github.com/harfbuzz/harfbuzz,access should be changed so that all of the 'Reference: refs/branch-heads/*' permissions be moved to refs/heads/chromium/* .

And while poking at the configuration, you may wish to check if 'Rights Inherit From All-Projects' is correct. It appears that maybe this should be 'Rights Inherit From external' (as other external mirrors seem to be).

Just to double check: branch-heads/* commits in chromium/external/github.com/harfbuzz/harfbuzz was never referenced in any DEPS? Because if we recreated it, the new repo may no longer have all commits referenced by historical DEPS, thus breaking them.

That is correct, no DEPS have ever pointed to any commits referenced directly by any refs in /refs/branch-heads/* . The few refs which exist there are solely for testing purposes (which is why this request exists). Indeed, I don't think any DEPS file has ever pointed at this repo at all yet.

Done: https://chromium.googlesource.com/external/github.com/harfbuzz/harfbuzz/

I omitted upstream tags. Let me know if I should add them too (refs/tags/upstream?).

I created a checkout of this repo, touched some headers for testing and tried pushing as follows:
$ git push origin testTouchHeaders:refs/branch-heads/testTouchHeaders
and got:
fatal: remote error: push at least one ref not permitted

Is the access configuration permitting changes as Ben requested in #6? Am I doing something wrong? Reopening.

The additional request is that the configuration at https://chromium-review.googlesource.com/admin/repos/external/github.com/harfbuzz/harfbuzz,access be set so that members of the gerrit group harfbuzz-commiters can create references and push to refs/heads/chromium/* . We are not planning on creating any tags, so leaving the tags alone should be fine.

I imagine that when the repo was either blown away and re-created or the settings were made to inherit from 'external' that the old settings were lost, which is probably why the wording in the second paragraph of comment 6 didn't make a lot of sense at the time.

Right, sorry, forgot to make refs/heads/* writable. Unfortunately, old repo is gone, so I can't lookup what exact permissions it head for branch-heads. Do you use (chromium standard) CodeReview+1 workflow or (Gerrit standard) CodeReview+2, Verified+1 workflow?

I setup following permissions for now (Gerrit standard): https://screenshot.googleplex.com/mrx0e40T4p0.png

Let me know if I need to change them for CodeReview+1 workflow.

Gerrit standard sounds fine. Is it ok to make refs/heads/* writable since refs/heads/upstream/* is owned by gob? That's why I originally requested these permissions on refs/heads/chromium/* (so they wouldn't accidentally conflict). I'm fine with the current setup I think, but want to be sure that the permissions are set up such that we don't accidentally mess anything up trying to push to refs/upstream/* .

Pushing to refs/heads/* (or refs/heads/chromium/*) is fine. Gerrit will forbid pushing to refs/heads/upstream/* though. So it should be impossible to mess up.

I'll mark this as fixed, please reopen if something else is needed.

vadimish@, thanks very much for helping us with this issue.

Unfortunately, I can't push to these branches:

$ git push origin testTouchHeaders:refs/heads/chromium/mTest
Counting objects: 8, done.
[...]
remote: Processing changes: refs: 1, done    
To https://chromium.googlesource.com/external/github.com/harfbuzz/harfbuzz
 ! [remote rejected]   testTouchHeaders -> chromium/mTest (prohibited by Gerrit: create not permitted for refs/heads/chromium/mTest)

$ git push origin testTouchHeaders:refs/heads/mTest
Counting objects: 8, done.
[...]
remote: Resolving deltas: 100% (5/5)
remote: Processing changes: refs: 1, done    
To https://chromium.googlesource.com/external/github.com/harfbuzz/harfbuzz
 ! [remote rejected]   testTouchHeaders -> mTest (prohibited by Gerrit: create not permitted for refs/heads/mTest)
error: failed to push some refs to 'https://chromium.googlesource.com/external/github.com/harfbuzz/harfbuzz'

Maybe we need a "Create" permission, if such a things exists?

Yeah, sorry, forgot about "Create Reference". Added it (and a bunch more). Please try again.

Thanks very much, vadimsh@. Pushing to chromium/mTest works now, as in
$ git push origin testTouchHeaders:refs/heads/chromium/mTest

However, I can't force push to this branch after creating it, and I can't delete branches. Could you add rights for that?

Done. Please confirm all necessary permissions are setup correctly now and mark the bug as fixed if so.

Confirming that I can force push and delete branches, thanks very much for fixing this so quickly.