Migrate catapult from GitHub+Rietveld to Gerrit |
|||||||||||||||
Issue descriptionNot sure if we misconfigure anything but anytime someone submit a catapult CL through rietveld, we can't +1 or trigger tryjob or commit.
,
Jun 9 2017
Sorry, an example CL is https://chromium-review.googlesource.com/c/527594/
,
Jun 12 2017
,
Jun 12 2017
,
Jun 12 2017
Yes, this is because catapult is a github repo. The repository on googlesource is just a mirror. I thought[1] I had previously disallowed uploading changes; turns out I hadn't correctly done so. I have now fixed the glitch[2]. If you want to use gerrit to review catapult changes, talk to your org about moving catapult off of GitHub. [1] https://chromium.googlesource.com/external/+/36b01fc56e1a0e21404badde89bdc6d746675b55 [2] https://chromium.googlesource.com/external/+/be6d08eda1c452480018d3621e4797ac7ac0b857
,
Jun 12 2017
When I upload a CL in catapult, I get the message... ===================================== NOTICE: Rietveld is being deprecated. You can upload changes to Gerrit with git cl upload --gerrit or set Gerrit to be your default code review tool with git config gerrit.host true ===================================== Is there anyway to disable this for catapult?
,
Jun 12 2017
Also if gerrit cannot support github repo, does that mean that the team will still keep rietveld around or we need to find a new code review client?
,
Jun 12 2017
We're not going to keep rietveld around just for catapult. You'll either need to move to Gerrit's built in codereview tools, or move your source of truth to *.googlesource.com and set up a github mirror, I think.
,
Jun 12 2017
"You'll either need to move to Gerrit's built in codereview tools" --> I thought Gerrit cannot support github. What do you mean by this?
,
Jun 12 2017
It means I typed it wrong :). I meant "move to *GitHub's* built in codereview tools". Sorry for the confusion.
,
Jun 12 2017
For example, here's the bug tracking the transition of the LUCI repos from using github as the source of truth to using googlesource: https://bugs.chromium.org/p/chromium/issues/detail?id=726507 The work hasn't been started because the person running most of the migrations (i.e. me) is focused on src.git right now; but I'll be tackling all the other repos which continue to use Rietveld shortly after that main switch is complete.
,
Jun 13 2017
Can we have some rationale behind the "If you want to use gerrit to review catapult changes, talk to your org about moving catapult off of GitHub." ? That is the way the CQ did work on the past for Rietveld, so this seems a regression to me introduced by the Gerrit. Why the CQ cannot keep reacting to "ticks" (now +1) and landing stuff to github like it did in the past? Alternatively, how we'd get a CQ if catapult moved to GitHub? Is there any chrome infra integration with Github webhooks in place?
,
Jun 13 2017
The rationale is mostly that Chrome Ops is trying hard to reduce the number of different usage patterns we need to support. If the Gerrit team has a story for how to integrate w/ GitHub, we can look into that, but we would prefer not to support doing additional work on our side to handle such an integration. Yes, this is a change from how things worked previously, and one could certainly view such a change as a regression :). I don't think we have any answer for CQ integration with a GH repo in a non-Rietveld, Gerrit-only world. We'd have to build something new to replace the existing mechanisms. I suppose it's possible that we could continue to support Rietveld plus the Rietveld/CQ integration for GH uses; we'd have to evaluate the cost of that versus the cost of migrating GH-based projects like Catapult to googlesource.com.
,
Jun 13 2017
Just replied on the other thread. I mistakenly though that we'd have lost the CQ in any case, which turned out to be just my misunderstanding.
,
Jun 15 2017
Based on the content of the other thread, I'm repurposing this bug to track switching catapult source-of-truth from github to googlesource, and switching code-review systems from Rietveld to Gerrit at the same time.
,
Jun 15 2017
,
Jul 5 2017
The "replication" plugin *may* be a good alternative to copybara for mirroring back to github: https://gerrit.googlesource.com/plugins/replication/+/master/src/main/resources/Documentation
,
Jul 5 2017
,
Jul 13 2017
,
Jul 25 2017
,
Jul 25 2017
,
Aug 9 2017
,
Aug 9 2017
https://docs.google.com/document/d/1_9SDp_7iIFXF4ASBCdH4JbcftYo6yq1wYS7dnEZce6s/edit provides the list of steps that the LUCI repos used to migrate from github to gerrit last night. Not all of them apply (updating go stuff for example) but basically these are the steps to use: 1) create a new repo (chromium.googlesource.com/catapult-project/catapult?) with the latest content (either via gob-ctl copy configs or via manual force-push) 2) update permissions on the new repo to match chromium/src 3) update infra/config/cq.cfg to add a gerrit cq 3a) update puppet to have the cq start running the new cq 3b) add a buildbucket.cfg listing the trybots syzygy uses 4) update codereview.settings to upload to gerrit by default 5) halt mirroring from github to external/github.com/catapult-project/catapult and mark both the github repo and the googlesource mirror as fully read-only 6) update rietveld to refuse uploads to the catapult project I can do all these steps (except for making the github repo read-only) in about an hour. I can do them today, if you like, or next week when I return from my next block of vacation. Note that these steps do *not* include mirroring back to the github repo. We're still working on solutions for that. If you're okay with having the github mirror go stale for a while we can do the migration immediately; if you strictly require that the github mirror stay up-to-date the entire time, we'll have to wait on this until after that solution is in place.
,
Aug 9 2017
agable@ Thanks for taking this bug! However, would the CQ bots still work as-is with the migration?
,
Aug 9 2017
The catapult CQ bots use bot_update, which knows how to apply gerrit patches. They should continue to work just fine. We would test this (in step 3b) before making the github repo read-only.
,
Aug 10 2017
Thanks for the clear outline in #23, agable! We'd strongly prefer to have the mirror stay up to date, so it would be better to wait until the solution for that is in place. When we're ready to migrate, we should set a clear date/time for the migration and announce it on relevant project mailing lists (tracing@chromium.org, telemetry-announce@chromium.org, chrome-speed-operations@google.com)
,
Sep 7 2017
agable@: 1. Any update on mirroring googlesource to github? 2. How should catapult's bugs be moved to monorail? 3. Is there a way to make monorail bug priority more fine-grained than 0-3? sullivan@: Is an up-to-date mirror still strongly preferred? Why? If it's just about having a homepage on github, can it just link to googlesource like this? https://github.com/chromium/chromium
,
Sep 7 2017
An up-to-date mirror is still strongly preferred. We have a lot of documentation on github, and lots of third-party tooling that we're not aware of pulls from the github repo. I'd really like to make as few changes as possible here.
,
Sep 7 2017
Also re #27: is the bug tracker used by the project in scope for this bug? It'd be better to have separation of concerns if possible.
,
Sep 7 2017
1) I just got back from vacation, and am focusing on the mirroring now. I expect to have the solution running within a week. The bug blocking this one will have the updates for you to follow along. 2-3) As sullivan@ suggested, bugs are a wholly separate concern.
,
Sep 14 2017
Mirroring is now in place! I'd like to set a migration date of Tuesday, september 26. Does that work for you?
,
Sep 18 2017
Lots of people will be OOO for blinkon and promo committee, which should be a good thing because there will be fewer attempts to submit CLs during migration. However I'm usually the point of contact on issues like this and I'll be OOO all day in committee. Maybe the following day?
,
Sep 18 2017
I *just* got an email saying "Hey we put you on a committee go watch the training session that already happened" so yeah, we're going to move this to Wednesday instead.
,
Sep 18 2017
I need one more thing from you: What would you like catapult's home on chromium.googlesource.com to be? We won't be using "external/github.com/catapult-project/catapult" since that repo is explicitly named to be a mirror (although it will continue to exist for historical purposes). What repo path do you want me to create for you?
,
Sep 18 2017
Can we just call it "catapult"?
,
Sep 18 2017
Can do.
,
Sep 27 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/puppet/+/e345b4a43bed5bbbf930b5e3ab0d79b72455c96a commit e345b4a43bed5bbbf930b5e3ab0d79b72455c96a Author: Aaron Gable <agable@chromium.org> Date: Wed Sep 27 19:48:25 2017
,
Sep 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/c0daba28f4d64e87ae53b7f7f492cb510562327a commit c0daba28f4d64e87ae53b7f7f492cb510562327a Author: Aaron Gable <agable@chromium.org> Date: Wed Sep 27 20:19:07 2017 Fix typo in cq.cfg TBR=tandrii@chromium.org Bug: chromium:731091 Change-Id: I59106a1bec15818159eb474a9d106c4fb95abcbf Reviewed-on: https://chromium-review.googlesource.com/687889 Reviewed-by: Aaron Gable <agable@chromium.org> [modify] https://crrev.com/c0daba28f4d64e87ae53b7f7f492cb510562327a/infra/config/cq.cfg
,
Sep 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/catapult/+/e5e3843f330ed7b84082c2e44c1b17bc6caa4487 commit e5e3843f330ed7b84082c2e44c1b17bc6caa4487 Author: Aaron Gable <agable@chromium.org> Date: Wed Sep 27 20:57:06 2017 Make Gerrit the default code review system for Catapult Bug: chromium:731091 Change-Id: Id14bf6b58e54df5ea21e8907083d81cfddee93cf Reviewed-on: https://chromium-review.googlesource.com/687873 Reviewed-by: Annie Sullivan <sullivan@chromium.org> Commit-Queue: Aaron Gable <agable@chromium.org> [modify] https://crrev.com/e5e3843f330ed7b84082c2e44c1b17bc6caa4487/CONTRIBUTING.md [modify] https://crrev.com/e5e3843f330ed7b84082c2e44c1b17bc6caa4487/codereview.settings
,
Sep 27 2017
Done! Please update this bug or the PSA threads with any problems that are found.
,
Sep 28 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3db1b3b3a7c8d164bab2f6501b8c44685b9b8df0 commit 3db1b3b3a7c8d164bab2f6501b8c44685b9b8df0 Author: Aaron Gable <agable@chromium.org> Date: Thu Sep 28 03:17:10 2017 Update DEPS to reference new catapult repo location R=sullivan@chromium.org Bug: 731091 Change-Id: Ie48518b6fdc4d9e5b671f117943978ee18ed5d0d Reviewed-on: https://chromium-review.googlesource.com/688742 Reviewed-by: Annie Sullivan <sullivan@chromium.org> Commit-Queue: Aaron Gable <agable@chromium.org> Cr-Commit-Position: refs/heads/master@{#504882} [modify] https://crrev.com/3db1b3b3a7c8d164bab2f6501b8c44685b9b8df0/DEPS
,
Sep 28 2017
The following revision refers to this bug: https://webrtc.googlesource.com/src.git/+/78ba000d7e71554981c0caa4c4876181840af84e commit 78ba000d7e71554981c0caa4c4876181840af84e Author: Oleh Prypin <oprypin@webrtc.org> Date: Thu Sep 28 13:27:39 2017 Change DEPS URL for catapult to match Chromium See https://chromium-review.googlesource.com/688742 Bug: chromium:731091 Change-Id: I5904e87ac76b08bd3e71dff5ba791dc17de7240f Reviewed-on: https://webrtc-review.googlesource.com/4424 Reviewed-by: Henrik Kjellander <kjellander@webrtc.org> Commit-Queue: Oleh Prypin <oprypin@webrtc.org> Cr-Commit-Position: refs/heads/master@{#20016} [modify] https://crrev.com/78ba000d7e71554981c0caa4c4876181840af84e/DEPS
,
Sep 29 2017
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/879ac56890d419b9ee82603f2aa587c56be276e3 commit 879ac56890d419b9ee82603f2aa587c56be276e3 Author: Michael Achenbach <machenbach@chromium.org> Date: Fri Sep 29 15:08:12 2017 [build] Update DEPS to reference new catapult repo location This ports: https://chromium-review.googlesource.com/c/chromium/src/+/688742 TBR=agable@chromium.org Bug: chromium:731091 Change-Id: I99b6bfcc769a5f3c429668c2d36518bcac80f3cf Reviewed-on: https://chromium-review.googlesource.com/692634 Reviewed-by: Michael Achenbach <machenbach@chromium.org> Commit-Queue: Michael Achenbach <machenbach@chromium.org> Cr-Commit-Position: refs/heads/master@{#48236} [modify] https://crrev.com/879ac56890d419b9ee82603f2aa587c56be276e3/DEPS
,
Oct 2 2017
I get the following error: achuith@achuithz620:~/code/catapult/telemetry/telemetry$ git cl upload --gerrit Credentials for the following hosts are required: github-review.com github.com These are read from /usr/local/google/home/achuith/.gitcookies (or legacy /usr/local/google/home/achuith/.netrc) You can (re)generate your credentials by visiting https://github-review.com/new-password Are there some instructions on migrating my repo that were sent out that I missed?
,
Oct 2 2017
Yep, instructions for that were included in one of the later messages on the internal PSA thread. Repeated here for your convenience: """ Please run in your repo: `git config remote.origin.url https://chromium.googlesource.com/catapult` This will update your repo to be pulling from (and pushing to) the new source of truth. For those of you who work on catapult from inside src/third_party/catapult, this update will happen for you when src.git updates its DEPS file to point at the new source of truth repo. """
,
Oct 3 2017
Thanks Aaron, that worked! This doc should probably be updated: https://catapult.gsrc.io/CONTRIBUTING.md
,
Oct 3 2017
Filed a bug to update doc: https://bugs.chromium.org/p/chromium/issues/detail?id=770986 |
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by tandrii@chromium.org
, Jun 8 2017