New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 690513 link

Starred by 6 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Task


Sign in to add a comment

Move Chrome OS code back from AOSP

Project Member Reported by derat@chromium.org, Feb 9 2017

Issue description

A bunch of Chrome OS code got moved into AOSP to support the Brillo project. As far as I'm aware, the current plan is that most (all?) of that code will not be used in Brillo. We're carrying technical debt due to this, and moving this code back to Chrome OS repositories would make Chrome OS development easier.

Looking in the aosp/ directory in a Chrome OS checkout, I think it may be possible to move the following back out of AOSP:

external/dbus-binding-generator
external/libbrillo <-- should also be renamed back to "libchromeos"
external/libchrome <-- possibly depended on by other AOSP code :-/
external/libmojo
system/connectivity/apmanager
system/connectivity/shill
system/firewalld

I'm not sure whether these are still needed by Brillo (or Android):

external/minijail
system/tpm
system/update_engine
system/webservd

There may be some dependencies from checked-in-but-unused Brillo code like the C++ portion of WeaveService.
 
external/dbus-binding-generator

external/libbrillo <-- should also be renamed back to "libchromeos"
This is used by Android in update_engine.

external/libchrome <-- possibly depended on by other AOSP code :-/
external/libmojo

system/connectivity/apmanager
This can likely move back.

system/connectivity/shill
This can likely move back.

system/firewalld
This can move back.

external/minijail
Minijail is currently used by Android. We could consider putting it in github or something so that upstream is not in Android.

system/tpm

system/update_engine
update_engine is currently used by Android.

system/webservd

Comment 2 by lee...@google.com, Feb 9 2017

Update Engine is used by Android mainline now (we use it on Pixel). So these 3 repos are in active use. both librillo and libchrome are dependacies of update_engine. We should rename libbrillo!
external/libbrillo
external/libchrome
system/update_engine

external/minijail is used by mainline android too now.

system/tpm is not used as far as I know but its talked about sometimes :)

While I wasn't fond of the fact that we moved all our code to AOSP, do we benefit if not all of the repos can be brought back?  Given that update_engine is definitely an AOSP thing now, are we stuck with a flow where we will always have an AOSP dependency for pure chromeos?  If so, what advantage do we have of moving some of these repos back?  
I'd argue that the individual daemons/services would still benefit from moving back even if some of the libs don't.
+snanda 

I heard shill development is already moved back to chromeos tree.

BTW, if we are to move back some other stuff, I think it's worth considering to make it part of 'platform2'. I like what Antoine said in the following thread:

https://groups.google.com/a/chromium.org/d/msg/chromium-os-dev/1MWtJe9o-2w/ClBiOcSqagwJ
Yeah, I assumed that's what Dan meant -- even if development has moved back to Chrome OS, the point was more to move things back to platform2 and away from src/aosp. I believe this can be done today with firewalld at least.
+1 to moving shill out of AOSP and into platform2.  Same for apmanager.

Comment 8 by de...@chromium.org, Feb 14 2017

Cc: de...@chromium.org
Hi, I did a lot of the setup for the shared repos between CrOS and Android, which was discussed and agreed with the owners of the code at the time. Note that "move back to Chrome OS" in the current setup doesn't require any work to be done; you can land CLs in Chrome OS, using the CrOS CQ and get them in CrOS; nobody is blocking you from doing that just because there is a copy of the code in AOSP. The code compiled in pure Chrome OS is pulled from the chromium gerrit instance.

What you really mean by "move back to Chrome OS" is to merge the "src/aosp/*" repos to the single platform2 repo, which has pros and cons:

Some of the repos that moved to AOSP for Brillo are not used anymore in Brillo and can be merged back to platform2 with no cons, if that's what you want. I can help with some scripts to git merge these changes into a subdirectory of platform2 while keeping the history. The repos in src/aosp that are NOT used for Brillo are:

aosp/platform/external/dbus-binding-generator
aosp/platform/system/connectivity/apmanager
aosp/platform/system/connectivity/shill
aosp/platform/system/firewalld
aosp/platform/system/tpm
aosp/platform/system/webservd


The following repos are used in Android and Android Things:

aosp/platform/system/update_engine
aosp/platform/external/libbrillo
aosp/platform/external/libchrome
aosp/platform/external/modp_b64 (small libchrome dep with different license)
aosp/platform/external/minijail

Merging update_engine, libchrome and libbrillo in the Chrome OS side to platform2 is possible, but it makes it harder to share CLs between Android and ChromeOS because of the different paths. Renaming "libbrillo" and all the callsites of "brillo::" is a waste of time with no benefit.
For update_engine for example, senj@ and I have been working on new features and memory/performance improvements in AOSP that would benefit Chrome OS as well, and merging those changes to Chrome OS will be very simple once  bug 614164  is resolved. The other way around is also true, we can take CLs that landed in Chrome OS and merge them in AOSP. We are actually working with adlr@ and Amin on a new patching tool for update_engine for patching the ARC++ container which will also benefit Android devices; so keeping the projects in different repos makes this process simpler. There are ways to automatically sync AOSP and CrOS to branches so you can send a "git merge" CL with all the other code to the CQ and that's already set up for Chrome OS on these shared repos.

On the other hand, the benefits of having a giant platform2 repo are not that important nowadays that gerrit supports topics and the CQ supports CQ-DEPEND. That didn't exists The thread from 2013 linked in #5 is really outdated.


Finally, external/libmojo was created for ARC++ as a temporary solution, so never  used by Android or Brillo. See b/27041357 for details and check with the author if it is needed in AOSP at all.

I'm spinning the firewalld case into  issue 715621 .
Cc: jcivelli@chromium.org
external/libmojo was not created as a temporary solution. We'd like to keep that in one place because we do need to keep it in sync between CrOS and Android. We don't use it directly in AOSP, but that was the only way that it can reach the relevant Android branches easily.

external/libchrome is a dependency of external/libmojo, so we also want to keep them both more or less consistent. jcivelli@ is currently going through a cross-AOSP/CrOS uprev of libchrome, he can weigh in on this.
Cc: lhchavez@chromium.org
I don't have a strong opinion on this but if in the end external/libchrome is only required because it's a dependency of external/libmojo, maybe it should become part of external/libmojo? (so there is only 1 repo to uprev)

Comment 13 by de...@google.com, May 15 2017

external/libchrome and external/libbrillo are used by update_engine; which is part of Android Phones now (the A/B update thing on Android) and therefore used in AOSP. Not all of it, but the big basic parts are used.
Cc: vapier@chromium.org
i think i chatted with Ben on this (or maybe Jorge?) in general.

for the repos which are only used by CrOS (like shill), i think there is advantage to moving them back into platform2.  for projects that can't (like minijail & UE), the main dev work is happening on the AOSP side, so CrOS treating it as the upstream is fine.  i don't think we need to move them to GH or some other location ... AOSP is not a problem.

wrt the actual migration, i think we should for each project:
- revert the CL that deleted the code in platform2
- cherry pick in each CL that landed in AOSP (we can even chump these or git push since nothing uses the code)
- update the ebuilds to use platform2 again (and this would go through the CQ)
- drop the aosp mirrors from our manifest

i don't think we want to do what we did when we originally imported each project as the history would be duplicated and completely unrelated.

> external/libbrillo <-- should also be renamed back to "libchromeos"

i don't think that's worth the effort.  the chromeos->brillo rename was a bit hairy, and it's not clear that the "chromeos" name is even better.  especially (as noted by others), it's used by AOSP too.

> On the other hand, the benefits of having a giant platform2 repo are not that important nowadays that gerrit supports topics and the CQ supports CQ-DEPEND.

that wasn't the main driving factor with platform2.  the much larger concerns were about code sharing and the ease of doing changes across projects in a single repo (like libchrome updates).  topics/CQ-DEPEND don't address that.
Cc: hidehiko@chromium.org hashimoto@chromium.org
So, given that libbrillo is only used by update_engine on Android, and that the Android team does not seem to be very excited about upreving libbrillo, I think Chrome OS should move libbrillo back to CrOS repos (essentially forking it) and then libbrillo can be more easily updated.

This will require some care making sure CrOS's libbrillo still works with UE, but I feel that situation will be better than the current situation.
not sure what you mean by upreving wrt libbrillo ... it isn't like libbase/libchrome which is based on upstream Chromium's base/ tree.  libbrillo is where we host our own extensions.  if we want to add stuff, then add it ?
I meant Android is not actively developing libbrillo.
Which means there's no reason to leave it in AOSP.
Blockedon: 785137
Blockedon: 788211
Blockedon: 788227
Blockedon: 789863
Components: OS>Systems
Cc: semenzato@chromium.org
Blockedon: 795898
Cc: -lhchavez@chromium.org

Sign in to add a comment