New issue
Advanced search Search tips

Issue 682071 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Rearrange ChromeOS Buildroot

Project Member Reported by dgarr...@chromium.org, Jan 18 2017

Issue description

Currently, ChromeOS builders overlap a lot of stuff into the source code directory.

The current directory tree looks like this:

buildroot/
  .repo
  <all source checkouts>
  manifest-versions/
  manifest-versions-internal/
  chroot/
  .cache/

It seems a lot cleaner to instead organize things on builders like:

buildroot/
  repo/
    .repo/
    <all source checkouts>
  manifest-versions/
  manifest-versions-internal/
  chroot/
  cache/

 
I'm trying to decide how much effort solving this would take.

It could never be solved on existing branches, which means maintaining support for two buildroot layouts. This is more easily done, given the bootstrap script.

There is also a transition problem, again reasonable to solve via the bootstrap script.

Moving manifest-versions paths is well isolated, and easy to fix.

Moving the cache, and the chroot is what strikes me as scary since they are both accessed from a wide variety of standalone scripts with complex call paths.

Comment 2 by dshi@chromium.org, Jan 23 2017

Labels: -current-issue
Owner: dgarr...@chromium.org
dgarrett, seems you know the most of what to do here.
Status: Available (was: Untriaged)
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 18 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/6096792a75bc85a641e331494c209ea21eac846c

commit 6096792a75bc85a641e331494c209ea21eac846c
Author: Don Garrett <dgarrett@google.com>
Date: Tue Apr 18 05:50:25 2017

cbuildbot_launch: Add supported method to clean buildroots.

Add a cleaner method for invalidating of buildroot directories, if
they are not in a known layout. This would have given a cleaner
solution for chromium:710900, and will help with chromium:682071.

BUG= chromium:682071 
TEST=cbuildbot_launch_unittest

Change-Id: Iece17ce90bb7808857a94c3ee2f0b70954230e68
Reviewed-on: https://chromium-review.googlesource.com/476156
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/6096792a75bc85a641e331494c209ea21eac846c/scripts/cbuildbot_launch.py
[modify] https://crrev.com/6096792a75bc85a641e331494c209ea21eac846c/scripts/cbuildbot_launch_unittest.py

Status: Fixed (was: Available)
Status: Assigned (was: Fixed)
Project Member

Comment 7 by bugdroid1@chromium.org, May 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/783b257a6ee52e9db95fe32e359271afec6d09bb

commit 783b257a6ee52e9db95fe32e359271afec6d09bb
Author: Don Garrett <dgarrett@google.com>
Date: Wed May 31 09:15:29 2017

cbuildbot_launch: Move cbuildbot checkout to subdirectory.

Move the repo checkout to a subdirectory of cbuildbot_launches working
directory. This allows us to break apart various directories (such as
the cache) apart from the source control managed directories. In
particular, this allows the buildroot state value to be preserved even
when we wipe out the repo checkout.

BUG= chromium:682071 
TEST=Unittests + local build/test script.

Change-Id: I84d868703888deae5ae4c8f698c8ffb580d529a1
Reviewed-on: https://chromium-review.googlesource.com/510019
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/783b257a6ee52e9db95fe32e359271afec6d09bb/scripts/cbuildbot_launch.py
[modify] https://crrev.com/783b257a6ee52e9db95fe32e359271afec6d09bb/scripts/cbuildbot_launch_unittest.py

Project Member

Comment 8 by bugdroid1@chromium.org, May 31 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/30e5aaca227b75a7968809559bd4f0e2112a576c

commit 30e5aaca227b75a7968809559bd4f0e2112a576c
Author: Ben Zhang <benzh@chromium.org>
Date: Wed May 31 15:03:23 2017

Revert "cbuildbot_launch: Move cbuildbot checkout to subdirectory."

This reverts commit 783b257a6ee52e9db95fe32e359271afec6d09bb.

Reason for revert: seems to break many builders, causing chromite/cbuildbot/repository.py to complain /b/cbuild/repository is not the root of a repository.

Original change's description:
> cbuildbot_launch: Move cbuildbot checkout to subdirectory.
> 
> Move the repo checkout to a subdirectory of cbuildbot_launches working
> directory. This allows us to break apart various directories (such as
> the cache) apart from the source control managed directories. In
> particular, this allows the buildroot state value to be preserved even
> when we wipe out the repo checkout.
> 
> BUG= chromium:682071 
> TEST=Unittests + local build/test script.
> 
> Change-Id: I84d868703888deae5ae4c8f698c8ffb580d529a1
> Reviewed-on: https://chromium-review.googlesource.com/510019
> Commit-Ready: Don Garrett <dgarrett@chromium.org>
> Tested-by: Don Garrett <dgarrett@chromium.org>
> Reviewed-by: Don Garrett <dgarrett@chromium.org>
> 

BUG= chromium:682071 

Change-Id: I04fb798242418e0a417fe7cf82962cdd11524f41
Reviewed-on: https://chromium-review.googlesource.com/519364
Reviewed-by: Ben Zhang <benzh@chromium.org>
Commit-Queue: Ben Zhang <benzh@chromium.org>
Tested-by: Ben Zhang <benzh@chromium.org>

[modify] https://crrev.com/30e5aaca227b75a7968809559bd4f0e2112a576c/scripts/cbuildbot_launch.py
[modify] https://crrev.com/30e5aaca227b75a7968809559bd4f0e2112a576c/scripts/cbuildbot_launch_unittest.py

Project Member

Comment 9 by bugdroid1@chromium.org, Jun 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/bf90cdf55e3acd70243fce6875ef9fc910a4f527

commit bf90cdf55e3acd70243fce6875ef9fc910a4f527
Author: Don Garrett <dgarrett@google.com>
Date: Tue Jun 13 23:11:54 2017

cbuildbot_launch: Move cbuildbot checkout to subdirectory.

Move the repo checkout to a subdirectory of cbuildbot_launches working
directory. This allows us to break apart various directories (such as
the cache) apart from the source control managed directories. In
particular, this allows the buildroot state value to be preserved even
when we wipe out the repo checkout.

This is relanding CL:510019, which was reverted. Rev 1 is the initial
CL, and Rev 2 includes a fix discovered during testing. Also there was
an intermediate CL tweaking repository.py.

Also includes function renames to be slightly clearer.

BUG= chromium:682071 
TEST=Created a buildroot with an earlier version of cbuildbot_launch.
     Ran builds against this buildroot, to prove we could update.

Change-Id: Iaeac65fa01aa8e7ab7a96fa6c861b15a8d60f138
Reviewed-on: https://chromium-review.googlesource.com/520225
Tested-by: Don Garrett <dgarrett@chromium.org>
Trybot-Ready: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/bf90cdf55e3acd70243fce6875ef9fc910a4f527/scripts/cbuildbot_launch.py
[modify] https://crrev.com/bf90cdf55e3acd70243fce6875ef9fc910a4f527/scripts/cbuildbot_launch_unittest.py

Status: Archived (was: Assigned)

Sign in to add a comment