New issue
Advanced search Search tips

Issue 864886 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 1
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 827092



Sign in to add a comment

chromeos sdk doesn't work with repo reference to mirror

Project Member Reported by kcwu@chromium.org, Jul 18

Issue description

"repo init --reference" can reuse existing repo tree. But chromeos sdk doesn't work with reference to mirror.

A) This is good:
  cd $tree1
  repo init ...skip...
  repo sync
  cd $tree2
  repo init ...skip... --reference=$tree1
  repo sync
  cros_sdk

B) this doesn't work:
  cd $tree1
  repo init ...skip... --mirror
  repo sync
  cd $tree2
  repo init ...skip... --reference=$tree1
  repo sync
  cros_sdk

When entering cros_sdk, the sdk will modify git's alternates information in order to make the git reference work both inside and outside chroot. However, the logic only support case A, not case B.

Detail analysis:
The underlying magic of "repo reference" is git's .git/objects/info/alternates. Take chromite as an example, 

A) good case:
After repo init and sync, the content of chromite/.git/objects/info/alternates is "/path/to/tree1/.repo/projects/chromite.git/objects", so git knows where to search git objects in additional to current folder.

In order to make repo reference work for both inside and outside chroot, the sdk will modify chromite's git alternates to somewhere inside .repo/alternates and bind mount .repo/alternates to .repo/chroot/alternates inside chroot. The final result of chromite's alternates is
/path/to/tree1/.repo/project-objects/chromiumos/chromite.git/objects (outside chroot)
/mnt/host/source/.repo/chroot/external1/.repo/project-objects/chromiumos/chromite.git/objects (inside chroot)
(sdk will bind mount $tree1 into .repo/chroot/external1 to make it visible inside chroot)

(some detail skipped since this is good case)

B) bad case:
After repo init and sync, the content of chromite/.git/objects/info/alternates is "/path/to/tree1/chromiumos/chromite.git/objects". This is correct because the tree layout of mirror is different to case A.

When entering chroot, it will execute 
    ./chromite/lib/rewrite_git_alternates.py $tree2 $tree1 /mnt/host/source 
In rewrite_git_alternates.py, _UpdateAlternatesDir()
68        suffix = os.path.join('.repo', 'project-objects', project, 'objects')
69        if os.path.exists(os.path.join(k, suffix)):
70          paths.append(os.path.join(v, suffix))

It searches git objdir in .repo/project-objects/$project/objects. However, ".repo/project-objects" is only created for normal repo checkout, not for repo mirror. 
Refers to https://chromium.googlesource.com/external/repo.git/+/master/manifest_xml.py#845
    if self.IsMirror:       
      ...
      gitdir = os.path.join(self.topdir, '%s.git' % name)                                                                                                                                                                                                                                                                 
      objdir = gitdir
    else:
      ...
      objdir = os.path.join(self.repodir, 'project-objects', '%s.git' % name)


==============
Given this issue existing several years, nobody use "repo reference to mirror" feature. This issue is worth to be fixed
 1. This benefits whom want to repo sync several chromeos tree, especially different milestones, versions, etc. (i.e. bisect)
 2. bisect-kit's chromeos bisector already uses "repo mirror" (to analyze full git history), so it's natural step for bisect-kit to use "reference to mirror". This issue need to be fixed or workaround.
 
More info: for bad case, chromite/.git/objects/info/alternates becomes empty ("\n") after rewrite_git_alternates.py ran. Then all git operations break.

Labels: -Restrict-View-Google
please don't use RVG unless absolutely necessary
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3afe77217e0161c264b890c4e48fb81d8714e308

commit 3afe77217e0161c264b890c4e48fb81d8714e308
Author: Kuang-che Wu <kcwu@chromium.org>
Date: Fri Jul 20 12:31:26 2018

rewrite_git_alternates: support repo reference to mirror

Currently, the existing code only supports repo reference to checked-out
tree. If repo referenced to mirror, rewrite_git_alternates.py is unable
to find git object dir in correct path, .git/objects/info/alternates is
corrupted, and git operations are totally broken.

BUG= chromium:864886 
TEST='repo init --mirror' and repo sync to setup a mirror of chromeos tree;
     'repo init --reference' to the said mirrror; after entering chroot once,
     'git log' still works both inside and outside chroot

Change-Id: I0b2ee6f25f2089c0cab4a1aeb05bca418327b687
Reviewed-on: https://chromium-review.googlesource.com/1141560
Commit-Ready: Kuang-che Wu <kcwu@chromium.org>
Tested-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/3afe77217e0161c264b890c4e48fb81d8714e308/lib/rewrite_git_alternates.py

Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>Build
Status: Assigned (was: Untriaged)
This bug has an owner, thus, it's been triaged. Changing status to "assigned".
Status: Fixed (was: Assigned)
1. chromite after Jul 20 2018 is fixed by CL mentioned in comment 4
2. Probably nobody besides me will use bisect-kit to bisect regressions before Jul 20, so I think it's unnecessary to commit workaround script into bisect-kit.
So there nothing remaining to do, let's close this bug.

For anyone is interested, I attached my workaround script here.

workaround-crbug864886.py
662 bytes View Download

Sign in to add a comment