New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 767666 link

Starred by 0 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

cbuildbot bootstrap manifest CL apply logic is off

Project Member Reported by vapier@chromium.org, Sep 22 2017

Issue description

looking at these two CLs:
  https://chrome-internal-review.googlesource.com/460012
  https://chromium-review.googlesource.com/676750

they have a CQ-DEPEND on each other (which is OK).  but when trying to run remote tryjobs, the Bootstrap stage failed.

when given a public manifest config (like chromiumos-sdk), we see:
Preconditions for the stage successfully met. Beginning to execute stage...
11:22:11: INFO: RunCommand: git clone https://chromium.googlesource.com/chromiumos/chromite /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/chromite --reference /b/c/cbuild/repository/chromite/.git in /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/chromite
11:22:35: INFO: RunCommand: chromite/bin/cbuildbot --reexec-api-version in /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI
11:22:36: INFO: RunCommand: git clone https://chromium.googlesource.com/chromiumos/manifest /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/manfest-checkout in /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/manfest-checkout
11:22:36: INFO: RunCommand: git fetch -f https://chromium-review.googlesource.com/chromiumos/manifest refs/changes/50/676750/2 in /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/manfest-checkout
11:22:36: INFO: Attempting to create transaction for rahulchaudhry:676750:4eb19b72
11:22:40: INFO: Transaction for rahulchaudhry:676750:4eb19b72 is rahulchaudhry:*460012:*b22f8b18, rahulchaudhry:676750:4eb19b72.
11:22:40: INFO: RunCommand: git fetch -f https://chrome-internal-review.googlesource.com/chromeos/manifest-internal refs/changes/12/460012/1 in /tmp/cbuildbot-tmp4biyGh/tmpKuFFgI/manfest-checkout
11:22:42: INFO: Attempting to cherry-pick change rahulchaudhry:*460012:*b22f8b18:"manifest-internal: sync binutils sources to tip of master branch on aosp."
Trying simple merge.
Simple merge failed, trying Automatic merge.
ERROR: external_full.xml: Not handling case 784937b1f9405f1cba2e3713608676fe1b1e6411 ->  -> 0213a2247d95b60721915f4cf26445631b2de533
fatal: merge program failed
error: could not apply b22f8b1... manifest-internal: sync binutils sources to tip of master branch on aosp.
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
11:22:42: INFO: Rewinding transaction: failed changes: rahulchaudhry:*460012:*b22f8b18:"manifest-internal: sync binutils sources to tip of master branch on aosp.", rahulchaudhry:676750:4eb19b72 .

when given an internal manifest config (like caroline-release), we see:
Preconditions for the stage successfully met. Beginning to execute stage...
11:24:42: INFO: RunCommand: git clone https://chromium.googlesource.com/chromiumos/chromite /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/chromite --reference /b/c/cbuild/repository/chromite/.git in /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/chromite
11:25:12: INFO: RunCommand: chromite/bin/cbuildbot --reexec-api-version in /tmp/cbuildbot-tmpHzweUP/tmpMNgulj
11:25:12: INFO: RunCommand: git clone https://chrome-internal-review.googlesource.com/chromeos/manifest-internal /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/manfest-checkout in /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/manfest-checkout
11:25:13: INFO: RunCommand: git fetch -f https://chrome-internal-review.googlesource.com/chromeos/manifest-internal refs/changes/12/460012/1 in /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/manfest-checkout
11:25:14: INFO: Attempting to create transaction for rahulchaudhry:*460012:*b22f8b18
11:25:14: INFO: Transaction for rahulchaudhry:*460012:*b22f8b18 is rahulchaudhry:676750:4eb19b72, rahulchaudhry:*460012:*b22f8b18.
11:25:14: INFO: RunCommand: git fetch -f https://chromium-review.googlesource.com/chromiumos/manifest refs/changes/50/676750/2 in /tmp/cbuildbot-tmpHzweUP/tmpMNgulj/manfest-checkout
11:25:15: INFO: Attempting to cherry-pick change rahulchaudhry:676750:4eb19b72:"manifest: sync binutils sources to tip of master branch on aosp."
Trying simple merge.
Simple merge failed, trying Automatic merge.
Auto-merging full.xml
ERROR: content conflict in full.xml
fatal: merge program failed
error: could not apply 4eb19b7... manifest: sync binutils sources to tip of master branch on aosp.
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
11:25:15: INFO: Rewinding transaction: failed changes: rahulchaudhry:676750:4eb19b72:"manifest: sync binutils sources to tip of master branch on aosp.", rahulchaudhry:*460012:*b22f8b18 .


in both cases we attempted to apply public & private manifest CLs to whatever manifest repo we happened to check out.  the chromiumos-sdk run failed because the private manifest CL doesn't apply at all the public manifest repo, and the caroline-release failed because we applied (successfully) the private manifest CL, but then tried to apply the public manifest CL on top of that which failed horribly.

for the Bootstrap phase, i think we need to be smarter about applying & skipping the relevant CLs.  it might mean that people get a false sense of success if they run an internal-only manifest change against a public bot (or vice versa).  i guess we could issue a buildbot warning and make the stage go orange ?
 

Comment 1 by vapier@chromium.org, Dec 15 2017

seems like it's even worse than this.  this CL:
  https://chromium-review.googlesource.com/815597

has a chain that does:
  chromiumos-overlay<->manifest<->manifest-internal

and bizarrely the bootstrap phase tries to apply all of those to the manifest repo.  i guess the filtering logic is off.
Project Member

Comment 2 by bugdroid1@chromium.org, Dec 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/db117c8bacdf5a3429ac291d57e47211e75b8fd6

commit db117c8bacdf5a3429ac291d57e47211e75b8fd6
Author: Mike Frysinger <vapier@chromium.org>
Date: Sat Dec 16 11:00:47 2017

cbuildbot_launch: respect --nosync

Have the --nosync option apply when using cbuildbot_launch and not
just cbuildbot.  This speeds up local builds.

BUG=chromium:767666
TEST=`cros tryjob --local --pass-through=--test-bootstrap --pass-through=--nosync ...` no longer syncs the full repo

Change-Id: I74914a3c377026e9a7d5b7b23e32f450efbc0f88
Reviewed-on: https://chromium-review.googlesource.com/831091
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/db117c8bacdf5a3429ac291d57e47211e75b8fd6/scripts/cbuildbot_launch.py

Project Member

Comment 3 by bugdroid1@chromium.org, Dec 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/94b3f09238904529e1698d55d069ae57c32846c8

commit 94b3f09238904529e1698d55d069ae57c32846c8
Author: Mike Frysinger <vapier@chromium.org>
Date: Sun Dec 17 04:40:17 2017

git: simplify FindCheckouts logic

The checkouts_by_name dict maps projects by their name,
so checking if the name field matches again is redundant.

BUG=chromium:767666
TEST=unittests & precq passes

Change-Id: Iae236da4566410f503e9e361575a4f24efeb2a0d
Reviewed-on: https://chromium-review.googlesource.com/830771
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/94b3f09238904529e1698d55d069ae57c32846c8/lib/git.py

Project Member

Comment 4 by bugdroid1@chromium.org, Dec 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/896319d152a19bf1234a7bf41c159e00eee922ce

commit 896319d152a19bf1234a7bf41c159e00eee922ce
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Dec 20 08:34:06 2017

patch: simplify local patch resolution

The lookup part of this function requires the checkouts list have
exactly one entry before the for loop, so the for loop itself ends
up being pointless, as does the need for the "ok" variable.  Throw
away the loop and use checkouts[0] directly.

BUG=chromium:767666
TEST=unittests & precq passes

Change-Id: I53d781c2e953d21a978e13fb2e937290e61d78d3
Reviewed-on: https://chromium-review.googlesource.com/831075
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/896319d152a19bf1234a7bf41c159e00eee922ce/lib/patch.py

Components: Infra>Client>ChromeOS>CI
Components: -Infra>Client>ChromeOS

Sign in to add a comment