New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 716639 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

cbuildbot_launch: Intial sync merges existing changes.

Project Member Reported by dgarr...@chromium.org, Apr 28 2017

Issue description

The initial sync in cbuildbot_launch may not produce "clean" checkouts in all cases.

In this build the initial checkout is much slower than expected (8m instead of 1.5m).

https://uberchromegw.corp.google.com/i/chromeos/builders/butterfly-paladin/builds/22185

Checking the logs, its attempting to do rebasing of local changes:

https://luci-logdog.appspot.com/v/?s=chromeos%2Fbb%2Fchromeos%2Fbutterfly-paladin%2F22185%2F%2B%2Frecipes%2Fsteps%2FInitialCheckout%2F0%2Fstdout

project src/platform2/
First, rewinding head to replay your work on top of it...
Applying: metrics: Add vmlog logging to metrics_daemon.

project src/third_party/chromiumos-overlay/
First, rewinding head to replay your work on top of it...
Applying: Marking 9999 ebuild for brillo-base/libbrillobinder as stable.
.git/rebase-apply/patch:1459: new blank line at EOF.
+
.git/rebase-apply/patch:1918: new blank line at EOF.
+
.git/rebase-apply/patch:4489: new blank line at EOF.
+
.git/rebase-apply/patch:9011: new blank line at EOF.
+
warning: 4 lines add whitespace errors.
Using index info to reconstruct a base tree...


When there are rebasing errors, it wipes local checkouts and refetches them.

Auto-merging brillo-base/libbrillobinder/libbrillobinder-0.0.1-r1352.ebuild
Patch failed at 0001 Marking 9999 ebuild for brillo-base/libbrillobinder as stable.
The copy of the patch that failed is found in: .git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".


real	0m23.670s
09:43:44: INFO: RunCommand: sudo -- rm -rf chrome_build chrome_release chromeos-admin chromite chromium cros-signing crostools docs infra infra_virtualenv manifest manifest-internal src in /b/cbuild/repository
09:45:03: INFO: RunCommand: repo --time sync --force-sync '--cache-dir=/b/cros_git_cache' -l in /b/cbuild/repository
Checking out files:  20% (36/173)   

 
Cc: nxia@chromium.org
The sync commands currently being used are:

repo --time sync --force-sync '--cache-dir=/b/cros_git_cache' -n
repo --time sync --force-sync '--cache-dir=/b/cros_git_cache' -l

Comment 2 by nxia@chromium.org, Apr 28 2017

I guess our builds clean up the git repos before executing syncs, see the clean up stage:

08:41:51: INFO: Cleaning up buildroot git repositories.
Interesting.

I'm a bit surprised that we don't do detached checkouts, which I *think* would just fix this.

Comment 4 by aut...@google.com, May 5 2017

Labels: -current-issue
Status: Started (was: Untriaged)
Project Member

Comment 6 by bugdroid1@chromium.org, May 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/7649691aa907dcf7aa6ae2a2200b6cd24abe4d7b

commit 7649691aa907dcf7aa6ae2a2200b6cd24abe4d7b
Author: Don Garrett <dgarrett@google.com>
Date: Sat May 13 03:58:00 2017

cbuildbot_launch: Cleanup repository before Syncing.

Currently cbuildbot_launch's initial sync ends up rebasing on top of
whatever happens to be in the buildroot before it starts up. That means that

 A) The intial codebase is corrupt (could have arbitrary CLs in it).
 B) If rebase fails, we recover in a very expensive way, timewise.

BUG= chromium:716639 
TEST=run_tests + local launch builds.

Change-Id: I0b97e3b6d1f8ce9d3a9963d0f5900bb8982f7ef7
Reviewed-on: https://chromium-review.googlesource.com/503483
Commit-Ready: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>

[modify] https://crrev.com/7649691aa907dcf7aa6ae2a2200b6cd24abe4d7b/scripts/cbuildbot_launch.py
[modify] https://crrev.com/7649691aa907dcf7aa6ae2a2200b6cd24abe4d7b/scripts/cbuildbot_launch_unittest.py

Status: Fixed (was: Started)

Comment 8 by dchan@chromium.org, Jan 22 2018

Status: Archived (was: Fixed)

Sign in to add a comment