New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 921051 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 16
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 0
Type: Bug



Sign in to add a comment

Build Failures on Uprev for rikku, sumo, winky, veyron-minnie, eve, terra

Project Member Reported by kbleicher@google.com, Jan 11

Issue description

Six boards failed with last night's Chrome OS M71 Build; all failed on  cros_mark_as_stable.

Sample Log:
https://luci-logdog.appspot.com/logs/chromeos/buildbucket/cr-buildbucket.appspot.com/8924653932975182192/+/steps/Uprev/0/stdout

Rebuilding currently, but need to identify the root cause / fix.

Same error / fail code for rikku, sumo, winky, veyron-minnie, eve, terra

Master Builder:
https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Prod/b8924654960627588720




 
Labels: -Pri-1 Pri-0
Build retries are failing with the same error.  Escalating to P0 since this is going to block six boards from being included in the M71 stable refresh (including eve).
Components: -Infra>Client>ChromeOS>Test Infra>Client>ChromeOS>CI
Owner: mikenichols@chromium.org
Status: Assigned (was: Untriaged)
Wrong queue.
--> CI bobby.
Initially it looks like the root cause is this CL:   https://chromium-review.git.corp.google.com/c/chromiumos/chromite/+/1373992

Excerpt from the log:
return code: 1; command: /b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_mark_as_stable commit --all '--boards=eve' '--drop_file=/b/swarming/w/ir/cache/cbuild/repository/src/scripts/cbuildbot_package.list' --buildroot /b/swarming/w/ir/cache/cbuild/repository --overlay-type both
cmd=['/b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_mark_as_stable', 'commit', '--all', u'--boards=eve', '--drop_file=/b/swarming/w/ir/cache/cbuild/repository/src/scripts/cbuildbot_package.list', '--buildroot', '/b/swarming/w/ir/cache/cbuild/repository', '--overlay-type', u'both'], cwd=/b/swarming/w/ir/cache/cbuild/repository
Running this command by hand on the bot does fail due to the use of the drop_file option.  

-- Mike
http://crrev.com/c/1406910 has been chumped to resolve the issue.

Sorry for the inconvenience.  Please retry the boards in question.  

-- Mike
Thanks, rebuilding winky since it was the first to fail with the re-try today

Status: Verified (was: Assigned)
Confirmed fixed with rebuild attempts
Labels: Merge-TBD
[Auto-generated comment by a script] We noticed that this issue is targeted for M-71; it appears the fix may have landed after branch point, meaning a merge might be required. The owner of this bug should confirm if a merge is required here. If so, add Merge-Request-71 label and indicate which commits/CLs are to be merged. Otherwise, remove Merge-TBD label. Thanks.

Comment 8 by kbleicher@google.com, Jan 16 (6 days ago)

Labels: -ReleaseBlock-Stable ReleaseBlock-Beta M-72
Status: Assigned (was: Verified)
Hey, I see this happening for grunt now in M72. Same Issue?

https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?id=3362576

23:03:36: ERROR: /b/swarming/w/ir/cache/cbuild/repository/chromite/bin/cros_mark_as_stable failed (code=1) 

Comment 9 by mikenichols@chromium.org, Jan 16 (6 days ago)

This does appear to be the same issue.  Some confusion regarding the branched  code as the changes were made in ToT and resolved the issue.  

Will look at cherry-picking the change back to 72 to resolve.  

-- Mike

Comment 10 by jclinton@chromium.org, Jan 16 (6 days ago)

Status: Fixed (was: Assigned)
Not the same issue. The error is:

23:02:53: ERROR: Package chromeos-kernel-experimental has a chromeos-version.sh script but it returned no valid version for "/b/swarming/w/ir/cache/cbuild/repository/src/third_party/kernel/experimental"

Please open a new bug and assign it to the Kernel team. Or maybe djmm@ already did since he reported a similar issue yesterday: https://bugs.chromium.org/p/chromium/issues/detail?id=921764#c8

Sign in to add a comment