New issue
Advanced search Search tips

Issue 913643 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: 5
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocked on: View detail
issue 904950
issue 914045

Blocking:
issue 904881
issue 904930
issue 917502


Show other hotlists

Hotlists containing this issue:
CrOSParallelCQ


Sign in to add a comment

Annealing recipe

Project Member Reported by la...@chromium.org, Dec 10

Issue description

Per https://docs.google.com/document/d/1RA3z4yYiYEBWU41FHI65hugyNbqQRvRlCrgR7jZtQl4/edit#heading=h.ue0wfrlvie9m, we will have an "annealing" builder that runs in a tight loop doing the following:

1. Checkout ToT
2. Rewind (i.e. checkout an ancestor) projects with missing dependencies; this prevents a bad tree state due to e.g. Gerrit replication latency.
3. Make a manifest snapshot (aka "revlocked manifest"), and push it
4. Perform a few post-submit tasks:
  * uprev portage packages
  * push metadata for e.g. Goldeneye, findit
 
My tentative plan is to push manifest snapshots to chromiumos/manifest.git/snapshot.xml (and the manifest-internal equivalent).

We can also push into e.g. chromiumos/manifest.git/lkgm.xml from green post-submit builders.

Developers will be able to opt-in to syncing to one of these via e.g. `repo sync -m lkgm.xml` at any time.

Anyone care to bikeshed these names?
Blocking: 904881
Blocking: 904930
Blockedon: 914045
Owner: athilenius@chromium.org
To get this started we'll just do steps 1 and 3.
Status: Started (was: Assigned)
- Doesn't step 4 need to happened before 3 (aka don't we want the uprev to land before creating a manifest snapshot?
- Are the annealing builders going to be a parent/child relationship, or are we using Chrome CQ to launch them? For the former, what are we using to pin the version from the parent so that children can checkout a pinned repo (was manifest-versions)?
> Doesn't step 4 need to happened before 3

Good call. Uprevs should be before 3. If we're pushing metadata that would still happen at the end.

> Are the annealing builders going to be a parent/child relationship, or are we using Chrome CQ to launch them?

Neither, I think. The annealing builder should be able to run as a single build that just runs continuously in a loop from luci-scheduler.
> Neither, I think. The annealing builder should be able to run as a single build that just runs continuously in a loop from luci-scheduler.

That will mean each builder (coral vs elm vs...) will checkout a slightly different 'version' of ToT. I think that's a problem because we are committing manifest snapshots at the end for single-line-of-CLs tracking, right?
Sorry got confused; the annealing builder is only one builder (equivalent to what the master-paladin does today, minus the child orchestration)
It'll be very, very similar to what a buildspec builder does. Looking at the uprev steps there, might be useful.

https://cs.corp.google.com/chromeos_public/chromite/cbuildbot/builders/workspace_builders.py?rcl=572100bbdaa15bb70d6d7ae4d5d3259fb9481d36&l=15

Note that uprevving is board specific. It can be done for all boards, but becomes slower.
Project Member

Comment 13 by bugdroid1@chromium.org, Dec 12

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/recipes/+/a8265585e4d1dd029e97075f6f2c671a7ac40ac6

commit a8265585e4d1dd029e97075f6f2c671a7ac40ac6
Author: Alec Thilenius <athilenius@google.com>
Date: Wed Dec 12 17:54:00 2018

Added the recpie stub for the annealing builder (which is the same as
the prototype recipe for now).

Bug: 913643
Change-Id: I1c138480993c0fd5bedc9662b868859974213079
Reviewed-on: https://chromium-review.googlesource.com/c/1374054
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>

[add] https://crrev.com/a8265585e4d1dd029e97075f6f2c671a7ac40ac6/recipes/annealing.py

Project Member

Comment 14 by bugdroid1@chromium.org, Dec 12

Labels: merge-merged-config
The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/50567fe6c664b8e6115b5a560264c472f7de8123

commit 50567fe6c664b8e6115b5a560264c472f7de8123
Author: Alec Thilenius <athilenius@google.com>
Date: Wed Dec 12 21:52:04 2018

Project Member

Comment 15 by bugdroid1@chromium.org, Dec 12

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/recipes/+/5446e5aca69a614553e930c14bf0d41f058f391a

commit 5446e5aca69a614553e930c14bf0d41f058f391a
Author: Alec Thilenius <athilenius@google.com>
Date: Wed Dec 12 22:41:11 2018

Fixed annealing recipe and trained tests

Annealing recipe was using the old config (before Lann refactored it),
fixed and retrained recipe tests.

Bug: 913643
Change-Id: I9675338720847e07edc72b7ca684b8ab21157b68
Reviewed-on: https://chromium-review.googlesource.com/c/1374061
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>

[add] https://crrev.com/5446e5aca69a614553e930c14bf0d41f058f391a/recipes/annealing.expected/basic.json
[modify] https://crrev.com/5446e5aca69a614553e930c14bf0d41f058f391a/recipes/annealing.py
[modify] https://crrev.com/5446e5aca69a614553e930c14bf0d41f058f391a/README.recipes.md

Blockedon: 904950
Project Member

Comment 17 by bugdroid1@chromium.org, Dec 13

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/recipes/+/6bfb6e09ea42da8614120b3d852788122cff0075

commit 6bfb6e09ea42da8614120b3d852788122cff0075
Author: Lann Martin <lannm@google.com>
Date: Thu Dec 13 17:45:14 2018

Add git and git_txn recipe modules

The git module is a simple wrapper around git commands.

The git_txn module implements "transactional" updates to remote git
refs.

Bug: 913643
Change-Id: I9c841e826853c99fb6130c83987f3d3d71ec97ae
Reviewed-on: https://chromium-review.googlesource.com/c/1374056
Tested-by: Lann Martin <lannm@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>
Commit-Queue: Lann Martin <lannm@chromium.org>

[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git/examples/full.py
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.expected/basic.json
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.expected/retry too many times.json
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.expected/retry succeed.json
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/api.py
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.py
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.expected/other failure.json
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/examples/full.expected/rejected no update.json
[modify] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/README.recipes.md
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git/examples/full.expected/basic.json
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git_txn/__init__.py
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git/__init__.py
[add] https://crrev.com/6bfb6e09ea42da8614120b3d852788122cff0075/recipe_modules/git/api.py

Project Member

Comment 18 by bugdroid1@chromium.org, Dec 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/0d4874d2f874f3ec59db6461012921c74ae92243

commit 0d4874d2f874f3ec59db6461012921c74ae92243
Author: Alec Thilenius <athilenius@google.com>
Date: Fri Dec 14 03:27:54 2018

Remove drop file output from git ignore (it's not longer generated)

BUG=913643
CQ-DEPEND=CL:1373992

Change-Id: I45c194e630546eff12c85b8af047bc207daf472a
Reviewed-on: https://chromium-review.googlesource.com/1373956
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/0d4874d2f874f3ec59db6461012921c74ae92243/.gitignore

Project Member

Comment 19 by bugdroid1@chromium.org, Dec 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/c325fff556b1d312bfadfafd5510c2df71eb06d4

commit c325fff556b1d312bfadfafd5510c2df71eb06d4
Author: Alec Thilenius <athilenius@google.com>
Date: Fri Dec 14 03:27:54 2018

Removed the unused drop_file from chromite/cbuildbot

The drop file was used to list all packages that were upreved by
cbuildbot. It does not appear to be used any more.

BUG=913643
TEST=./run_tests
CQ-DEPEND=CL:1373956

Change-Id: I853251ce698702ab3ee70d1a53df4725eb0f8416
Reviewed-on: https://chromium-review.googlesource.com/1373992
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/c325fff556b1d312bfadfafd5510c2df71eb06d4/cbuildbot/commands.py
[modify] https://crrev.com/c325fff556b1d312bfadfafd5510c2df71eb06d4/scripts/cros_mark_as_stable.py
[modify] https://crrev.com/c325fff556b1d312bfadfafd5510c2df71eb06d4/cbuildbot/commands_unittest.py

Project Member

Comment 20 by bugdroid1@chromium.org, Dec 18

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/recipes/+/1ff2499a19beb062863daf3daec88a67c650ebec

commit 1ff2499a19beb062863daf3daec88a67c650ebec
Author: Alec Thilenius <athilenius@google.com>
Date: Tue Dec 18 18:45:52 2018

Add a dev recipe module, add dryrun flag

Add a `dev` recipe module. This just for storing global flags for now.
Also adds a dryrun flag, as well as plumbing into modules that have
persistent effect.

Bug: 913643
Change-Id: Ia4857c9e31309e402af65a5dbbaa54b89d0d260b
Reviewed-on: https://chromium-review.googlesource.com/c/1377584
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>

[add] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/dev/examples/full.py
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/cros/api.py
[add] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/dev/__init__.py
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/cros/__init__.py
[add] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/dev/api.py
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/README.recipes.md
[add] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/dev/examples/full.expected/basic.json
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/git/__init__.py
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipes/prototype.py
[modify] https://crrev.com/1ff2499a19beb062863daf3daec88a67c650ebec/recipe_modules/git/api.py

Project Member

Comment 22 by bugdroid1@chromium.org, Dec 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/infra/recipes/+/e10d019fe0ba797917718a403ea3a59ceec968d9

commit e10d019fe0ba797917718a403ea3a59ceec968d9
Author: Alec Thilenius <athilenius@google.com>
Date: Thu Dec 20 17:28:09 2018

Add a launch_annealing.sh script to contrib

Copied the launch_prototype.sh script to launch_annealing.sh as the two
will diverge once Lann adds CL support to the prototype build.

Bug: 913643
Change-Id: I6d24a1aacf99c9a324ec0bf377597bb8fcfd8275
Reviewed-on: https://chromium-review.googlesource.com/c/1384864
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>

[add] https://crrev.com/e10d019fe0ba797917718a403ea3a59ceec968d9/contrib/launch_annealing.sh

Blocking: 917502
Project Member

Comment 25 by bugdroid1@chromium.org, Jan 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/f7f13a85db948f25b1c1a2d2d5b061d15669be4b

commit f7f13a85db948f25b1c1a2d2d5b061d15669be4b
Author: Mike Nichols <mikenichols@chromium.org>
Date: Fri Jan 11 17:27:03 2019

Revert "Removed the unused drop_file from chromite/cbuildbot"

This reverts commit c325fff556b1d312bfadfafd5510c2df71eb06d4.

Reason for revert: This is still being used in the release builds.  We should determine why that is but for now I'm going to revert to allow the builds to complete.  

Original change's description:
> Removed the unused drop_file from chromite/cbuildbot
> 
> The drop file was used to list all packages that were upreved by
> cbuildbot. It does not appear to be used any more.
> 
> BUG=913643
> TEST=./run_tests
> CQ-DEPEND=CL:1373956
> 
> Change-Id: I853251ce698702ab3ee70d1a53df4725eb0f8416
> Reviewed-on: https://chromium-review.googlesource.com/1373992
> Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
> Tested-by: Alec Thilenius <athilenius@google.com>
> Reviewed-by: Don Garrett <dgarrett@chromium.org>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>

Bug: 913643
Change-Id: Id901922143e52af5858dd4032b6d32731c878ef2
Reviewed-on: https://chromium-review.googlesource.com/c/1406910
Reviewed-by: Alec Thilenius <athilenius@google.com>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/f7f13a85db948f25b1c1a2d2d5b061d15669be4b/cbuildbot/commands.py
[modify] https://crrev.com/f7f13a85db948f25b1c1a2d2d5b061d15669be4b/scripts/cros_mark_as_stable.py
[modify] https://crrev.com/f7f13a85db948f25b1c1a2d2d5b061d15669be4b/cbuildbot/commands_unittest.py

Project Member

Comment 26 by bugdroid, Today (7 hours ago)

Sign in to add a comment