New issue
Advanced search Search tips

Issue 625492 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Chromite waterfall board layout should be an explicit list.

Project Member Reported by d...@chromium.org, Jul 3 2016

Issue description

Currently, in Chromite, release board selection is handled the same way it is on ToT: it is inferred through a combination of "important=True" heuristics (https://chromium.googlesource.com/chromiumos/chromite/+/b2f195323154639da7f2c51569c7113512505934/cbuildbot/chromeos_config.py#21) and an explicit list (https://chromium.googlesource.com/chromiumos/chromite/+/b2f195323154639da7f2c51569c7113512505934/cbuildbot/chromeos_config.py#649).

This has worked for a while, but has been a point of confusion more than once, especially when people not super familiar with the configuration need to manually add/remove a board from it. I am thinking that the following change should be made:

1) Continue inferring all non-Release/Canary builders as-is.
2) Break the waterfall inclusion list by builder type rather than waterfall type (we can infer waterfall from the config parameters, I think).
3) Move experimental non-release builders to their respective builder type lists from their waterfall list.
4) Add *all* canary/release builders in that list. No waterfall inference will be made for them.
5) Add a PRESUBMIT check to assert that all "important=True" builders are on a waterfall.

(4), in particular, will make it so that the transition from ToT to branched config is as simple as setting IS_RELEASE_BRANCH to True and pruning undesired ToT items from that list. (5) will make it safe.

This should reduce the complexity of adding new builders. The additional burden of ensuring that the builder is in the list on ToT (enforced by (5)) should, in practice, be a non-issue, since currently one would have added to the list as experimental anyway.

WDYT?
 
I like the idea a lot. This has been on my list of want to haves for a while.

Though... to do this all the way right all of the parts should respect the single source of truth here.

That means that cbuildbot's sync code should be changed, our 'waiting for slaves' code should be changed, and the waterfall's "which slaves to fire off based on a new manifest-versions change" logic should also use this.

I think I looked at that and decided the cost of implementation was too high for a background cleanup project. Maybe I was letting "perfect" be the enemy of "good".

Labels: okr

Comment 3 by d...@chromium.org, Jul 7 2016

RE #1, I'm not convinced that this needs to be so thorough. I think using cbuildbot's inner config structures to define *how* things are built is still really useful. I also think we can leave that largely alone and focus specifically on simplifying waterfall layout. Adding a test that fails if any important=True boards aren't in the layout would close the loop.
Owner: dgarr...@chromium.org
this is either the same as what Don is now working on, or will be obsoleted by what he is working on

Comment 5 by autumn@chromium.org, Sep 27 2016

Labels: -OKR
Owner: d...@chromium.org
This was done in the following CL.

In config_dump.json, each master config now has an explicit list of slaves. This is not yet true for release builders, but will be in a few months.

If it's useful, we could also add that information into waterfall_layout_dump.txt, after we come up with the right format.

https://chromium-review.googlesource.com/c/457127/
Status: Assigned (was: Untriaged)
This bug is Untriaged and very old.  Because it has an owner, the status will be set to assigned to avoid closing a bug someone is using.  If this bug still needs triage, change it back to Untriaged.
Components: Infra>Client>ChromeOS>CI
Components: -Infra>Client>ChromeOS

Sign in to add a comment