Issue metadata
Sign in to add a comment
|
Allocate GCE instances for try servers |
||||||||||||||||||||||||
Issue descriptionFollowing up on a conversation with dgarret@ regarding current limitations on try servers and possibility of allocating GCE instances for these too. Background: =========== As part of Chrome OS releases, there are times when we need to patch a build with release blocker fixes/reverts and then re-kick new builds from a mini branch (ref: https://sites.google.com/a/google.com/chromeos/for-team-members/chronos-download/pmo/cros-releasetasks/stabilize) For branches we can use existing branch builders but for ToT we have to use try servers. The number of build slaves on release tryservers (https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/release) is not enough to kick off the number of board/targets we need hence it takes days to build them all plus causing a lot of disruption to anyone else using try servers The end result is that we are no longer able to patch ToT builds for Dev releases causing not having Dev releases on regular basis. In other words, we would only push dev when no blocking issues on nightly builds. Request ======= Allocate more builders to try servers by provisioning new GCE instances for these Alternatively, we could explore re using some of the branch/canary builders but this may require more work/logic
,
Jul 15 2016
You should be able to use the release waterfall for this. Just manually enter "master" for ToT into the master builder branch?
,
Jul 18 2016
Do you mean one of the release branch builders? e.g M53 here: https://uberchromegw.corp.google.com/i/chromeos_release/builders/master-release%20release-R53-8530.B Note that that will miss some targets that are not in the branch e.g. new boards
,
Jul 18 2016
Yes. Missing targets is obviously a problem, and the degree to which that's a problem will determine the best course of action. If it's reasonable to use release for 90% of the targets and tryserver the missing boards, doubling the tryserver capacity is probably overkill. If possible, utilizing the release waterfall instead of upping tryserver capacity is a much better option, since most of the release builders are already board-dedicated and heavily underutilized.
,
Sep 27 2016
,
Sep 28 2016
Was there a change in the master logic when using release waterfall? It used to work few weeks back but it seems like Master is not kicking off slaves when entering an stabilize branch from ToT anymore ..
,
Oct 12 2016
Do you have an example of this? +nxia as she's been working in this area too
,
Oct 25 2016
nxia, any comment? Or Don is the better owner?
,
Oct 25 2016
There are two questions. 1) Why aren't slaves kicking off? I think I know the answer, but Ningning is the better owner. 2) Creating a pile of new trybot slaves. If we are certain we want to do that, I'm the better owner (unless someone else wants to learn how).
,
Oct 25 2016
I don't know why slaves were not kicked off. Found an example and looks like slaves were triggered https://uberchromegw.corp.google.com/i/chromiumos.release/builders/master-release%20release-R53-8530.B/builds/90
,
Oct 25 2016
pass to josafat@ to confirm if slaves could be triggered by the master build. It looks to me slaves were successfully triggered in the example pasted above.
,
Nov 2 2016
Slaves are kicked off properly when using main release branch but those are not when using stabilize branch Common scenario below: a) Canary build is chosen as Dev release candidate b) Testing finds a Dev release blocker c) TPM creates an stabilize branch from ToT release candidate d) Dev blocker fix is merged into the stabilize branch e) New build (for all boards) needed from stabilize branch f) Release branch builders are used to kick off new build specifying the stabilize branch in the Master e) Slaves are not triggered by Master ps. I know that there may be some targets that are not in the release branch builders and they won't get build but majority should be covered with all other slaves
,
Nov 15 2016
dgarrett@ is working on a pre-bootstrap script. I guess that will solve this branch issue? dgarrett@, can you confirm?
,
Nov 15 2016
It will help solve some issues around this, but I think the key issue is how builds are launched. I think this requires switching the branch builders to buildbucket.
,
Nov 15 2016
Release branches rely on git-commit-mechanism to trigger slaves. buildbucket shouldn't be required to support this?
,
Nov 15 2016
I THINK the git-commit-mechanism will be branch specific in this case, and that's why slaves aren't triggering if the branch is forced on the master. Do we know exactly what the manifest-versions commits look like for both the normal branch release builds, and the ones with a forced branch on the master? comparing the two would confirm or deny my understanding.
,
Dec 27 2016
,
Dec 27 2016
we can allocate more gces when the nesting feature is released by the GCE team.
,
Mar 11 2017
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by dgarr...@chromium.org
, Jul 15 2016