implement a way to manually pause builds in a buildbucket bucket |
|||
Issue descriptionbuildbucket seems to be a good central place to have manual administrative controls over the load on buildbots. Given that we don't yet have a perfectly load-managed mechanism for not overloading buildbots, it seems like it would be good to have a way to "pause" buildbucket and tell it to stop scheduling builds in a given bucket. Implementing this feature probably should also include some way of exposing to users that scheduling builds is temporarily paused, but maybe that's not strictly necessary. Separately, we might want buildbucket to auto-throttle when loads are too high, but I'd like to track that as a separate request. There is more context and similar discussions on this problem in bug 657328 ; in particular, in that bug we talk about wanting quota mechanisms or rate-limiting particular users (particularly the CQ), but I don't want to do that as part of this bug. There is also bug 656862 for the generic concept of finer-grained quotas for either buildbucket or swarming. Assigning to estaab to further triage and/or find a better owner to do the work. +andybons for thoughts on how a paused buildbucket might be reflected in buildbucket (and/or sheriff-o-matic, trooper-o-matic, etc.).
,
Jan 26 2017
> Separately, we might want buildbucket to auto-throttle when loads are too high, but I'd like to track that as a separate request. there is a mechanism for this specific to buildbot in place, not used: https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/master/master_utils.py#529 this is a limit on number of builds buildbot can lease from buildbucket on schedule on its slaves. In this past this was set to the dynamic value of "number of currently connected slaves". The feature was disabled because users wanted to see their builds among pending builds in a builder, and using the feature builds appear there with a delay during peak times. Enabling the feature back is easy. Implementing real "pause a bucket" feature in buildbucket is also possible and seems easy: that would be an extra bit per bucket. When true, the "is there anything else to build" buildbucket query will return nothing. It won't work for swarmbucket right away ATM because swarmbucket creates tasks when a build is scheduled, but I was thinking to change swarmbucket so that it polls buildbucket and then schedules tasks, like buidlbot does. It would solve some other sawrmbucket problems too.
,
Jan 27 2017
in comment #2, nodir@ wrote: > Enabling the feature back is easy. That's good to know, but I agree that given the way the UI exposes things this is probably not what we want, so let's focus on "pause" instead.
,
May 9 2017
We can close this, yes?
,
May 9 2017
Yes |
|||
►
Sign in to add a comment |
|||
Comment 1 by estaab@chromium.org
, Jan 20 2017