New issue
Advanced search Search tips

Issue 888603 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 894201
issue 800146
issue 888560



Sign in to add a comment

MP: Desynchronize machine recycling

Project Member Reported by mar...@chromium.org, Sep 24

Issue description

When Machine Provider recycles the fleet on a period, it does it synchronously, which causes instantaneous downtime. The recycling should be staged/smeared so that at any point of time there's at least one bot alive (either the old image or the new image).

This would help with issue 800146 and  issue 888560 , and reduce instantaneous delays whenever this happens, which would make the CI smoother.
 
Cc: vadimsh@chromium.org estaab@chromium.org tandrii@chromium.org
We discussed this last week.

The AI is to add a fudge value so that VMs are recycled at variable duration* so that over time, the VMs recycling is smeared over time instead of all happening simultaneously.

* let's say +/-10%, but there's multiple algorithms to choose from
Blocking: 894201

Sign in to add a comment