New issue
Advanced search Search tips

Issue 923569 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Chrome
Pri: 3
Type: Bug

Blocking:
issue 922840



Sign in to add a comment

signer: skip lock refreshing if last refresh was "recent"

Project Member Reported by vapier@chromium.org, Jan 18 (4 days ago)

Issue description

as seen in issue 922840 (only the latest occurrence), our infra sometimes run into flakes in GS/gsutil, particularly related to locks/generation handling.  b/112110347 has more details too.

we could mitigate this on our side by refreshing the locks less often.  we have to refresh the locks so that we don't (incorrectly) break the lock because a particular job is taking longer than normal.  but we should be able to add some logic to the core refresh func like:
- if last refresh was <1 min ago, return immediately
- otherwise, refresh the lock, and update the "last refresh" time

the signers currently set the lock timeout to 60 min.  so using 1min (or maybe even 5min?) shouldn't be too bad as that's still <10% of the time for a timeout to be hit.
 

Sign in to add a comment