New issue
Advanced search Search tips

Issue 665130 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Dec 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Why are git timeouts so prevalent on bots but not locally?

Project Member Reported by pdr@chromium.org, Nov 14 2016

Issue description

During sheriffing I've seen quite a few network-related timeout flakiness (e.g.,  https://crbug.com/664678 ), but I have not had git timeout locally in the previous year of working on Chrome. Is there a reason timeouts occur so much more on the bots?

Ideas: Could real failures be hiding behind timeout failures? Could there be real network contention in the datacenter?
 
Components: -Infra Infra>Platform
Status: Available (was: Untriaged)
There are a number of possibilities:
 - There might be some obscure quota issue - all bots use the same quota pool (though quota limits should normally terminate connection very fast)
 - There might be network problems, especially for physical machines (GCE bots shouldn't have this problem - may need cross-checking if we see timeouts on GCE)
 - Something else I can't think of right now?

Local timeouts might not be observed as much because of different network / quota pool / small scale. The scale in particular is important: if timeouts happen once in 10K times, we'll see plenty of them on bots, but hardly any locally. Though I recently saw maybe 1/5 timeout rate on gnumbd - issue 663052.

Adding Infra>Platform component for more input.
Project Member

Comment 2 by sheriffbot@chromium.org, Nov 20 2017

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: WontFix (was: Untriaged)
We have monitoring in place now to tell us when this is particularly bad. I'm going to close this since besides some recent windows slowness that we've fixed and some current limitations with the git protocol that the git server team is addressing things have been steady.

Sign in to add a comment