New issue
Advanced search Search tips
Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Feb 28
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

chromeos2-row3-rack1-host13 is down, and this is killing soraka-release

Project Member Reported by ejcaruso@chromium.org, Feb 27 Back to list

Issue description

sshing into chromeos2-row3-rack1-host13 times out, and it has failed dummy_PassServer.sanity twice. We've lost 2 runs in a row to this DUT (#920, #921) so I don't think this is just lab network flakiness anymore.

Example failure: https://luci-milo.appspot.com/buildbot/chromeos/soraka-release/921

Assigning to deputy.
 
locked the dut and filed b/73965859
Status: Fixed
re-balanced the pool
This issue appears to be more widespread than a few DUTs.

https://viceroy.corp.google.com/chromeos/provision?groups=Provision&groups=Auto+Update&breakdowns=dut_host_name&breakdowns=build&board=soraka&build_type=release&success=&devserver=chromeos%5B246%5D-devserver.*&dut=&topstreams=10&delta_window=4h&duration=8d&percentile=90&prior_alpha=0&prior_beta=0&refresh=-1&repository_behavior=DO_NOT_SKIP&type=Provision&use_precomputation=1&utc_end=1519423906

Note that due to the fact failures often take longer than successes, this will result in spikes in the failure percentage as the denominator essentially goes to zero.

If these provision failures started happening recently, I'd look for a product issue.

Sign in to add a comment