ben do you have the hammers for this?

Oh ya. We've got a cron ticket filer for dead devices, but it's only for the chrome-perf pool. I'll enable it for chrome-perf-pinpoint pool as well.

For now tho we need labs

Note that the phones failing with uid_low (build185-b4 and build225-b4) need a reflash. The global app counter on those are reaching the max and need to be reset

Quarantined: No available devices. [usb_failure]
https://chromium-swarm.appspot.com/bot?id=build15-b1--device3

The USB hub seemed to be flaking out. Noticed the connector on the host end was loose. Replaced the cable, all devices are seen now.

Bus 001 Device 113: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 112: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 111: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 110: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 114: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 108: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 107: ID 18d1:4ee7 Google Inc. 

Quarantined: get_state(): expected a dict, got None
https://chromium-swarm.appspot.com/bot?id=build73-b1--device2
https://chromium-swarm.appspot.com/bot?id=build75-b1--device2
These are no longer quarantined

Quarantined: No available devices. [uid_low]
https://chromium-swarm.appspot.com/bot?id=build185-b4
https://chromium-swarm.appspot.com/bot?id=build225-b4
Re-flashed all these as indicated #3 w/MOB30K-asop
Last Seen	4s ago
Last Seen	46s ago

Dead.
https://chromium-swarm.appspot.com/bot?id=build109-b4 (Android)
Rebooted, back in swarming

https://chromium-swarm.appspot.com/bot?id=build111-b4 (Android)
Rebooted, back in swarming

https://chromium-swarm.appspot.com/bot?id=build224-m4 (Mac)
hard powercycled: Back up
Last Seen	29s ago

🙏

Thank you!

 Issue 811843  has been merged into this issue.