New issue
Advanced search Search tips

Issue 730093 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug



Sign in to add a comment

6 devices down on build9-b1

Project Member Reported by martiniss@chromium.org, Jun 6 2017

Issue description

6 out of the 7 devices on build9-b1 are down. The seventh device is still running for some reason.

I don't know what's causing this really, but I looked on the bot, and saw this in the docker log file:

170606 11:00:01.813 DEBUG Host uptime over max uptime (1482 > 1440)
170606 11:00:01.814 DEBUG Still 1 containers running. Shutting them down first.
170606 11:00:01.814 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:00:01.815 DEBUG "POST /v1.24/exec/f58a8b9608f5b473cc1afd7643a94f67374bd6211e31592df54efb8c7c0df360/start HTTP/1.1" 101 0
170606 11:00:01.889 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:00:01.890 DEBUG "POST /v1.24/exec/a2584ce4da0e2e0a0d697be8a93a595e3521274fcda423555ef6980dac05ba76/start HTTP/1.1" 101 0
170606 11:00:01.907 INFO Sent SIGTERM to swarming bot of android_070b074f.
170606 11:05:01.672 DEBUG Killing any host-side ADB processes.
170606 11:05:01.675 DEBUG Running launch on devices: all
170606 11:05:01.675 DEBUG Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
170606 11:05:01.675 DEBUG No config file found
170606 11:05:01.679 DEBUG "GET /v1.24/_ping HTTP/1.1" 200 2
170606 11:05:01.921 DEBUG "GET /v1.24/containers/json?all=0&limit=-1&trunc_cmd=0&size=0 HTTP/1.1" 200 None
170606 11:05:01.922 DEBUG "GET /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/json HTTP/1.1" 200 None
170606 11:05:01.923 DEBUG Host uptime over max uptime (1487 > 1440)
170606 11:05:01.923 DEBUG Still 1 containers running. Shutting them down first.
170606 11:05:01.924 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:05:01.925 DEBUG "POST /v1.24/exec/73a7c2c583a3496d5a27c2303275c061861cd3be813f4a70c54a808259465a27/start HTTP/1.1" 101 0
170606 11:05:01.998 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:05:01.999 DEBUG "POST /v1.24/exec/c29633ae7471092d52ef581fcdd2da6b2aec63b8ed9f83147171c853f49f1f7e/start HTTP/1.1" 101 0
170606 11:05:02.016 INFO Sent SIGTERM to swarming bot of android_070b074f.
170606 11:10:01.317 DEBUG Killing any host-side ADB processes.
170606 11:10:01.320 DEBUG Running launch on devices: all
170606 11:10:01.320 DEBUG Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
170606 11:10:01.320 DEBUG No config file found
170606 11:10:01.324 DEBUG "GET /v1.24/_ping HTTP/1.1" 200 2
170606 11:10:01.567 DEBUG "GET /v1.24/containers/json?all=0&limit=-1&trunc_cmd=0&size=0 HTTP/1.1" 200 None
170606 11:10:01.568 DEBUG "GET /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/json HTTP/1.1" 200 None
170606 11:10:01.569 DEBUG Host uptime over max uptime (1492 > 1440)
170606 11:10:01.569 DEBUG Still 1 containers running. Shutting them down first.
170606 11:10:01.570 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:10:01.571 DEBUG "POST /v1.24/exec/d5d1ec8985a42b34ca32356ce9bdac40d6a25a2e039f9773368d42139d558e3e/start HTTP/1.1" 101 0
170606 11:10:01.644 DEBUG "POST /v1.24/containers/5792923a89b7cd16a9691d63f566f25b2d5752944306b14c5f4316a199be187a/exec HTTP/1.1" 201 74
170606 11:10:01.645 DEBUG "POST /v1.24/exec/6b2efb316d3872036c1f32ca098d6055849bad5778e0b11c9018e8dd45cc2c31/start HTTP/1.1" 101 0
170606 11:10:01.662 INFO Sent SIGTERM to swarming bot of android_070b074f.

It looks like one container is stuck, and can't be shut down. I don't really know much about docker and this whole set up though, so I'm scared to touch it too much.

cc-ing some people who might know about the docker setup on swarming.

This is breaking the Android Nexus 7v2 Perf bot on the chromium.perf waterfall.
 
Status: WontFix (was: Available)
wai

see my comment at https://bugs.chromium.org/p/chromium/issues/detail?id=726874#c5

Sign in to add a comment