Shards are timing out on Mac10.11 Tests, Mac10.12 Tests and Linux Tests (dbg) |
||||
Issue description--- Mac10.11 Tests --- https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.11%20Tests/25600 Purple colored box: browser_tests on Mac-10.11 Run on OS: 'Mac-10.11' Max pending time: 0:01:02.471440 (shard #9) Max shard duration: 34s (shard #7) ... some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ... --- Mac10.12 Tests --- https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.12%20Tests/12295 Purple colored box: browser_tests on Mac-10.12 Run on OS: 'Mac-10.12' Max pending time: 28s (shard #9) Max shard duration: 30s (shard #4) ... some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ... --- Linux Tests (dbg) --- https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Tests%20%28dbg%29%281%29/71625 Red colored box: telemetry_perf_unittests Run on OS: 'Ubuntu-14.04' Max shard duration: 0:16:34.240000 (shard #8) Total tests: 236 * Passed: 212 (212 expected, 0 unexpected) * Skipped: 24 (24 expected, 0 unexpected) * Failed: 0 (0 expected, 0 unexpected) * Flaky: 0 (0 expected, 0 unexpected) ... shard #8 timed out, took too much time to complete ...
,
Apr 25 2018
I recently landed <https://chromium-review.googlesource.com/c/chromium/src/+/1025638> for Mac - the timeouts could (?) be related to issue 828031 overall, but I'm not sure if/how that CL could have introduced bot timeouts.
,
Apr 25 2018
The Mac failures are caused by crbug.com/828031 . I'll look into the Linux red.
,
Apr 25 2018
we should be looking at why those shards are getting picked up as timeouts and not failures. swarming certainly doesn't seem to think that they're timeouts, e.g. https://chromium-swarm.appspot.com/task?id=3d152f8e997ea710&refresh=10&show_raw=1
,
Apr 25 2018
Re #4, I think these shards are identified as failures, am I understanding it incorrectly? browser_tests on Mac-10.11 Run on OS: 'Mac-10.11' Max pending time: 0:01:02.471440 (shard #9) Max shard duration: 34s (shard #7) stdout some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 swarming.summary step_metadata shard #0 (failed) shard #1 (failed) shard #2 (failed) shard #3 (failed) shard #4 (failed) shard #5 (failed) shard #6 (failed) shard #7 (failed) shard #8 (failed) shard #9 (failed) missing shard #0 missing shard #1 missing shard #2 missing shard #3 missing shard #4 missing shard #5 missing shard #6 missing shard #7 missing shard #8 missing shard #9
,
Apr 25 2018
"shards did not complete" + purple step is an indication to sheriffs etc that shard execution timed out, rather than that shards completed and failed.
,
Apr 25 2018
I see, thanks for explaining, I'll look into them.
,
Apr 25 2018
The telemetry_perf_unittests flakiness is tracked here: crbug.com/836447 , both issues are being actively worked on, so I'm going to close this bug, and will file a separate bug to investigate the timeout/failure thing. |
||||
►
Sign in to add a comment |
||||
Comment 1 by wjmaclean@chromium.org
, Apr 25 2018Components: Infra>Platform>Swarming