New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 836765 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocked on:
issue 828031



Sign in to add a comment

Shards are timing out on Mac10.11 Tests, Mac10.12 Tests and Linux Tests (dbg)

Project Member Reported by hbos@chromium.org, Apr 25 2018

Issue description

--- Mac10.11 Tests ---
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.11%20Tests/25600

Purple colored box:
  browser_tests on Mac-10.11 Run on OS: 'Mac-10.11'
  Max pending time: 0:01:02.471440 (shard #9)
  Max shard duration: 34s (shard #7)
  ...
  some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  ...

--- Mac10.12 Tests ---
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.12%20Tests/12295

Purple colored box:
  browser_tests on Mac-10.12 Run on OS: 'Mac-10.12'
  Max pending time: 28s (shard #9)
  Max shard duration: 30s (shard #4)
  ...
  some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  ...

--- Linux Tests (dbg) ---
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Tests%20%28dbg%29%281%29/71625

Red colored box:
  telemetry_perf_unittests Run on OS: 'Ubuntu-14.04'
  Max shard duration: 0:16:34.240000 (shard #8)
  Total tests: 236
  * Passed: 212 (212 expected, 0 unexpected)
  * Skipped: 24 (24 expected, 0 unexpected)
  * Failed: 0 (0 expected, 0 unexpected)
  * Flaky: 0 (0 expected, 0 unexpected)
  ...
  shard #8 timed out, took too much time to complete
  ...

 
Cc: kbr@chromium.org ellyjo...@chromium.org
Components: Infra>Platform>Swarming
Not sure if the Mac part of this report is related to the ongoing work on the Mac bots, looping in ellyjones@ and kbr@
I recently landed <https://chromium-review.googlesource.com/c/chromium/src/+/1025638> for Mac - the timeouts could (?) be related to  issue 828031  overall, but I'm not sure if/how that CL could have introduced bot timeouts.
Blockedon: 828031
Cc: liaoyuke@chromium.org
Owner: liaoyuke@chromium.org
Status: Assigned (was: Untriaged)
The Mac failures are caused by  crbug.com/828031 . I'll look into the Linux red.
we should be looking at why those shards are getting picked up as timeouts and not failures. swarming certainly doesn't seem to think that they're timeouts, e.g. https://chromium-swarm.appspot.com/task?id=3d152f8e997ea710&refresh=10&show_raw=1
Cc: jbudorick@chromium.org
Re #4, I think these shards are identified as failures, am I understanding it incorrectly?

browser_tests on Mac-10.11 Run on OS: 'Mac-10.11'
Max pending time: 0:01:02.471440 (shard #9)
Max shard duration: 34s (shard #7)
stdout
some shards did not complete: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
swarming.summary
step_metadata
shard #0 (failed)
shard #1 (failed)
shard #2 (failed)
shard #3 (failed)
shard #4 (failed)
shard #5 (failed)
shard #6 (failed)
shard #7 (failed)
shard #8 (failed)
shard #9 (failed)
missing shard #0
missing shard #1
missing shard #2
missing shard #3
missing shard #4
missing shard #5
missing shard #6
missing shard #7
missing shard #8
missing shard #9


"shards did not complete" + purple step is an indication to sheriffs etc that shard execution timed out, rather than that shards completed and failed.
I see, thanks for explaining, I'll look into them.
Status: WontFix (was: Assigned)
The telemetry_perf_unittests flakiness is tracked here:  crbug.com/836447 , both issues are being actively worked on, so I'm going to close this bug, and will file a separate bug to investigate the timeout/failure thing.

Sign in to add a comment