New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 787680 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 794085
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug



Sign in to add a comment

webkit_layout_tests timing out (flake) on chromium.webkit/WebKit Mac10.11

Project Member Reported by mgiuca@chromium.org, Nov 22 2017

Issue description

webkit_layout_tests failing on chromium.webkit/WebKit Mac10.11

Builders failed on: 
- WebKit Mac10.11: 
  https://build.chromium.org/p/chromium.webkit/builders/WebKit%20Mac10.11

Builders are periodically timing out (on one of the shards). A normal build seems to take around 10 min. These periodically take 20 min and then time out.

Failures are difficult to notice due to  Issue 787548  which was consistently failing from r518269 to r518449 (plugins/webview-plugin-type-change.html).

Timeout failures (unrelated to that bug) have occurred on these recent builds:

- https://luci-milo.appspot.com/buildbot/chromium.webkit/WebKit%20Mac10.11/25761
- https://luci-milo.appspot.com/buildbot/chromium.webkit/WebKit%20Mac10.11/25762
- https://luci-milo.appspot.com/buildbot/chromium.webkit/WebKit%20Mac10.11/25777
- https://luci-milo.appspot.com/buildbot/chromium.webkit/WebKit%20Mac10.11/25778
 
Looking... all the shards that timed out ran on different VMs (except for vm137-m1 that shows up twice) so I think we can rule out that it's not a VM-specific problem.
Cc: mar...@chromium.org
cc'ing maruel@

Comment 4 by mar...@chromium.org, Nov 22 2017

Components: -Infra>Platform>Buildbot Tests
Looking at the actual Swarming tasks;
https://chromium-swarm.appspot.com/task?id=39f817d03171d510
https://chromium-swarm.appspot.com/task?id=39f83a53c8a0a910
https://chromium-swarm.appspot.com/task?id=39fa14bd5132a810
https://chromium-swarm.appspot.com/task?id=39fa375b81e12d10

They all timed out due to I/O timeout; that is, no output to stdout for too long; currently 20 minutes.

Each task has a help section to show how to reproduce, and there's a "Debug" button to help you reproduce the task. Either either (or both) functionality to help diagnose the problem.
that's strange; I have no idea how there would be *no* output at all. Maruel, any theories?

Comment 6 by mar...@chromium.org, Nov 22 2017

There's no output because when SIGTERM is sent to the task's process, it doesn't react and just dies before it had time to write anything to the output directory.
Components: Infra>Platform>Swarming
Labels: -Infra-Troopers
Labels: Infra-Troopers
Status: Untriaged (was: Available)
This is happening somewhat frequently for the non-Webkit Mac bots too. For example, on Mac10.12 Tests, 3 of the last 10 builds have experienced this failure:

https://uberchromegw.corp.google.com/i/chromium.mac/builders/Mac10.12%20Tests

->

https://uberchromegw.corp.google.com/i/chromium.mac/builders/Mac10.12%20Tests/builds/9466
https://uberchromegw.corp.google.com/i/chromium.mac/builders/Mac10.12%20Tests/builds/9464
https://uberchromegw.corp.google.com/i/chromium.mac/builders/Mac10.12%20Tests/builds/9470

I notice that all three of these failures are on "m4" but I don't know if that's significant. Tagging for trooper attention.
Owner: jbudorick@chromium.org
Status: Started (was: Untriaged)
I don't think -m4 is relevant here.
Mergedinto: 794085
Status: Duplicate (was: Started)
Believe we've seen this before -- shard timeout after 20 minutes but, according the log, shortly after RWT starts.

Sign in to add a comment