New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 847263 link

Starred by 5 users

Issue metadata

Status: Duplicate
Owner: ----
Closed: Sep 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: ----

Blocked on:
issue 869227



Sign in to add a comment

SpawnedTestServer failing to start on chromium.mac/Mac10.10 Tests

Project Member Reported by sheriff-...@appspot.gserviceaccount.com, May 28 2018

Issue description

Filed by sheriff-o-matic@appspot.gserviceaccount.com on behalf of lunalu@google.com

net_unittests failing on chromium.mac/Mac10.10 Tests

Builders failed on: 
- Mac10.10 Tests: 
  https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.10%20Tests


 
Components: Internals>Network Test>iOS
Cc: ricea@chromium.org
Hi ricea@, it seems like you touched the src/net/ code last, would you mind to take a look?

Thanks

Comment 3 by ricea@chromium.org, May 29 2018

Components: -Test>iOS Infra>Client>Chrome
Labels: -Pri-2 OS-Mac Pri-3
testserver.py is failing to start:

[ RUN      ] SSLClientSocketReadTest.Read/0
[72416:779:0528/084024.305441:32512912579320:ERROR:local_test_server_posix.cc(81)] poll() timed out; bytes_read=0
[72416:779:0528/084024.305532:32512912654680:ERROR:local_test_server_posix.cc(169)] Could not read server_data_len
[72416:779:0528/084024.309851:32512916983903:ERROR:ssl_client_socket_unittest.cc(868)] Could not start SpawnedTestServer
../../net/socket/ssl_client_socket_unittest.cc:1349: Failure
Value of: StartTestServer(SpawnedTestServer::SSLOptions())
  Actual: false
Expected: true
Stack trace:
0   net_unittests                       0x000000010adf398b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91
1   net_unittests                       0x000000010adf3349 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89
2   net_unittests                       0x000000010a2e9bb8 net::SSLClientSocketReadTest_Read_Test::TestBody() + 520
[  FAILED  ] SSLClientSocketReadTest.Read/0, where GetParam() = false (10031 ms)
[30709/30773] SSLClientSocketReadTest.Read/0 (10031 ms)

This happens sometimes due to a bad bot, or a bad bot configuration. In any case, it appears to have been transient as the tests are now passing again.

It's probably okay to close this. Adding the Infra>Client>Chrome component in case someone wants to do some additional investigation.

Comment 4 by xhw...@chromium.org, May 29 2018

Labels: -Pri-3 Pri-2
This is happening again so I don't feel it's transient:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.10%20Tests/32619

Comment 6 by mmenke@chromium.org, May 31 2018

Summary: SpawnedTestServer failing to start on chromium.mac/Mac10.10 Tests (was: net_unittests failing on chromium.mac/Mac10.10 Tests)
From https://logs.chromium.org/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8944856349501667728%2F%2B%2Fsteps%2Fnet_unittests_on_Mac-10.10%2F0%2Flogs%2FSSLClientSocketReadTest.Read_WithSynchronousError__x2f_0%2F0

[ RUN      ] SSLClientSocketReadTest.Read_WithSynchronousError/0
[39955:779:0601/224336.301157:14630605511083:ERROR:local_test_server_posix.cc(81)] poll() timed out; bytes_read=0
[39955:779:0601/224336.301219:14630605561851:ERROR:local_test_server_posix.cc(169)] Could not read server_data_len
[39955:779:0601/224336.305998:14630610349184:ERROR:ssl_client_socket_unittest.cc(868)] Could not start SpawnedTestServer
../../net/socket/ssl_client_socket_unittest.cc:1437: Failure
Value of: StartTestServer(SpawnedTestServer::SSLOptions())
  Actual: false
Expected: true
Stack trace:
0   net_unittests                       0x000000010f768e7b testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 91
1   net_unittests                       0x000000010f768839 testing::internal::AssertHelper::operator=(testing::Message const&) const + 89
2   net_unittests                       0x000000010ec5baf8 net::SSLClientSocketReadTest_Read_WithSynchronousError_Test::TestBody() + 600
[  FAILED  ] SSLClientSocketReadTest.Read_WithSynchronousError/0, where GetParam() = false (10137 ms)

I am a bit out of ideas here... This means that the python process did not produce any data within 10 seconds. Not sure what we can do to debug this as the python executable seemed to happily start.


Labels: Network-Triaged
Fascinating flakiness dashboard.  The failures look like zebra stripes.  Lots of tests fail at the same time.  (not all though)

The failures are still happening pretty often, about every twelve hours, so I'm keeping this in the sheriff queue for further investigation.  It should be resolved soon, as it would such to have to disable the whole test suite.
^such^suck
Cc: dpranke@chromium.org
I'm going to remove this from the sheriff queue. 

We don't have a good mechanism for distinguishing between "this is a new thing that a sheriff needs to deal with" and "here's something a sheriff might want to be aware of", but I think we normally try to keep the Sheriff-Chromium label restricted to the former category, and I think this bug largely falls into the latter category.

Ideally we'd have a good mechanism in sheriff-o-matic for displaying the latter category somewhere other than at the top of the page, but displaying them at the top (above current consistent failures) IMO makes sheriffing harder.
Labels: -Sheriff-Chromium
Issue 855684 has been merged into this issue.
Cc: s...@chromium.org xunji...@chromium.org
 Issue 876563  has been merged into this issue.
Blockedon: 869227
The SpawnedTestServer is failing to start up again on Mac10.10: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.10%20Tests/34415

Is there any solution here?
Mergedinto: 869227
Status: Duplicate (was: Available)

Sign in to add a comment