New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 6 users
Status: WontFix
Owner:
Closed: Sep 7
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 713921


Participants' hotlists:
bling-infra-backlog


Sign in to add a comment
Various tests are flaky (timeout) on the "ios-simulator" bot
Project Member Reported by foolip@chromium.org, Apr 7 2017 Back to list
https://uberchromegw.corp.google.com/i/chromium.mac/builders/ios-simulator/builds/16338

This was ios_net_unittests (iPad Air 2 iOS 10.0). The stdout is:
https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.mac%2Fios-simulator%2F16338%2F%2B%2Frecipes%2Fsteps%2Fios_net_unittests__iPad_Air_2_iOS_10.0_%2F0%2Fstdout

And it ends like this:

+-------------------------------------------------------------------------------+
| Shard 0  https://chromium-swarm.appspot.com/user/task/355feddae6e80c10        |
+-------------------------------------------------------------------------------+
Device wiped.
/b/s/w/irheAvLd/src/out/Debug-iphonesimulator/iossim -d iPad Air 2 -s 10.0 /b/s/w/irheAvLd/src/out/Debug-iphonesimulator/ios_net_unittests.app

User defaults from command line:
    IDETestRunSpecificationPath = /var/folders/p4/py6nlvcx4fs5s3fg51_pzqmc0000gm/T/DE98128C-EBA4-4FF1-A9C9-543627ACC7E1
+-------------------------------------------------------------------------------+
| End of shard 0  Pending: 8.7s  Duration: 1202.7s  Bot: build216-m4  Exit: -15 |
+-------------------------------------------------------------------------------+
Total duration: 1202.7s
step returned non-zero exit code: 241
 
Comment 1 by s...@google.com, Apr 7 2017
Cc: smut@chromium.org
Labels: -Infra-Troopers
iossim hangs, known issue, unknown cause. Nothing for troopers to do, removing Infra-Troopers.
Comment 2 by vabr@chromium.org, Apr 18 2017
 Issue 712512  has been merged into this issue.
Comment 3 by vabr@chromium.org, Apr 18 2017
 Issue 712368  has been merged into this issue.
Project Member Comment 4 by chromium...@appspot.gserviceaccount.com, Apr 21 2017
Labels: Sheriff-Chromium
Detected 3 new flakes for test/step "ios_net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHAsSBUZsYWtlIhFpb3NfbmV0X3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
The same behaviour could be observed for components_unittests (iPad Air 2 iOS 10.0) on chromium.mac/ios-simulator:
https://uberchromegw.corp.google.com/i/chromium.mac/builders/ios-simulator/builds/16791

Exactly the same log:
https://chromium-swarm.appspot.com/task?id=35a8377904064d10&refresh=10&show_raw=1 

It really seems to be an issue of chromium.mac/builders/ios-simulator.
Blocking: 713921
Project Member Comment 7 by chromium...@appspot.gserviceaccount.com, Apr 27 2017
Detected 13 new flakes for test/step "crypto_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGwsSBUZsYWtlIhBjcnlwdG9fdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Comment 8 by sky@chromium.org, Apr 28 2017
Owner: pkl@chromium.org
smut: is this an infra bug, or a bug in the test?

I'm assigning to pkl in hopes of finding an owner and fixing this as it's contributing to flake on the waterfall.
Comment 9 by s...@google.com, Apr 28 2017
It's not really in the test, but it's also not really in the part of the infrastructure that I own. iossim [1] invocations are hanging.

[1] https://chromium.googlesource.com/chromium/src/+/master/testing/iossim/
Comment 10 by pkl@chromium.org, Apr 29 2017
Cc: sergeybe...@chromium.org baxley@chromium.org pkl@chromium.org
Owner: ----
Labels: -Sheriff-Chromium
Removing sheriff label since there's nothing for sheriff to do here.
Status: Available
Re: #c9: if it's in between tests and the "infrastructure" (i.e. the machine and related services like swarming), then it's likely a problem with the Apple layer (?). IMHO we (Ops) still should own this, even if it's hard to investigate it at the moment.
Project Member Comment 13 by chromium...@appspot.gserviceaccount.com, May 5 2017
Labels: Sheriff-Chromium
Detected 3 new flakes for test/step "ios_net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHAsSBUZsYWtlIhFpb3NfbmV0X3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Cc: jif@chromium.org
 Issue 718669  has been merged into this issue.
Labels: -Sheriff-Chromium
Removing from sheriff queue (again)
Project Member Comment 16 by chromium...@appspot.gserviceaccount.com, May 10 2017
Labels: Sheriff-Chromium
Detected 4 new flakes for test/step "ios_net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHAsSBUZsYWtlIhFpb3NfbmV0X3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Labels: -Sheriff-Chromium
Removing from sheriff queue (again).
Project Member Comment 18 by chromium...@appspot.gserviceaccount.com, May 24 2017
Labels: Sheriff-Chromium
Detected 3 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Labels: -Sheriff-Chromium Infra-Troopers
Removing from sheriff queue (again).

+Infra can this get looked at?
Comment 20 by s...@google.com, May 26 2017
Cc: justincohen@chromium.org rohitrao@chromium.org
Labels: -Infra-Troopers
The root issue (see #0) is not infra, we're shelling out to iossim and it hangs. The issue in #18 seems to be a legitimate failure in PaymentRequestViewControllerTest.TestSignedInStringFormatting, not flake.
Project Member Comment 21 by chromium...@appspot.gserviceaccount.com, May 27 2017
Labels: Sheriff-Chromium
Detected 179 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
Project Member Comment 22 by chromium...@appspot.gserviceaccount.com, May 27 2017
Cc: stale-flakes-reports@google.com
Reporting to stale-flakes-reports@google.com to investigate why this issue has been in the appropriate queue 5 times or more.
Cc: -stale-flakes-reports@google.com
Owner: sergeybe...@chromium.org
Status: Assigned
Sergey, can you please find an owner for this and assign this issue to them so that we can stop bugging sheriffs?
Project Member Comment 24 by chromium...@appspot.gserviceaccount.com, May 29 2017
Detected 7 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Labels: -Pri-3 -Sheriff-Chromium Pri-1
Summary: Various tests are flaky (timeout) on the "ios-simulator" bot (was: ios_net_unittests timed out, seemingly without running any tests)
Bumping up to P1 as it's frequent.

 Issue 725979  has been merged into this issue.
Labels: Infra-Troopers Hotlist-Infra-Troopers
 Issue 727491  has been merged into this issue.
Project Member Comment 29 by chromium...@appspot.gserviceaccount.com, May 30 2017
Detected 88 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 30 by chromium...@appspot.gserviceaccount.com, May 31 2017
Detected 3 new flakes for test/step "ios_net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHAsSBUZsYWtlIhFpb3NfbmV0X3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
 Issue 728056  has been merged into this issue.
Detected 3 new flakes for test/step "url_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg11cmxfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 45 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Why did we think this was related to iossim?
(I now see this error consistently on downstream iOS9 device bots. Ping me for links if needed.)

It fails with the same error code:
  WARNING:root:collect_cmd had non-zero return code: 241

Do we know what that means? I can't find anything specific using code search. I traced it to the result value from vield_results inside swarming.py https://cs.chromium.org/chromium/infra/luci/client/swarming.py?q=swarming.p&l=609

(Web search comes up with a SQL error?)
Sorry, you can likely ignore my comment. I was able to repo this locally and will report back if I see anything related to this.
Detected 3 new flakes for test/step "url_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg11cmxfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Cc: liaoyuke@chromium.org
CC - liaoyuke@ has been investigating this.
Detected 40 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "crypto_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGwsSBUZsYWtlIhBjcnlwdG9fdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "boringssl_crypto_tests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyIQsSBUZsYWtlIhZib3Jpbmdzc2xfY3J5cHRvX3Rlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "ios_net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHAsSBUZsYWtlIhFpb3NfbmV0X3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
This is currently marked as an "Infra-Troopers" bug, but it has an ownder. Is trooper intervention needed here?
Labels: -Infra-Troopers -Hotlist-Infra-Troopers
Owner: liaoyuke@chromium.org
Status: Started
No, we have identified the root cause of part of the problems and the fix is on the way.

Thanks for checking in!
Detected 3 new flakes for test/step "url_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg11cmxfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 29 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
 Issue 706464  has been merged into this issue.
Cc: svaldez@chromium.org davidben@chromium.org agl@chromium.org mab@chromium.org
 Issue 730703  has been merged into this issue.
Cc: rogerta@chromium.org
 Issue 713921  has been merged into this issue.
Cc: iannucci@chromium.org pinkerton@chromium.org rogerm@chromium.org
 Issue 713156  has been merged into this issue.
Cc: eugene...@chromium.org
 Issue 694906  has been merged into this issue.
Detected 3 new flakes for test/step "components_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRjb21wb25lbnRzX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "boringssl_crypto_tests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyIQsSBUZsYWtlIhZib3Jpbmdzc2xfY3J5cHRvX3Rlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "google_apis_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyIAsSBUZsYWtlIhVnb29nbGVfYXBpc191bml0dGVzdHMM. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "components_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRjb21wb25lbnRzX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "components_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRjb21wb25lbnRzX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "ios_chrome_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyHwsSBUZsYWtlIhRpb3NfY2hyb21lX3VuaXR0ZXN0cww. This message was posted automatically by the chromium-try-flakes app.
Cc: kbr@chromium.org
 Issue 693131  has been merged into this issue.
Detected 3 new flakes for test/step "google_apis_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyIAsSBUZsYWtlIhVnb29nbGVfYXBpc191bml0dGVzdHMM. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "google_apis_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyIAsSBUZsYWtlIhVnb29nbGVfYXBpc191bml0dGVzdHMM. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "gfx_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1nZnhfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Can we run the following when we hit a suspected timeout?
  `printf '\n' | xcrun simctl diagnose -b --output=/path/to/foo`

And then collect `/path/to/foo.tar.gz` and send it to Apple?
Status: WontFix
It's Apple's bug in their provided tools and now it has been fixed in Xcode 9. Tests are not hanging anymore after switching to Xcode 9!

https://build.chromium.org/p/chromium.mac/builders/ios-simulator?numbuilds=200
Sign in to add a comment