Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Issue 641708 "net_unittests" is flaky
Starred by 4 users Project Member Reported by chromium...@appspot.gserviceaccount.com, Aug 27 2016 Back to list
Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: iOS
Pri: 1
Type: Bug



Sign in to add a comment
"net_unittests" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA.



This flaky test/step was previously tracked in issue 624946.
 
Comment 1 by perkj@chromium.org, Aug 29 2016
Labels: -Sheriff-Chromium
Owner: lgar...@chromium.org
Status: Assigned
Please- seems to be flaky again or is the problem that the AppLauncher has crashed before running the app? 
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 7 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Components: Internals>Network
Net stack tests failing on iOS.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 6 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 9 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Owner: ----
Status: Untriaged
Not sure why this is assigned to me.
Labels: OS-iOS
Looks like iOS only. The tests I looked at were using EmbeddedTestServer and somehow wound up receiving an unexpected / empty response.
Issue 667466 has been merged into this issue.
Cc: droger@chromium.org marq@chromium.org
Owner: kapishnikov@chromium.org
Status: Assigned
Please triage. 
Owner: tbansal@chromium.org
Hi Tarun,

Could you take a look at this flake? The failure is related to NetworkQualityEstimator.

https://build.chromium.org/p/tryserver.chromium.mac/builders/ios-simulator/builds/118840

NetworkQualityEstimatorTest.TestEffectiveConnectionTypeObserver: 
../../net/nqe/network_quality_estimator_unittest.cc:1401: Failure
Value of: observer.effective_connection_types().size()
  Actual: 0
Expected: 1U
Which is: 1
../../base/test/histogram_tester.cc:158: Failure
Value of: actual_count
  Actual: 0
Expected: expected_count
Which is: 1
Histogram "NQE.MainFrame.EffectiveConnectionType" does not have the right number of samples (1) in the expected bucket (3). It has (0).
../../base/test/histogram_tester.cc:174: Failure
Value of: actual_count
  Actual: 0
Expected: expected_count
Which is: 1
Histogram "NQE.MainFrame.EffectiveConnectionType" does not have the right total number of samples (1). It has (0).
../../base/test/histogram_tester.cc:158: Failure
Value of: actual_count
  Actual: 0
Expected: expected_count
Which is: 1
Histogram "NQE.MainFrame.EffectiveConnectionType.Unknown" does not have the right number of samples (1) in the expected bucket (3). It has (0).
../../base/test/histogram_tester.cc:174: Failure
Value of: actual_count
  Actual: 0
Expected: expected_count
Which is: 1
Histogram "NQE.MainFrame.EffectiveConnectionType.Unknown" does not have the right total number of samples (1). It has (0).
../../net/nqe/network_quality_estimator_unittest.cc:1415: Failure
Value of: observer.effective_connection_types().size()
  Actual: 0
Expected: 1U
Which is: 1
../../net/nqe/network_quality_estimator_unittest.cc:1422: Failure
Value of: observer.effective_connection_types().size()
  Actual: 1
Expected: 2U
Which is: 2
../../net/nqe/network_quality_estimator_unittest.cc:1429: Failure
Value of: observer.effective_connection_types().size()
  Actual: 1
Expected: 2U
Which is: 2

Do you know what can cause this failure?
I am not sure why it is flaky only on iOS. I am going to disable the test on iOS for now.
Project Member Comment 30 by bugdroid1@chromium.org, Dec 9
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/30d83c2a9a1bdfc437962297ff0b415a797ea1e3

commit 30d83c2a9a1bdfc437962297ff0b415a797ea1e3
Author: tbansal <tbansal@chromium.org>
Date: Fri Dec 09 20:48:01 2016

Disable the flaky NetworkQualityEstimator unittest on ios_simulator

Disable the test until it is clear why it is flaky.

BUG=641708
TBR=bengr@chromium.org

Review-Url: https://codereview.chromium.org/2560253002
Cr-Commit-Position: refs/heads/master@{#437637}

[modify] https://crrev.com/30d83c2a9a1bdfc437962297ff0b415a797ea1e3/net/nqe/network_quality_estimator_unittest.cc

Owner: kapishnikov@chromium.org
Punting it back to kapishnikov.
Cc: tbansal@chromium.org
Detected 7 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Cc: mef@chromium.org
Owner: lilyhoughton@chromium.org
I am on vacation till Dec 16. For the time being reassigning this bug to lilyhoughton@.
Project Member Comment 35 by bugdroid1@chromium.org, Dec 15
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9b0a183012bd52bdd368a32b0849624981511a08

commit 9b0a183012bd52bdd368a32b0849624981511a08
Author: tbansal <tbansal@chromium.org>
Date: Thu Dec 15 03:07:15 2016

Disable the flaky NetworkQualityEstimator unittest on ios_simulator

NetworkQualityEstimator::TestEffectiveConnectionTypeObserver is disabled on
ios-simulator until it is clear why it is flaky.

BUG=641708

Review-Url: https://codereview.chromium.org/2559373005
Cr-Commit-Position: refs/heads/master@{#438722}

[modify] https://crrev.com/9b0a183012bd52bdd368a32b0849624981511a08/net/nqe/network_quality_estimator_unittest.cc

Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Cc: -droger@chromium.org
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 5 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 7 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 8 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 4 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
kapishkov@ and I looked at this flakes link. The last few are shard timeout: "shard #0 timed out, took too much time to complete." How do we tell which tests are taking too long? Is this timeout test specific or infrastructure specific?



Cc: kapishnikov@chromium.org
+kapishnikov@
Doesn't look like any one test is timing out, but rather the whole shard.

One of the failure logs is this (21.5 minutes):

https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.mac%2Fios-simulator%2F162283%2F%2B%2Frecipes%2Fsteps%2Fnet_unittests__iPhone_6s_Plus_iOS_9.0_%2F0%2Fstdout

And for comparison a success one at 4.3 minutes (which runs all the tests as well) is:

https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.mac%2Fios-simulator%2F162603%2F%2B%2Frecipes%2Fsteps%2Fnet_unittests__iPhone_6s_Plus_iOS_9.0_%2F0%2Fstdout

At a glance nothing really jumps out at me from comparing those. The last test can't have taken too long since if it did it would have exceeded the per test timeout. And the other runtimes don't look appreciably different, so unclear where the lost time is.

Not sure if we have better tools for analyzing the logs, but you may want to start by adding all the test runtimes and seeing if it actually sums to 21 minutes (because it doesn't look like it would).
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 58 by chromium...@appspot.gserviceaccount.com, Mar 25 (4 days ago)
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Project Member Comment 59 by chromium...@appspot.gserviceaccount.com, Mar 27 (2 days ago)
Detected 3 new flakes for test/step "net_unittests". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyGAsSBUZsYWtlIg1uZXRfdW5pdHRlc3RzDA. This message was posted automatically by the chromium-try-flakes app.
Sign in to add a comment