New issue
Advanced search Search tips

Issue 668008 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner:
Closed: Nov 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

"TransportSecurityStateTest.BuiltinCertPins" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Nov 23 2016

Issue description

"TransportSecurityStateTest.BuiltinCertPins" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNQsSBUZsYWtlIipUcmFuc3BvcnRTZWN1cml0eVN0YXRlVGVzdC5CdWlsdGluQ2VydFBpbnMM.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 
Components: Internals>Network
Owner: rdsmith@chromium.org
Status: Assigned (was: Untriaged)
rdsmith@, I wonder if you could please take a look or route to a more appropriate //net owner?

I'll go ahead and prepare a CL to disable the flaky test (FWIW, it only seems to flake on android_n5x_swarming_rel).
Snippet from the logs:

483s run_tests_on_device(00b295c62e5531f3)  [ RUN      ] TransportSecurityStateTest.OptionalHSTSCertPins
I  169.483s run_tests_on_device(00b295c62e5531f3)  ../../net/http/transport_security_state_unittest.cc:1338: Failure
I  169.483s run_tests_on_device(00b295c62e5531f3)  Value of: HasStaticPublicKeyPins("google.com")
I  169.483s run_tests_on_device(00b295c62e5531f3)    Actual: false
I  169.483s run_tests_on_device(00b295c62e5531f3)  Expected: true
I  169.483s run_tests_on_device(00b295c62e5531f3)  ../../net/http/transport_security_state_unittest.cc:1339: Failure
I  169.483s run_tests_on_device(00b295c62e5531f3)  Value of: HasStaticPublicKeyPins("www.google.com")
I  169.483s run_tests_on_device(00b295c62e5531f3)    Actual: false
I  169.483s run_tests_on_device(00b295c62e5531f3)  Expected: true
Labels: -Sheriff-Chromium
CL to disable the flaky test is in CQ: https://codereview.chromium.org/2527723002
Project Member

Comment 4 by bugdroid1@chromium.org, Nov 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/d0d5a2921f045da504fb1a8d4cfbcf89d8535ca5

commit d0d5a2921f045da504fb1a8d4cfbcf89d8535ca5
Author: lukasza <lukasza@chromium.org>
Date: Wed Nov 23 20:57:14 2016

Disable flaky TransportSecurityStateTest.BuiltinCertPins on Android.

BUG= 668008 
TBR=rdsmith@chromium.org, rockot@chromium.org

Review-Url: https://codereview.chromium.org/2527723002
Cr-Commit-Position: refs/heads/master@{#434245}

[modify] https://crrev.com/d0d5a2921f045da504fb1a8d4cfbcf89d8535ca5/net/http/transport_security_state_unittest.cc

Cc: jbudorick@chromium.org rsleevi@chromium.org
Hmmm... I am beginning to wonder if this might potentially be a duplicate of issue 666821.  OTOH, I don't see "This build is more than" substring when looking at the log output linked from try-flakes link in #c0.
I wouldn't be surprised.
Mergedinto: 666821
Status: Duplicate (was: Assigned)
Yes, this looks identical to Issue 666821, but wouldn't show up until Eric's new CL (which moved it to net test suite), because the CT job may have run on a different shard.

I'd like to revert the disabling for now, so that we don't forget to late - is there a reason not to?

Comment 8 by eroman@chromium.org, Nov 23 2016

We can safely revert the disabling.

Now that commit ada22007f4df4381ed1092a96cec8dd836be5cb5 has landed, the tools should stop whining about individual flaky tests, as they won't even be run in this case.

Comment 9 by eroman@chromium.org, Nov 23 2016

I kicked off a revert for this as from the perspective of this bug it is "fixed"

Thanks for being intolerant of flaky tests!
Project Member

Comment 10 by bugdroid1@chromium.org, Nov 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1a19ef587d94c13186eafeb7997336678b320091

commit 1a19ef587d94c13186eafeb7997336678b320091
Author: eroman <eroman@chromium.org>
Date: Wed Nov 23 22:46:05 2016

Revert of Disable flaky TransportSecurityStateTest.BuiltinCertPins on Android. (patchset #1 id:1 of https://codereview.chromium.org/2527723002/ )

Reason for revert:
This test should no longer be flaky on Android, as we instead fail at the test-suite level when the device failed to set its clock properly.

Original issue's description:
> Disable flaky TransportSecurityStateTest.BuiltinCertPins on Android.
>
> BUG= 668008 
> TBR=rdsmith@chromium.org, rockot@chromium.org
>
> Committed: https://crrev.com/d0d5a2921f045da504fb1a8d4cfbcf89d8535ca5
> Cr-Commit-Position: refs/heads/master@{#434245}

TBR=rdsmith@chromium.org,rockot@chromium.org,lukasza@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 668008 

Review-Url: https://codereview.chromium.org/2524023003
Cr-Commit-Position: refs/heads/master@{#434269}

[modify] https://crrev.com/1a19ef587d94c13186eafeb7997336678b320091/net/http/transport_security_state_unittest.cc

Owner: eroman@chromium.org

Sign in to add a comment