SSLUIWorkerFetchTest.MixedContentSettings and MixedContentSettingsWithBlockingCSP and MixedContentSubFrame are flaky |
|||||||||||
Issue descriptionThis test has failed off-and-on for several days on the win-asan builder: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/win-asan?limit=100 Some examples https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934284260312051568/+/steps/browser_tests_on_Windows-10-15063/0/stdout https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934700368693278080/+/steps/browser_tests_on_Windows-10-15063/0/stdout https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8934211564984691392/+/steps/browser_tests_on_Windows-10-15063/0/stdout
,
Oct 24
This is becoming exceptionally flaky, https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-rel/15111 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-rel/15110 Will need to disable this test if folks aren't going to work on this. ../../chrome/browser/ssl/ssl_browsertest_util.cc:64: Failure Expected equality of these values: expected_security_level Which is: 3 security_info.security_level Which is: 5 Google Test trace: ../../chrome/browser/ssl/ssl_browsertest.cc:3782: RunMixedContentSettingsTest :allow_running_insecure_content=false strict_mixed_content_checking=true strictly_block_blockable_mixed_content=true Stack trace: #0 0x0000021add2c testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop() #1 0x0000021ad6f9 testing::internal::AssertHelper::operator=() #2 0x000000e168b6 ssl_test_util::SecurityStyle::Check() #3 0x000000e16ea5 ssl_test_util::CheckSecurityState() #4 0x000000dd7c58 SSLUIWorkerFetchTest::RunMixedContentSettingsTest() #5 0x000000dd749b SSLUIWorkerFetchTest_MixedContentSettings_Test::RunTestOnMainThread() #6 0x000004768e7b content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() #7 0x0000043338f4 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() #8 0x00000433283c ChromeBrowserMainParts::PreMainMessageLoopRun() #9 0x000001d6d519 chromeos::ChromeBrowserMainPartsChromeos::PreMainMessageLoopRun() #10 0x000002d40d9a content::BrowserMainLoop::PreMainMessageLoopRun() #11 0x0000030cf397 content::StartupTaskRunner::RunAllTasksNow() #12 0x000002d3f9d6 content::BrowserMainLoop::CreateStartupTasks() #13 0x000002d430d0 content::BrowserMainRunnerImpl::Initialize() #14 0x000002d3da34 content::BrowserMain() #15 0x000003f91234 content::ContentMainRunnerImpl::Run() #16 0x000005f5bb20 service_manager::Main() #17 0x000003f8f6f4 content::ContentMain() #18 0x000004768aa3 content::BrowserTestBase::SetUp() #19 0x0000042e151e InProcessBrowserTest::SetUp() #20 0x000000e0477f SSLUITestBase::SetUp() ../../chrome/browser/ssl/ssl_browsertest_util.cc:45: Failure Expected equality of these values: !!(expected_authentication_state & AuthState::RAN_INSECURE_CONTENT) Which is: false ran_insecure_content Which is: true Google Test trace: ../../chrome/browser/ssl/ssl_browsertest.cc:3782: RunMixedContentSettingsTest :allow_running_insecure_content=false strict_mixed_content_checking=true strictly_block_blockable_mixed_content=true Stack trace: #0 0x0000021add2c testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop() #1 0x0000021ad6f9 testing::internal::AssertHelper::operator=() #2 0x000000e16580 ssl_test_util::AuthState::Check() #3 0x000000dd8198 SSLUIWorkerFetchTest::RunMixedContentSettingsTest() #4 0x000000dd749b SSLUIWorkerFetchTest_MixedContentSettings_Test::RunTestOnMainThread() #5 0x000004768e7b content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() #6 0x0000043338f4 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() #7 0x00000433283c ChromeBrowserMainParts::PreMainMessageLoopRun() #8 0x000001d6d519 chromeos::ChromeBrowserMainPartsChromeos::PreMainMessageLoopRun() #9 0x000002d40d9a content::BrowserMainLoop::PreMainMessageLoopRun() #10 0x0000030cf397 content::StartupTaskRunner::RunAllTasksNow() #11 0x000002d3f9d6 content::BrowserMainLoop::CreateStartupTasks() #12 0x000002d430d0 content::BrowserMainRunnerImpl::Initialize() #13 0x000002d3da34 content::BrowserMain() #14 0x000003f91234 content::ContentMainRunnerImpl::Run() #15 0x000005f5bb20 service_manager::Main() #16 0x000003f8f6f4 content::ContentMain() #17 0x000004768aa3 content::BrowserTestBase::SetUp() #18 0x0000042e151e InProcessBrowserTest::SetUp() #19 0x000000e0477f SSLUITestBase::SetUp()
,
Oct 24
,
Oct 24
I'm taking a look now, I'll either send a fix if it's a simple one, or disable and triage further.
,
Oct 24
+danakj in case this is related to issue 888156 and https://chromium-review.googlesource.com/c/chromium/src/+/1290140
,
Oct 24
Sounds like this is an older issue but thanks for the cc, could be something else you never know.
,
Oct 24
Assigning to horo (who authored the test) for further triage, since this is flaky on all parameter combinations, so it doesn't seem to be related to committed interstitials.
,
Oct 24
@danakj, builds after the revert are passing, so I think this is another instance of flake found by the speed up your CL landed.
,
Oct 24
,
Oct 24
,
Oct 24
This is already disabled on some windows bots for being flaky, which usually just means the test itself is flaky.
,
Oct 24
,
Oct 25
This test fails consistently by delaying the SwapOut_ACK message, just like ProcessManagementTest.ExtensionProcessBalancing in https://bugs.chromium.org/p/chromium/issues/detail?id=898628
,
Oct 25
it appears that because RunMixedContentSettingsTest() is run multiple times in the same test, the previous run through's navigation is interfering with verifying the expectations of the current run. Waiting for the previous navigation's processhost to be destroyed removes the flake. I can't say if that is a bug in the test though or a bug in our production code during navigation allowing incorrcect permissions until the old processhost is torn down. When we ClearErrorState() it does a navigation also, which may be throwing some complication into this story. But waiting on that navigation does not clear up the flakiness, we must wait for the teardown from the previous iterations navigate.
,
Oct 26
,
Nov 5
,
Nov 5
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b653a893e550bb429f0c3131931736e62ef07b68 commit b653a893e550bb429f0c3131931736e62ef07b68 Author: danakj <danakj@chromium.org> Date: Mon Nov 05 19:46:14 2018 Disable flaky SSL browser tests. TBR=cthomp@chromium.org, rsleevi Change-Id: Id59454fea00ef51fe6c4cf6aded5a2fd2c8a3b8e Bug: 890372 Reviewed-on: https://chromium-review.googlesource.com/c/1318379 Reviewed-by: danakj <danakj@chromium.org> Commit-Queue: danakj <danakj@chromium.org> Cr-Commit-Position: refs/heads/master@{#605421} [modify] https://crrev.com/b653a893e550bb429f0c3131931736e62ef07b68/chrome/browser/ssl/ssl_browsertest.cc
,
Nov 26
BTW a number of the tests made flaky by this did turn out to be production bugs, so I recommend not sleeping on this. |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by rsleevi@chromium.org
, Sep 28Components: -Internals>Network>SSL UI>Browser>Interstitials