Issue metadata
Sign in to add a comment
|
DCHECK hit in DnsConfigServiceWin in some tests |
||||||||||||||||||||||||
Issue descriptionTaskScheduler::GetInstance is returning an empty ref (see screenshot). Perhaps it's a shutdown race and the DnsConfigServiceThread isn't being torn down before the tests's ScopedTaskEnvironment? Just a guess. I don't have a repro, only screenshots from the bots like the one attached. I also don't know which test binary is causing it -- the "Fatal error" message box is left over from some previous test run on the bot when interactive_ui_tests is run (which dumps the snapshot). This is presumably causing some other test to fail. It's interesting that said other test is even generating this dialog. It means that the test is calling logging::SetShowErrorDialogs(true), which should not be happening on the bots at all, as they have CHROME_HEADLESS=1 in their environment (which suppresses them). Assigning to a random member of net/OWNERS for triage and assignment.
,
Nov 29 2017
Do you have links to bot runs where this happened? The DnsConfigService conversion to TaskScheduler was only a few months ago, I wouldn't be surprised to still find some bugs. It looks like it's failing in initialization, not shutdown.
,
Nov 29 2017
No. I only have these screenshots from tests that run *after* the failing test on the same bot. I don't know what it was that ran before interactive_ui_tests on this particular swarming bot. I've seen this same error message on many runs. Maybe someone from infra could help figure out what is popping the dialog. +maruel, who added some logging recently that might help.
,
Dec 4 2017
Now that CHROME_HEADLESS=1 is properly being set (see issue 790985), we're seeing this crash in failing browser_tests; see, for example, https://chromium-swarm.appspot.com/task?id=3a2d610a75456610&refresh=10&show_raw=1: [ RUN ] ServiceProcessControlBrowserTest.DieOnDisconnect ... [4372:6280:1201/161614.652:FATAL:post_task.cc(72)] Check failed: TaskScheduler::GetInstance(). Ref. Prerequisite section of post_task.h. Hint: if this is in a unit test, you're likely merely missing a base::test::ScopedTaskEnvironment member in your fixture. Backtrace: base::debug::StackTrace::StackTrace [0x02BBD2B0+32] base::debug::StackTrace::StackTrace [0x02B7D2ED+13] logging::LogMessage::~LogMessage [0x02B2801E+78] base::PostDelayedTaskWithTraits [0x02B52F01+97] base::PostTaskWithTraits [0x02B53081+65] base::CreateCOMSTATaskRunnerWithTraits [0x02B53455+277] base::internal::PostTaskAndReplyImpl::PostTaskAndReply [0x02BC6D51+625] base::PostTaskWithTraitsAndReply [0x02B53011+97] net::SerialWorker::WorkNow [0x02E87709+329] net::internal::DnsConfigServiceWin::ReadNow [0x02DF5A17+23] net::DnsConfigService::WatchConfig [0x02D771CB+217] net::NetworkChangeNotifierWin::DnsConfigServiceThread::Init [0x02D76949+169] base::Thread::ThreadMain [0x02B6EBD4+548] base::PlatformThread::GetCurrentThreadPriority [0x02B4B1D3+467] BaseThreadInitThunk [0x75D3337A+18] RtlInitializeExceptionChain [0x77C692B2+99] RtlInitializeExceptionChain [0x77C69285+54] Received fatal exception EXCEPTION_BREAKPOINT Backtrace: base::debug::BreakDebugger [0x02BC656C+12] ?Run@?$Invoker@U?$BindState@P6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@base@@1@Z$$V@internal@base@@$$A6AXPBDHV?$BasicStringPiece@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@3@1@Z@internal@ [0x0353081F+31] logging::LogMessage::~LogMessage [0x02B283DE+1038] base::PostDelayedTaskWithTraits [0x02B52F01+97] base::PostTaskWithTraits [0x02B53081+65] base::CreateCOMSTATaskRunnerWithTraits [0x02B53455+277] base::internal::PostTaskAndReplyImpl::PostTaskAndReply [0x02BC6D51+625] base::PostTaskWithTraitsAndReply [0x02B53011+97] net::SerialWorker::WorkNow [0x02E87709+329] net::internal::DnsConfigServiceWin::ReadNow [0x02DF5A17+23] net::DnsConfigService::WatchConfig [0x02D771CB+217] net::NetworkChangeNotifierWin::DnsConfigServiceThread::Init [0x02D76949+169] base::Thread::ThreadMain [0x02B6EBD4+548] base::PlatformThread::GetCurrentThreadPriority [0x02B4B1D3+467] BaseThreadInitThunk [0x75D3337A+18] RtlInitializeExceptionChain [0x77C692B2+99] RtlInitializeExceptionChain [0x77C69285+54] ../../chrome/browser/service_process/service_process_control_browsertest.cc(126): error: Failed ../../chrome/browser/service_process/service_process_control_browsertest.cc(314): error: Value of: ServiceProcessControl::GetInstance()->IsConnected() Actual: false Expected: true [264:7424:1201/161616.580:INFO:chrome_cryptauth_service.cc(222)] Profile is not authenticated yet; waiting before starting CryptAuth managers. [264:7424:1201/161616.890:WARNING:url_request_context_getter.cc(43)] URLRequestContextGetter leaking due to no owning thread. [ FAILED ] ServiceProcessControlBrowserTest.DieOnDisconnect, where TypeParam = and GetParam() = (2635 ms) Without CHROME_HEADLESS=1, the dialog in the initial report is shown, which was likely leading to test timeouts. Now we're at least seeing the error where it happens. This is leading to flaky browser_tests, so please fix as soon as possible. I have bumped this to P1 since it should now be quite actionable.
,
Dec 4 2017
It sounds like this is a problem with initialization order in a specific set of tests, which already has a bug filed. The DnsConfigService initialization just happens to be something that needs the TaskScheduler early on. I don't think I'm the right person to debug this, but let me know if you see it anywhere other than ServiceProcessControl tests.
,
Dec 4 2017
The crash stacks for the initial reports in issue 517420 have nothing to do with those in comment 4 here. I don't think it's helpful to lump unrelated problems into the same bug. I just poked around in codesearch for a few minutes, and it seems pretty clear that the problem is in chrome/service/service_process.cc: bool ServiceProcess::Initialize(base::MessageLoopForUI* message_loop, const base::CommandLine& command_line, ServiceProcessState* state) { ... network_change_notifier_.reset(net::NetworkChangeNotifier::Create()); ... base::TaskScheduler::Create("CloudPrintServiceProcess"); ... } those two lines are out-of-order. Francois, could you take this?
,
Dec 4 2017
Oh, and if I'm reading it right, this is a product problem, not a test problem. That is, it's not initialization order in a specific set of tests,
,
Dec 4 2017
I can believe it's unrelated to the Mac flakiness, I didn't look. But it's definitely the cause of the Windows flakiness in comments 18-20 on the other bug, I did check those links, and I don't think it's helpful to not inform the people working on a bug that you've decided to separate it out. I'll go comment there.
,
Dec 5 2017
Good point. Thanks for updating the other bug. Cheers.
,
Dec 5 2017
Upon re-reading comment 6, I think I came across as more gruff than is appropriate. My apologies.
,
Dec 6 2017
,
Dec 6 2017
Issue 791791 has been merged into this issue.
,
Dec 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/1d7a91284b1ea021daaeeea39c8f6a242782f670 commit 1d7a91284b1ea021daaeeea39c8f6a242782f670 Author: Francois Doray <fdoray@chromium.org> Date: Thu Dec 07 00:01:50 2017 Initialize NetworkChangeNotifier after TaskScheduler in service process. The NetworkChangeNotifier must be created after TaskScheduler because it posts tasks to it. Bug: 789442 Change-Id: Ie6066064a5f840666160bac76d95ad16d3363708 Reviewed-on: https://chromium-review.googlesource.com/810925 Reviewed-by: Vitaly Buka <vitalybuka@chromium.org> Commit-Queue: François Doray <fdoray@chromium.org> Cr-Commit-Position: refs/heads/master@{#522262} [modify] https://crrev.com/1d7a91284b1ea021daaeeea39c8f6a242782f670/chrome/service/service_process.cc
,
Dec 8 2017
Thanks for the fix! Would you also please re-enable the tests that were disabled for this bug? https://bugs.chromium.org/p/chromium/issues/detail?id=517420#c18 and https://bugs.chromium.org/p/chromium/issues/detail?id=517420#c20 might both be okay now.
,
Dec 8 2017
CL to re-enabled the tests waiting for review https://chromium-review.googlesource.com/c/chromium/src/+/817574
,
Dec 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/29c78ed184f0b7c2a3e71462450030d72ce0bc70 commit 29c78ed184f0b7c2a3e71462450030d72ce0bc70 Author: Francois Doray <fdoray@chromium.org> Date: Fri Dec 08 23:23:38 2017 Re-enable ServiceProcessControlBrowserTest on Windows. This reverts commits d70e91da314d7d5b28bcfe97007933dc0aec9dfd and 448f299e4fb9825df68bad77b8f2352b288f5a63. Issue on Windows was fixed by https://chromium-review.googlesource.com/810925. Issue on Mac seems to have a different root cause. We should consider re-enabling the tests, which were disabled in 2015, and see if the issue is still there. Bug: 789442 , 517420 Change-Id: Id4ce64b5ec1ffa6d97c25b7967a92ac0ad293979 Reviewed-on: https://chromium-review.googlesource.com/817574 Reviewed-by: Vitaly Buka <vitalybuka@chromium.org> Commit-Queue: Vitaly Buka <vitalybuka@chromium.org> Cr-Commit-Position: refs/heads/master@{#522915} [modify] https://crrev.com/29c78ed184f0b7c2a3e71462450030d72ce0bc70/chrome/browser/service_process/service_process_control_browsertest.cc
,
Dec 11 2017
,
Dec 13 2017
Thanks! |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by mmenke@chromium.org
, Nov 29 2017Owner: mge...@chromium.org