Issue metadata
Sign in to add a comment
|
chromevox_tests flaky with FATAL:keep_alive_registry.cc(53): Check failed: !g_browser_process->IsShuttingDown(). |
||||||||||||||||||||||||
Issue descriptionSeen on this tester: https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng?numbuilds=200 Multiple builds are failing chromevox_tests. A few examples: https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/builds/192498 https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/builds/192484 https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/builds/192480 https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/builds/192478 Test failures seen: BackgroundTest.GlobsToRegExp BackgroundTest.NextNamespaces BackgroundTest.ClassicNamespaces The assertion failure is the same in all cases: [26379:26379:0627/150912:FATAL:keep_alive_registry.cc(53)] Check failed: !g_browser_process->IsShuttingDown(). #0 0x0000005e317e base::debug::StackTrace::StackTrace() #1 0x0000005a05ca logging::LogMessage::~LogMessage() #2 0x000000849901 KeepAliveRegistry::Register() #3 0x0000008c6728 ProfileNotification::ProfileNotification() #4 0x000000938f60 MessageCenterNotificationManager::Add() #5 0x0000058e6e32 extensions::NotificationsApiFunction::CreateNotification() #6 0x0000058e8181 extensions::NotificationsCreateFunction::RunNotificationsApi() #7 0x0000058e7f06 extensions::NotificationsApiFunction::RunAsync() #8 0x000002fc4f6b ChromeAsyncExtensionFunction::Run() #9 0x000003240d70 ExtensionFunction::RunWithValidation() #10 0x000003243843 extensions::ExtensionFunctionDispatcher::DispatchWithCallbackInternal() #11 0x0000032433ff extensions::ExtensionFunctionDispatcher::Dispatch() #12 0x00000325fb13 _ZN3IPC8MessageTI29ExtensionHostMsg_Request_MetaSt5tupleIJ31ExtensionHostMsg_Request_ParamsEEvE8DispatchIN10extensions28ExtensionWebContentsObserverES8_N7content15RenderFrameHostEMS8_FvPSA_RKS3_EEEbPKNS_7MessageEPT_PT0_PT1_T2_ #13 0x00000325fa5a extensions::ExtensionWebContentsObserver::OnMessageReceived() #14 0x000002fc5e7d extensions::ChromeExtensionWebContentsObserver::OnMessageReceived() #15 0x00000198ad72 content::WebContentsImpl::OnMessageReceived() #16 0x000001843f71 content::RenderFrameHostImpl::OnMessageReceived() #17 0x0000018d6bd6 content::RenderProcessHostImpl::OnMessageReceived() #18 0x0000014c4e85 IPC::ChannelProxy::Context::OnDispatchMessage() #19 0x0000005e4009 base::debug::TaskAnnotator::RunTask() #20 0x0000005a36c5 base::MessageLoop::RunTask() #21 0x0000005a39b8 base::MessageLoop::DeferOrRunPendingTask() #22 0x0000005a3ceb base::MessageLoop::DoWork() #23 0x0000005dd759 base::MessagePumpLibevent::Run() #24 0x0000005a320d base::MessageLoop::RunHandler() #25 0x0000005b5f60 base::RunLoop::Run() #26 0x0000036cb16c content::RunThisRunLoop() #27 0x0000036cb10a content::RunMessageLoop() #28 0x000000974fe1 InProcessBrowserTest::QuitBrowsers() #29 0x000000974d85 InProcessBrowserTest::RunTestOnMainThreadLoop() #30 0x000000975fa4 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() #31 0x0000038f7b99 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() #32 0x0000038f6c91 ChromeBrowserMainParts::PreMainMessageLoopRun() #33 0x000002d85291 chromeos::ChromeBrowserMainPartsChromeos::PreMainMessageLoopRun() #34 0x000001a4e685 content::BrowserMainLoop::PreMainMessageLoopRun() #35 0x000001b95c57 content::StartupTaskRunner::RunAllTasksNow() #36 0x000001a4ced4 content::BrowserMainLoop::CreateStartupTasks() #37 0x000001a50e32 content::BrowserMainRunnerImpl::Initialize() #38 0x0000040c51f6 content::BrowserMain() #39 0x0000036c7f2e content::ContentMainRunnerImpl::Run() #40 0x0000036c6910 content::ContentMain() #41 0x000000975958 content::BrowserTestBase::SetUp() #42 0x000000973fdb InProcessBrowserTest::SetUp() #43 0x00000097c108 testing::Test::Run() #44 0x00000097cb63 testing::TestInfo::Run() #45 0x00000097cff3 testing::TestCase::Run() #46 0x0000009841c9 testing::internal::UnitTestImpl::RunAllTests() #47 0x000000983e6e testing::UnitTest::Run() #48 0x0000038eb7a6 base::TestSuite::Run() #49 0x000000971ffb ChromeTestSuiteRunner::RunTestSuite() #50 0x0000036c838e content::LaunchTests() #51 0x0000005935b7 main #52 0x7f9d42ada7ed __libc_start_main #53 0x00000043d72d <unknown> I think this was caused by 5faf50626a19355c80218096e55a9c709634f241 in Issue 585080. Marking P1 because this is causing flaky failures on the commit queue.
,
Jun 28 2016
Did it recently become that flaky? The change landed weeks ago. Looking at the trace, the tests posted some callback to create a notification, but that didn't run until the end of the test case. Then when tearing down the test environment, we ran those callbacks, that attempted to create the notifcation while we are shutting down. It now hits a DCHECK, instead of being completely ignored as before. I'm not familiar with ChromeVox tests, is there a way to make them run pending tasks on TearDown()? I think this should be the fix. For more context: Notifications can now keep the browser running when everything else is closed (browser windows, apps, etc.). Because they can now do that, we check that we didn't start shutting down before creating one. A lot of tests were very liberal in their notification handling, hence issues like this one. I thought I fixed them though...
,
Jun 28 2016
I haven't checked the flakiness history of the test. It's quite possible the flakiness has been there since 5faf50626a19355c80218096e55a9c709634f241 landed, and nobody bothered to file a bug. These tests don't seem to run for all commits going into Chromium; the "analyze" step seems to filter them out most of the time. Please find the owner of this test (git blame) and work with them to get them fixed. Thanks.
,
Jun 29 2016
Issue 624116 has been merged into this issue.
,
Jun 30 2016
It looks like dtseng@ is the principal author of the ChromeVox tests along with flackr@. David, could you take and investigate this bug? The chromevox_tests are flaky on the linux_chromium_chromeos_ozone_rel_ng tryserver, causing random CLs to be rejected from the commit queue: https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng?numbuilds=200
,
Jun 30 2016
I can take a look.
,
Jun 30 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/258b0c4bcf5b0d277ee34d883f69ba15219f29e3 commit 258b0c4bcf5b0d277ee34d883f69ba15219f29e3 Author: dtseng <dtseng@chromium.org> Date: Thu Jun 30 18:26:05 2016 Fix tests flakes. BUG= 623727 CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:closure_compilation Review-Url: https://codereview.chromium.org/2103053006 Cr-Commit-Position: refs/heads/master@{#403227} [modify] https://crrev.com/258b0c4bcf5b0d277ee34d883f69ba15219f29e3/chrome/browser/resources/chromeos/chromevox/cvox2/background/background_test.extjs [modify] https://crrev.com/258b0c4bcf5b0d277ee34d883f69ba15219f29e3/chrome/browser/resources/chromeos/chromevox/cvox2/background/notifications.js
,
Jun 30 2016
Thanks a lot for fixing this. I have another CL in review to address similar crashes in other places. It will hopefully land tomorrow or Monday.
,
Jul 2 2016
ClusterFuzz testcase is verified as fixed, closing issue. If this is incorrect, please add ClusterFuzz-Wrong label and re-open the issue.
,
Jul 4 2016
Issue 625339 has been merged into this issue.
,
Jul 4 2016
,
Jul 4 2016
The ChromeVox test flakiness is fixed, I filed issue 625646 to follow up on fixing the root cause.
,
Jul 26 2016
Issue 631737 has been merged into this issue. |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by kbr@chromium.org
, Jun 27 2016