Use-after-free of WebRtcEventLogManager created by BrowserProcessImpl |
||
Issue description
browser_process_impl.cc leaks the instance it creates in PreMainMessageLoopRun():
WebRtcEventLogManager* const webrtc_event_log_manager =
WebRtcEventLogManager::CreateSingletonInstance();
content::WebRtcEventLogger::Set(webrtc_event_log_manager);
As such running out\Release\unit_tests --gtest_filter=*Browser* makes ChromeContentBrowserClientWindowTest.IsDataSaverEnabled flake but it's not at fault. The problem is that previous tests left an instance behind which points to a |task_runner_| belonging to a deleted TaskScheduler.
("out\Release\unit_tests --gtest_filter=*Browser* --test-launcher-jobs=48" also works to repro faster)
https://chromium-review.googlesource.com/c/chromium/src/+/857515 made it such that WebRtcEventLogManager was a raw pointer managed manually rather than a LazyInstance::Leaky.
The CL documents that this is to address problems where "its task runner was not destroyed
between tests". Unfortunately the CL doesn't quite solve the issue either as if a caller doesn't do the manual invocation, WebRtcEventLogManager can still outlive its |task_runner_| (i.e. TaskScheduler == TestBrowserThreadBundle in this case).
To solve this there are a few options :
1) Make the singleton a LazyInstance::DestructorAtExit (each unit tests forces recycling of DestructorAtExit LazyInstances).
2) Use a LazySequenceTaskRunner in webrtc_event_log_manager.cc instead of a member variable, this will make sure the TaskRunner instance is recycled between tests
I like (1) because it nicely scopes the instance to each environment (each test) without requiring additional code in BrowserProcessImpl but I know dcheng@ may not like it per wanting to get rid of non-Leaky LazyInstances.
I'm not a fan of (2) as it is merely working around the problem of the instance sticking around to the next tests
A few stack traces caused by this:
[ RUN ] ChromeContentBrowserClientWindowTest.IsDataSaverEnabled
[20724:17268:0319/113349.717:611032937:FATAL:lock.cc(32)] Check failed: owning_thread_ref_.is_null().
Backtrace:
base::debug::StackTrace::StackTrace [0x66AEF970+32] (D:\src\chrome\src\base\debug\stack_trace_win.cc:286)
base::debug::StackTrace::StackTrace [0x66AEF08D+13] (D:\src\chrome\src\base\debug\stack_trace.cc:199)
logging::LogMessage::~LogMessage [0x66B16FD4+84] (D:\src\chrome\src\base\logging.cc:595)
base::Lock::CheckUnheldAndMark [0x66B8396E+78] (D:\src\chrome\src\base\synchronization\lock.cc:33)
base::internal::SchedulerLockImpl::Acquire [0x66B8DB0E+46] (D:\src\chrome\src\base\task_scheduler\scheduler_lock_impl.cc:138)
base::internal::TaskTracker::WillScheduleSequence [0x66B9A1CA+218] (D:\src\chrome\src\base\task_scheduler\task_tracker.cc:326)
base::internal::SchedulerWorkerPool::PostTaskWithSequenceNow [0x66B926ED+317] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:211)
base::internal::SchedulerWorkerPool::PostTaskWithSequence [0x66B92564+372] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:152)
base::internal::SchedulerSequencedTaskRunner::PostDelayedTask [0x66B92E47+199] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:105)
base::TaskRunner::PostTask [0x66B89B10+64] (D:\src\chrome\src\base\task_runner.cc:44)
WebRtcEventLogManager::DisableForBrowserContext [0x0381C1F3+389] (D:\src\chrome\src\chrome\browser\media\webrtc\webrtc_event_log_manager.cc:141)
content::BrowserContext::~BrowserContext [0x64DCE4C6+240] (D:\src\chrome\src\content\browser\browser_context.cc:591)
Profile::~Profile [0x03663204+148] (D:\src\chrome\src\chrome\browser\profiles\profile.cc:85)
TestingProfile::~TestingProfile [0x02CE183A+478] (D:\src\chrome\src\chrome\test\base\testing_profile.cc:553)
TestingProfile::~TestingProfile [0x02CE0F3B+11] (D:\src\chrome\src\chrome\test\base\testing_profile.cc:521)
ProfileDestroyer::DestroyProfileWhenAppropriate [0x03CE9A46+426] (D:\src\chrome\src\chrome\browser\profiles\profile_destroyer.cc:72)
ProfileManager::ProfileInfo::~ProfileInfo [0x03676872+20] (D:\src\chrome\src\chrome\browser\profiles\profile_manager.cc:1712)
std::unique_ptr<ProfileManager::ProfileInfo,std::default_delete<ProfileManager::ProfileInfo> >::~unique_ptr [0x02CE32DB+17] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\memory:2203)
std::_Tree<std::_Tmap_traits<base::FilePath,std::unique_ptr<ProfileManager::ProfileInfo,std::default_delete<ProfileManager::ProfileInfo> >,std::less<base::FilePath>,std::allocator<std::pair<const base::FilePath,std::unique_ptr<ProfileManager::ProfileInfo, [0x02CE32A6+44] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\xtree:1997)
std::_Tree<std::_Tmap_traits<base::FilePath,std::unique_ptr<ProfileManager::ProfileInfo,std::default_delete<ProfileManager::ProfileInfo> >,std::less<base::FilePath>,std::allocator<std::pair<const base::FilePath,std::unique_ptr<ProfileManager::ProfileInfo, [0x02CE2EC4+50] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\xtree:1366)
ProfileManager::~ProfileManager [0x03677539+179] (D:\src\chrome\src\chrome\browser\profiles\profile_manager.cc:388)
testing::ProfileManager::~ProfileManager [0x02CE2AED+11] (D:\src\chrome\src\chrome\test\base\testing_profile_manager.cc:32)
TestingBrowserProcess::SetProfileManager [0x02CDDFAF+45] (D:\src\chrome\src\chrome\test\base\testing_browser_process.cc:192)
TestingProfileManager::~TestingProfileManager [0x02CE18D7+29] (D:\src\chrome\src\chrome\test\base\testing_profile_manager.cc:55)
std::unique_ptr<TestingProfileManager,std::default_delete<TestingProfileManager> >::reset [0x014EAA54+22] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\memory:2238)
BrowserWithTestWindowTest::TearDown [0x01A57FB1+163] (D:\src\chrome\src\chrome\test\base\browser_with_test_window_test.cc:117)
testing::Test::Run [0x023A6E7C+242] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2493)
testing::TestInfo::Run [0x023A75EE+204] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2665)
testing::TestCase::Run [0x023A79D2+238] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2778)
testing::internal::UnitTestImpl::RunAllTests [0x023ADC72+632] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:5034)
testing::UnitTest::Run [0x023AD8FC+156] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:4652)
base::TestSuite::Run [0x02CECB28+104] (D:\src\chrome\src\base\test\test_suite.cc:284)
base::OnceCallback<int ()>::Run [0x025AE57F+45] (D:\src\chrome\src\base\callback.h:95)
base::`anonymous namespace'::LaunchUnitTestsInternal [0x02CEEC7F+274] (D:\src\chrome\src\base\test\launcher\unit_test_launcher.cc:225)
base::LaunchUnitTests [0x02CEEB44+176] (D:\src\chrome\src\base\test\launcher\unit_test_launcher.cc:576)
main [0x048A1480+324] (D:\src\chrome\src\chrome\test\base\run_all_unittests.cc:30)
__scrt_common_main_seh [0x048A9F49+248] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283)
BaseThreadInitThunk [0x767162C4+36]
RtlSubscribeWnfStateChangeNotification [0x77690F79+1081]
RtlSubscribeWnfStateChangeNotification [0x77690F44+1028]
[166/166] ChromeContentBrowserClientWindowTest.IsDataSaverEnabled (CRASHED)
[ RUN ] ChromeContentBrowserClientWindowTest.IsDataSaverEnabled
Received fatal exception EXCEPTION_ACCESS_VIOLATION
Backtrace:
std::vector<base::internal::TaskTracker::PreemptedBackgroundSequence,std::allocator<base::internal::TaskTracker::PreemptedBackgroundSequence> >::emplace_back<scoped_refptr<base::internal::Sequence>,base::TimeTicks,base::internal::CanScheduleSequenceObserv [0x66D4BB73+51] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\vector:940)
std::priority_queue<base::internal::TaskTracker::PreemptedBackgroundSequence,std::vector<base::internal::TaskTracker::PreemptedBackgroundSequence,std::allocator<base::internal::TaskTracker::PreemptedBackgroundSequence> >,std::greater<base::internal::TaskT [0x66D4A27C+44] (d:\src\chrome\src\third_party\depot_tools\win_toolchain\vs_files\1180cb75833ea365097e279efb2d5d7a42dee4b0\vc\tools\msvc\14.11.25503\include\queue:337)
base::internal::TaskTracker::WillScheduleSequence [0x66D4A208+280] (D:\src\chrome\src\base\task_scheduler\task_tracker.cc:332)
base::internal::SchedulerWorkerPool::PostTaskWithSequenceNow [0x66D426ED+317] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:211)
base::internal::SchedulerWorkerPool::PostTaskWithSequence [0x66D42564+372] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:152)
base::internal::SchedulerSequencedTaskRunner::PostDelayedTask [0x66D42E47+199] (D:\src\chrome\src\base\task_scheduler\scheduler_worker_pool.cc:105)
base::TaskRunner::PostTask [0x66D39B10+64] (D:\src\chrome\src\base\task_runner.cc:44)
WebRtcEventLogManager::EnableForBrowserContext [0x0381BF1C+442] (D:\src\chrome\src\chrome\browser\media\webrtc\webrtc_event_log_manager.cc:126)
content::BrowserContext::Initialize [0x64F7DFE5+1251] (D:\src\chrome\src\content\browser\browser_context.cc:534)
TestingProfile::Init [0x02CDEC91+139] (D:\src\chrome\src\chrome\test\base\testing_profile.cc:409)
TestingProfile::TestingProfile [0x02CDF436+384] (D:\src\chrome\src\chrome\test\base\testing_profile.cc:361)
TestingProfile::Builder::Build [0x02CE0F12+224] (D:\src\chrome\src\chrome\test\base\testing_profile.cc:1048)
TestingProfileManager::CreateTestingProfile [0x02CE1C39+279] (D:\src\chrome\src\chrome\test\base\testing_profile_manager.cc:99)
BrowserWithTestWindowTest::CreateProfile [0x01A58298+138] (D:\src\chrome\src\chrome\test\base\browser_with_test_window_test.cc:180)
BrowserWithTestWindowTest::SetUp [0x01A57E26+230] (D:\src\chrome\src\chrome\test\base\browser_with_test_window_test.cc:83)
testing::Test::Run [0x023A6DF3+105] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2481)
testing::TestInfo::Run [0x023A75EE+204] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2665)
testing::TestCase::Run [0x023A79D2+238] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:2778)
testing::internal::UnitTestImpl::RunAllTests [0x023ADC72+632] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:5034)
testing::UnitTest::Run [0x023AD8FC+156] (D:\src\chrome\src\third_party\googletest\src\googletest\src\gtest.cc:4652)
base::TestSuite::Run [0x02CECB28+104] (D:\src\chrome\src\base\test\test_suite.cc:284)
base::OnceCallback<int ()>::Run [0x025AE57F+45] (D:\src\chrome\src\base\callback.h:95)
base::`anonymous namespace'::LaunchUnitTestsInternal [0x02CEEC7F+274] (D:\src\chrome\src\base\test\launcher\unit_test_launcher.cc:225)
base::LaunchUnitTests [0x02CEEB44+176] (D:\src\chrome\src\base\test\launcher\unit_test_launcher.cc:576)
main [0x048A1480+324] (D:\src\chrome\src\chrome\test\base\run_all_unittests.cc:30)
__scrt_common_main_seh [0x048A9F49+248] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283)
BaseThreadInitThunk [0x767162C4+36]
RtlSubscribeWnfStateChangeNotification [0x77690F79+1081]
RtlSubscribeWnfStateChangeNotification [0x77690F44+1028]
[166/166] ChromeContentBrowserClientWindowTest.IsDataSaverEnabled (CRASHED)
,
Mar 19 2018
Found LazySequencedTaskRunner. (More after I've had some time to think.)
,
Mar 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7264d6a81f319b570fc24f7c2d20977da41d4ef3 commit 7264d6a81f319b570fc24f7c2d20977da41d4ef3 Author: Elad Alon <eladalon@chromium.org> Date: Wed Mar 21 15:06:26 2018 Destroy WebRtcEventLogManager from ~BrowserProcessImpl() Destroy WebRtcEventLogManager rather than leak it, so as to deflake unit tests. Bug: 823318 Change-Id: Ibe255217eafe13569e1d86091dc71929adbaf646 Reviewed-on: https://chromium-review.googlesource.com/968874 Commit-Queue: Elad Alon <eladalon@chromium.org> Reviewed-by: Jochen Eisinger <jochen@chromium.org> Reviewed-by: Tommi <tommi@chromium.org> Reviewed-by: Gabriel Charette <gab@chromium.org> Cr-Commit-Position: refs/heads/master@{#544704} [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/browser_process_impl.cc [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/browser_process_impl.h [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/media/webrtc/webrtc_event_log_manager.cc [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/media/webrtc/webrtc_event_log_manager.h [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/media/webrtc/webrtc_event_log_manager_unittest.cc [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/chrome/browser/media/webrtc/webrtc_event_log_uploader.cc [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/content/public/browser/webrtc_event_logger.cc [modify] https://crrev.com/7264d6a81f319b570fc24f7c2d20977da41d4ef3/content/public/browser/webrtc_event_logger.h
,
Apr 5 2018
|
||
►
Sign in to add a comment |
||
Comment 1 by eladalon@chromium.org
, Mar 19 2018