Various sync_integration_tests failing on Mac-10.10 and 10.11 |
|||
Issue descriptionSee https://build.chromium.org/p/chromium.mac/builders/Mac10.10%20Tests and https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests. Most (but not all) of the builds since this morning have been failing a large, fairly random set of sync tests. First red builds: https://build.chromium.org/p/chromium.mac/builders/Mac10.11%20Tests/builds/11043 and https://build.chromium.org/p/chromium.mac/builders/Mac10.10%20Tests/builds/16612 The set of failing tests seems to differ from build to build. Here's a sample set from build 11043 on 10.11: TwoClientAppsSyncTest.UpdateCWSOrdinals_E2ETest TwoClientBookmarksSyncTest.SC_DuplicateBMWithDifferentURLSameName TwoClientTypedUrlsSyncTest.AddExpired MigrationSingleClientTest.PrefsOnlyModifyPref TwoClientPasswordsSyncTest.Race_E2ETest TwoClientAppsSyncTest.Merge_E2ETest SingleClientExtensionsSyncTest.StartWithNoExtensions TwoClientSearchEnginesSyncTest.UpdateKeyword_E2ETest TwoClientsPasswordManagerSettingMigratorServiceSyncTest.ChangeNewPrefTestBothClientsWithMigration_E2ETest TwoClientBookmarksSyncTest.SC_SinkBMs10LevelDown TwoClientBookmarksSyncTest.MC_DuplicateFolders TwoClientBookmarksSyncTest.MC_Merge_SimpleBMHierarchy_Under_BMFolder TwoClientAutofillSyncTest.NoCreditCardSync TwoClientAutofillSyncTest.UpdateFields TwoClientAppsSyncTest.StartWithSameApps_E2ETest TwoClientBookmarksSyncTest.MC_Merge_SimpleBMHierarchy_DisjointSets_Under_BookmarkBar TwoClientTypedUrlsSyncTest.SkipImportedVisits TwoClientBookmarksSyncTest.SC_SetFaviconHiDPI TwoClientBookmarksSyncTest.SC_RenameBMName TwoClientBookmarksSyncTest.SC_TwiceRenamingBookmarkURL SingleClientSearchEnginesSyncTest.Sanity SingleClientWalletSyncTest.ClearOnDisableWalletSync SingleClientTypedUrlsSyncTest.Sanity TwoClientAutofillSyncTest.AddMultipleProfiles MigrationSingleClientTest.AllTypesAtOnceTriggerNotification TwoClientThemesSyncTest.CustomThenSyncDefault_E2ETest TwoClientAppsSyncTest.InstallDifferentApps_E2ETest TwoClientAppsSyncTest.StartWithNoApps_E2ETest TwoClientPasswordsSyncTest.SetPassphraseAndAddPassword_E2ETest TwoClientBookmarksSyncTest.SC_DelMiddleBMUnderBMFoldNonEmptyFoldAfterwards TwoClientPreferencesSyncTest.Bidirectional_E2ETest TwoClientAppsSyncTest.UpdatePageOrdinal_E2ETest Sample failure output: [ RUN ] TwoClientAppsSyncTest.UpdateCWSOrdinals_E2ETest 2017-04-14 08:05:24.622 sync_integration_tests[21180:209549] NSWindow warning: adding an unknown subview: <FullSizeContentView: 0x7fac5264b8e0> 2017-04-14 08:05:24.623 sync_integration_tests[21180:209549] Call stack: ( "+callStackSymbols disabled for performance reasons" ) [08:05:24.679] vtDecompressionDuctCreate signalled err=-8973 (err) (Could not select and open decoder instance) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTDecompressionSession.c line 1181 <<<< VTVideoEncoderSelection >>>> VTSelectAndCreateVideoEncoderInstanceInternal: no video encoder found for 'avc1' [08:05:24.747] VTSelectAndCreateVideoEncoderInstanceInternal signalled err=-12908 (err) (Video encoder not available) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTVideoEncoderSelection.c line 1245 [08:05:24.747] VTCompressionSessionCreate signalled err=-12908 (err) (Could not select and open encoder instance) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTCompressionSession.c line 946 [21180:38403:0414/080525.829295:ERROR:service_manager.cc(425)] InterfaceProviderSpec prevented connection from: content_utility to: content_browser [21180:104199:0414/080525.926024:WARNING:syncer_proto_util.cc(338)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: SYNC_AUTH_ERROR [21180:104199:0414/080525.927174:WARNING:sync_encryption_handler_impl.cc(971)] Nigori had empty encryption keybag. [21180:104199:0414/080525.927732:WARNING:sync_encryption_handler_impl.cc(971)] Nigori had empty encryption keybag. [21180:104199:0414/080525.937826:WARNING:sync_encryption_handler_impl.cc(347)] Ignoring new implicit passphrase. Keystore migration already performed. [21180:109331:0414/080525.975516:WARNING:syncer_proto_util.cc(338)] Error posting from syncer: Response Code (bogus on error): -1 Content-Length (bogus on error): -1 Server Status: SYNC_AUTH_ERROR [21180:109331:0414/080525.982229:WARNING:sync_encryption_handler_impl.cc(347)] Ignoring new implicit passphrase. Keystore migration already performed. BrowserTestBase received signal: Segmentation fault: 11. Backtrace: 0 sync_integration_tests 0x00000001105e86dc base::debug::StackTrace::StackTrace(unsigned long) + 28 1 sync_integration_tests 0x0000000114a33d28 content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 200 2 libsystem_platform.dylib 0x00007fff8513bf1a _sigtramp + 26 3 libobjc.A.dylib 0x00007fff89714925 objc_object::sidetable_clearDeallocating() + 95 4 sync_integration_tests 0x0000000112f1ec9d -[BrowserWindowController .cxx_destruct] + 93 5 libobjc.A.dylib 0x00007fff896ffae8 object_cxxDestructFromClass(objc_object*, objc_class*) + 127 6 libobjc.A.dylib 0x00007fff896fa637 objc_destructInstance + 119 7 sync_integration_tests 0x00000001115776c9 (anonymous namespace)::ZombieDealloc(objc_object*, objc_selector*) + 89 8 AppKit 0x00007fff83f23cec -[NSResponder dealloc] + 139 9 AppKit 0x00007fff8419b376 -[NSWindowController dealloc] + 551 10 sync_integration_tests 0x0000000112fd6875 -[TabWindowController dealloc] + 85 11 sync_integration_tests 0x0000000112f1a65d -[BrowserWindowController dealloc] + 349 12 libobjc.A.dylib 0x00007fff8971489c objc_object::sidetable_release(bool) + 236 13 AppKit 0x00007fff84015524 -[NSWindowController release] + 154 14 Foundation 0x00007fff8bb93975 __delayedPerformCleanup + 46 15 CoreFoundation 0x00007fff89fb3c4d CFRunLoopTimerInvalidate + 605 16 CoreFoundation 0x00007fff89fb7fa9 __CFRunLoopDoTimer + 1113 17 CoreFoundation 0x00007fff8a02b53d __CFRunLoopDoTimers + 301 18 CoreFoundation 0x00007fff89f73608 __CFRunLoopRun + 2024 19 CoreFoundation 0x00007fff89f72bd8 CFRunLoopRunSpecific + 296 20 HIToolbox 0x00007fff8eb8756f RunCurrentEventLoopInMode + 235 21 HIToolbox 0x00007fff8eb871ee ReceiveNextEventCommon + 179 22 HIToolbox 0x00007fff8eb8712b _BlockUntilNextEventMatchingListInModeWithFilter + 71 23 AppKit 0x00007fff83f2f8ab _DPSNextEvent + 978 24 AppKit 0x00007fff83f2ee58 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 346 25 sync_integration_tests 0x00000001108eeae0 __71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 64 26 sync_integration_tests 0x0000000110605e4a base::mac::CallWithEHFrame(void () block_pointer) + 10 27 sync_integration_tests 0x00000001108eea24 -[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 164 28 AppKit 0x00007fff83f24af3 -[NSApplication run] + 594 29 sync_integration_tests 0x000000011061601e base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 334 30 sync_integration_tests 0x000000011061560c base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 92 31 sync_integration_tests 0x000000011061191e base::MessageLoop::RunHandler() + 94 32 sync_integration_tests 0x0000000110633f43 base::RunLoop::Run() + 115 33 sync_integration_tests 0x0000000114a378f2 content::WindowedNotificationObserver::Wait() + 194 34 sync_integration_tests 0x0000000110388f7d InProcessBrowserTest::CloseBrowserSynchronously(Browser*) + 77 35 sync_integration_tests 0x000000011037e20c SyncTest::TearDownOnMainThread() + 140 36 sync_integration_tests 0x0000000110389476 InProcessBrowserTest::RunTestOnMainThreadLoop() + 1238 37 sync_integration_tests 0x0000000114a33a86 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 310 38 sync_integration_tests 0x00000001108f3c6f ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 4415 39 sync_integration_tests 0x00000001108f2a2e ChromeBrowserMainParts::PreMainMessageLoopRun() + 62 40 sync_integration_tests 0x000000010f23ea73 content::BrowserMainLoop::PreMainMessageLoopRun() + 67 41 sync_integration_tests 0x000000010f57fe97 content::StartupTaskRunner::RunAllTasksNow() + 39 42 sync_integration_tests 0x000000010f23d072 content::BrowserMainLoop::CreateStartupTasks() + 658 43 sync_integration_tests 0x000000010f241314 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 756 44 sync_integration_tests 0x000000010f23aa34 content::BrowserMain(content::MainFunctionParams const&) + 100 45 sync_integration_tests 0x0000000110345c3b content::ContentMainRunnerImpl::Run() + 891 46 sync_integration_tests 0x0000000111dd9078 service_manager::Main(service_manager::MainParams const&) + 1032 47 sync_integration_tests 0x0000000110344f64 content::ContentMain(content::ContentMainParams const&) + 68 48 sync_integration_tests 0x0000000114a33762 content::BrowserTestBase::SetUp() + 1906 49 sync_integration_tests 0x0000000110388566 InProcessBrowserTest::SetUp() + 406 50 sync_integration_tests 0x000000011037ac3b SyncTest::SetUp() + 635 51 sync_integration_tests 0x000000011050cae1 testing::Test::Run() + 97 52 sync_integration_tests 0x000000011050d3b0 testing::TestInfo::Run() + 288 53 sync_integration_tests 0x000000011050d917 testing::TestCase::Run() + 263 54 sync_integration_tests 0x00000001105137b7 testing::internal::UnitTestImpl::RunAllTests() + 871 55 sync_integration_tests 0x0000000110513423 testing::UnitTest::Run() + 163 56 sync_integration_tests 0x00000001105d38b3 base::TestSuite::Run() + 163 57 sync_integration_tests 0x0000000110386a5f ChromeTestSuiteRunner::RunTestSuite(int, char**) + 31 58 sync_integration_tests 0x0000000114a34439 content::LaunchTests(content::TestLauncherDelegate*, int, int, char**) + 313 59 sync_integration_tests 0x0000000110358d3a main + 90 60 libdyld.dylib 0x00007fff90de05c9 start + 1 61 ??? 0x0000000000000008 0x0 + 8 The blamelist is long (looks like both bots weren't processing builds for a while before these), and nothing immediately jumps out, plus this could be an earlier build given that these failures are flaky. It could also be an infra issue. Adding some sync folks - can anyone help triage what this failure is about?
,
Apr 15 2017
Running on my mac, stack trace looks slightly different BrowserTestBase received signal: Segmentation fault: 11. Backtrace: 0 libbase.dylib 0x000000011dfb8d7e base::debug::StackTrace::StackTrace(unsigned long) + 174 1 libbase.dylib 0x000000011dfb8e1d base::debug::StackTrace::StackTrace(unsigned long) + 29 2 libbase.dylib 0x000000011dfb727c base::debug::StackTrace::StackTrace() + 28 3 sync_integration_tests 0x00000001185b5ee0 content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 560 4 libsystem_platform.dylib 0x00007fffb3e79b3a _sigtramp + 26 5 ??? 0x0000000000000001 0x0 + 1 6 sync_integration_tests 0x0000000116844def OmniboxPopupModel::RemoveObserver(OmniboxPopupModelObserver*) + 47 7 sync_integration_tests 0x000000011738101a ScopedObserver<OmniboxPopupModel, OmniboxPopupModelObserver>::RemoveAll() + 154 8 sync_integration_tests 0x0000000117380f39 ScopedObserver<OmniboxPopupModel, OmniboxPopupModelObserver>::~ScopedObserver() + 25 9 sync_integration_tests 0x000000011737fcf5 ScopedObserver<OmniboxPopupModel, OmniboxPopupModelObserver>::~ScopedObserver() + 21 10 sync_integration_tests 0x0000000117382337 (anonymous namespace)::OmniboxPopupModelObserverBridge::~OmniboxPopupModelObserverBridge() + 39 11 sync_integration_tests 0x0000000117382305 (anonymous namespace)::OmniboxPopupModelObserverBridge::~OmniboxPopupModelObserverBridge() + 21 12 sync_integration_tests 0x000000011737d326 -[BrowserWindowController .cxx_destruct] + 294 13 libobjc.A.dylib 0x00007fffb3380bb4 object_cxxDestructFromClass(objc_object*, objc_class*) + 127 14 libobjc.A.dylib 0x00007fffb33795f6 objc_destructInstance + 92 15 libzombies.dylib 0x000000012215952b (anonymous namespace)::ZombieDealloc(objc_object*, objc_selector*) + 395 16 AppKit 0x00007fff9bfeda2b -[NSResponder dealloc] + 139 17 AppKit 0x00007fff9c2ba989 -[NSWindowController dealloc] + 532 18 sync_integration_tests 0x000000011759978d -[TabWindowController dealloc] + 125 19 sync_integration_tests 0x000000011737101d -[BrowserWindowController dealloc] + 717 20 libobjc.A.dylib 0x00007fffb337f2e7 objc_object::sidetable_release(bool) + 285 21 AppKit 0x00007fff9c009645 -[NSWindowController release] + 155 22 Foundation 0x00007fff9ff570d4 __delayedPerformCleanup + 50 23 CoreFoundation 0x00007fff9e507097 CFRunLoopTimerInvalidate + 647 24 CoreFoundation 0x00007fff9e5069c6 __CFRunLoopDoTimer + 1126 25 CoreFoundation 0x00007fff9e5064ea __CFRunLoopDoTimers + 298 26 CoreFoundation 0x00007fff9e4fdc31 __CFRunLoopRun + 2065 27 CoreFoundation 0x00007fff9e4fd1c4 CFRunLoopRunSpecific + 420 28 HIToolbox 0x00007fff9da5eebc RunCurrentEventLoopInMode + 240 29 HIToolbox 0x00007fff9da5ecf1 ReceiveNextEventCommon + 432 30 HIToolbox 0x00007fff9da5eb26 _BlockUntilNextEventMatchingListInModeWithFilter + 71 31 AppKit 0x00007fff9bff9e24 _DPSNextEvent + 1120 32 AppKit 0x00007fff9c77585e -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796 33 sync_integration_tests 0x000000011217361a __71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 106 34 libbase.dylib 0x000000011e0590fa base::mac::CallWithEHFrame(void () block_pointer) + 10 35 sync_integration_tests 0x00000001121734b8 -[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 248 36 AppKit 0x00007fff9bfee7ab -[NSApplication run] + 926 37 libbase.dylib 0x000000011e0bd0b2 base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 306 38 libbase.dylib 0x000000011e0bc09a base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 122 39 libbase.dylib 0x000000011e0a8cc2 base::MessageLoop::RunHandler() + 610 40 libbase.dylib 0x000000011e1876db base::RunLoop::Run() + 267 41 sync_integration_tests 0x00000001185d5584 content::RunThisRunLoop(base::RunLoop*) + 52 42 sync_integration_tests 0x00000001185d5511 content::RunMessageLoop() + 49 43 sync_integration_tests 0x000000011167871e InProcessBrowserTest::QuitBrowsers() + 206 44 sync_integration_tests 0x0000000111678525 InProcessBrowserTest::RunTestOnMainThreadLoop() + 2517 45 sync_integration_tests 0x00000001185b5a38 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 408 46 sync_integration_tests 0x00000001185b76e5 void base::internal::FunctorTraits<void (content::BrowserTestBase::*)(), void>::Invoke<content::BrowserTestBase*>(void (content::BrowserTestBase::*)(), content::BrowserTestBase*&&) + 133 47 sync_integration_tests 0x00000001185b7624 void base::internal::InvokeHelper<false, void>::MakeItSo<void (content::BrowserTestBase::* const&)(), content::BrowserTestBase*>(void (content::BrowserTestBase::* const&&&)(), content::BrowserTestBase*&&) + 68 48 sync_integration_tests 0x00000001185b75b3 void base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::RunImpl<void (content::BrowserTestBase::* const&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&, 0ul>(void (content::BrowserTestBase::* const&&&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&&&, base::IndexSequence<0ul>) + 99 49 sync_integration_tests 0x00000001185b74ec base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::Run(base::internal::BindStateBase*) + 44 50 sync_integration_tests 0x000000010fa0bc8d base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1>::Run() const + 61 51 sync_integration_tests 0x000000011217eeba ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 6986 52 sync_integration_tests 0x000000011217d2bd ChromeBrowserMainParts::PreMainMessageLoopRun() + 253 53 libcontent.dylib 0x00000001237acc71 content::BrowserMainLoop::PreMainMessageLoopRun() + 337 54 libcontent.dylib 0x00000001237b40a5 int base::internal::FunctorTraits<int (content::BrowserMainLoop::*)(), void>::Invoke<content::BrowserMainLoop*>(int (content::BrowserMainLoop::*)(), content::BrowserMainLoop*&&) + 133 55 libcontent.dylib 0x00000001237b3fe4 int base::internal::InvokeHelper<false, int>::MakeItSo<int (content::BrowserMainLoop::* const&)(), content::BrowserMainLoop*>(int (content::BrowserMainLoop::* const&&&)(), content::BrowserMainLoop*&&) + 68 56 libcontent.dylib 0x00000001237b3f73 int base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::RunImpl<int (content::BrowserMainLoop::* const&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&, 0ul>(int (content::BrowserMainLoop::* const&&&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&&&, base::IndexSequence<0ul>) + 99 57 libcontent.dylib 0x00000001237b3e7c base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::Run(base::internal::BindStateBase*) + 44 58 libcontent.dylib 0x000000012487833d base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1>::Run() const + 61 59 libcontent.dylib 0x00000001248c78d0 content::StartupTaskRunner::RunAllTasksNow() + 528 60 libcontent.dylib 0x00000001237a93c9 content::BrowserMainLoop::CreateStartupTasks() + 1785 61 libcontent.dylib 0x00000001237b7dbf content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 2095 [0414/171106.654444:ERROR:kill_posix.cc(84)] Unable to terminate process group 71863: No such process [15/17] TwoClientAutofillSyncTest.MaxLength (2373 ms)
,
Apr 15 2017
https://chromium.googlesource.com/chromium/src/+/d1040f6d95719c52a3ec442c76be29d5c0630bf4 seems to have created a class that we seem to be UAFing OmniboxPopupModel, given my crash stack. Running out/Default/sync_integration_tests --gtest_filter=TwoClientAutofillSyncTest.* on ToT I see 5-10 cases crash out of 17. Reverting d1040f6d95719c52a3ec442c76be29d5c0630bf4 locally, I see all 17/17 pass. I don't understand what exactly owns the OmniboxPopupModel and how it's life cycle is related to the BrowserWindowController. The times match up as well. This CL landed Apr 14 07:28:34 2017, first failure we saw was Apr 14 07:57. Reverting.
,
Apr 17 2017
,
Apr 17 2017
Issue 711899 has been merged into this issue. Issue 711900 has been merged into this issue. Issue 711909 has been merged into this issue. Issue 711918 has been merged into this issue. Issue 711919 has been merged into this issue. Issue 711924 has been merged into this issue. Issue 711925 has been merged into this issue. Issue 711931 has been merged into this issue. Issue 711933 has been merged into this issue. Issue 711939 has been merged into this issue.
,
Aug 2 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by s...@chromium.org
, Apr 14 2017Status: Started (was: Untriaged)