Issue metadata
Sign in to add a comment
|
KitchenSync Mac and Linux Dev Waterfalls are broken due to time outing tests (TryDismissSyncConfirmationDialog) |
||||||||||||||||||||||
Issue descriptionA bunch of tests timeout on KitchenSync Mac waterfall. Sample build: https://uberchromegw.corp.google.com/i/internal.client.kitchensync/builders/Mac%20Dev/builds/15545 These tests time out: TwoClientAppsSyncTest.UninstallThenInstall_E2ETest TwoClientAppsSyncTest.UpdateCWSOrdinals_E2ETest TwoClientAppsSyncTest.UpdateAppLaunchOrdinal_E2ETest TwoClientPasswordsSyncTest.E2ETest_TwoClientAddPass TwoClientAppsSyncTest.Uninstall_E2ETest TwoClientPasswordsSyncTest.Merge_E2ETest SingleClientBookmarksSyncTest.E2ETest_SanitySetup TwoClientExtensionsSyncTest.InstallDifferentExtensions_E2ETest TwoClientPasswordsSyncTest.E2ETest_Delete TwoClientAppsSyncTest.StartWithNoApps_E2ETest TwoClientBookmarksSyncTest.AddBookmarkWhileDisabled_E2ETest TwoClientBookmarksSyncTest.E2ETest_TwoClientsAddBookmarks TwoClientAutofillSyncTest.E2ETest_TwoClientsAddAutofillProfiles TwoClientAppsSyncTest.StartWithSameApps_E2ETest TwoClientAppsSyncTest.Merge_E2ETest TwoClientExtensionsSyncTest.StartWithSameExtensions_E2ETest TwoClientAppsSyncTest.InstallDifferentApps_E2ETest TwoClientAppsSyncTest.Add_E2ETest TwoClientExtensionsSyncTest.UninstallPreinstalledExtensions_E2ETest TwoClientAppsSyncTest.UpdateEnableDisableApp_E2ETest TwoClientExtensionsSyncTest.StartWithNoExtensions_E2ETest TwoClientPasswordsSyncTest.SetPassphraseAndAddPassword_E2ETest TwoClientPreferencesSyncTest.E2ETest_Sanity TwoClientAppsSyncTest.UpdateLaunchType_E2ETest TwoClientBookmarksSyncTest.E2ETest_OneClientAddsBookmark TwoClientPasswordsSyncTest.Add_E2ETest TwoClientExtensionsSyncTest.UpdateIncognitoEnableDisable_E2ETest TwoClientAppsSyncTest.UpdateIncognitoEnableDisable_E2ETest TwoClientBookmarksSyncTest.E2ETest_SanitySetup These tests crash (but I will create another bug for them): TwoClientAppsSyncTest.UpdatePageOrdinal_E2ETest TwoClientPasswordsSyncTest.Race_E2ETest The time out log: objc[16860]: Class RTCDispatcher is implemented in both /bot/w/ir/out/Debug/./libcontent.dylib and /bot/w/ir/out/Debug/libchrome_dll.dylib. One of the two will be used. Which one is undefined. [16847:18947:0111/200653.244330:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.362996:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.363582:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.544066:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.544660:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.545112:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.545550:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.546727:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.547714:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.548213:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.548699:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.549775:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.551331:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.551818:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space [16847:18947:0111/200653.552304:ERROR:mac_util.mm(142)] Couldn't get the sRGB color space BrowserTestBase received signal: Terminated: 15. Backtrace: 0 libbase.dylib 0x000000011816eede base::debug::StackTrace::StackTrace(unsigned long) + 174 1 libbase.dylib 0x000000011816ef9d base::debug::StackTrace::StackTrace(unsigned long) + 29 2 libbase.dylib 0x000000011816d41c base::debug::StackTrace::StackTrace() + 28 3 sync_integration_tests 0x000000010a623b6e content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 622 4 libsystem_platform.dylib 0x00007fff8ef765aa _sigtramp + 26 5 ??? 0x00007fab9d9173f8 0x0 + 140375059690488 6 CoreFoundation 0x00007fff915a1f15 __CFRunLoopServiceMachPort + 181 7 CoreFoundation 0x00007fff915a1539 __CFRunLoopRun + 1161 8 CoreFoundation 0x00007fff915a0e75 CFRunLoopRunSpecific + 309 9 HIToolbox 0x00007fff8a705a0d RunCurrentEventLoopInMode + 226 10 HIToolbox 0x00007fff8a7057b7 ReceiveNextEventCommon + 479 11 HIToolbox 0x00007fff8a7055bc _BlockUntilNextEventMatchingListInModeWithFilter + 65 12 AppKit 0x00007fff87e4324e _DPSNextEvent + 1434 13 AppKit 0x00007fff87e4289b -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 122 14 sync_integration_tests 0x00000001038fb96a __71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 106 15 libbase.dylib 0x000000011820b6ca base::mac::CallWithEHFrame(void () block_pointer) + 10 16 sync_integration_tests 0x00000001038fb808 -[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 248 17 AppKit 0x00007fff87e3699c -[NSApplication run] + 553 18 libbase.dylib 0x000000011826a8e2 base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 354 19 libbase.dylib 0x0000000118267764 base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 116 20 libbase.dylib 0x0000000118259bc3 base::MessageLoop::Run(bool) + 579 21 libbase.dylib 0x000000011835a7f1 base::RunLoop::Run() + 593 22 sync_integration_tests 0x000000010a66f244 content::RunThisRunLoop(base::RunLoop*) + 52 23 sync_integration_tests 0x000000010a670f8f content::MessageLoopRunner::Run() + 239 24 sync_integration_tests 0x000000010a63181d content::DOMMessageQueue::WaitForMessage(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) + 349 25 sync_integration_tests 0x000000010a62c51b content::(anonymous namespace)::ExecuteScriptHelper(content::RenderFrameHost*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::unique_ptr<base::Value, std::__1::default_delete<base::Value> >*) + 603 26 sync_integration_tests 0x000000010a62b2b9 content::ExecuteScriptAndExtractString(content::ToRenderFrameHost const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) + 393 27 sync_integration_tests 0x000000010a7a6739 login_ui_test_utils::SigninViewControllerTestUtil::TryDismissSyncConfirmationDialog(Browser*) + 937 28 sync_integration_tests 0x000000010a7a631d login_ui_test_utils::DismissSyncConfirmationDialog(Browser*, base::TimeDelta) + 125 29 sync_integration_tests 0x0000000102e2480a ProfileSyncServiceHarness::SetupSync(syncer::EnumSet<syncer::ModelType, (syncer::ModelType)2, (syncer::ModelType)39>, bool) + 2922 30 sync_integration_tests 0x0000000102e23a8f ProfileSyncServiceHarness::SetupSync() + 79 31 sync_integration_tests 0x0000000102e73be6 SyncTest::SetupSync() + 822 32 sync_integration_tests 0x0000000100600983 TwoClientPasswordsSyncTest_DISABLED_E2ETest_Delete_Test::RunTestOnMainThread() + 51 33 sync_integration_tests 0x000000010a62364b content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 491 34 sync_integration_tests 0x000000010a625e7d void base::internal::FunctorTraits<void (content::BrowserTestBase::*)(), void>::Invoke<content::BrowserTestBase*>(void (content::BrowserTestBase::*)(), content::BrowserTestBase*&&) + 125 35 sync_integration_tests 0x000000010a625dc4 void base::internal::InvokeHelper<false, void>::MakeItSo<void (content::BrowserTestBase::* const&)(), content::BrowserTestBase*>(void (content::BrowserTestBase::* const&&&)(), content::BrowserTestBase*&&) + 68 36 sync_integration_tests 0x000000010a625d53 void base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::RunImpl<void (content::BrowserTestBase::* const&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&, 0ul>(void (content::BrowserTestBase::* const&&&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserTestBase> > const&&&, std::__1::integer_sequence<unsigned long, 0ul>) + 99 37 sync_integration_tests 0x000000010a625c8c base::internal::Invoker<base::internal::BindState<void (content::BrowserTestBase::*)(), base::internal::UnretainedWrapper<content::BrowserTestBase> >, void ()>::Run(base::internal::BindStateBase*) + 44 38 sync_integration_tests 0x000000010087dc8d base::RepeatingCallback<void ()>::Run() const & + 61 39 sync_integration_tests 0x0000000103908417 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 8391 40 sync_integration_tests 0x000000010390629d ChromeBrowserMainParts::PreMainMessageLoopRun() + 253 41 libcontent.dylib 0x000000011f7b21f1 content::BrowserMainLoop::PreMainMessageLoopRun() + 337 42 libcontent.dylib 0x000000011f7b97dd int base::internal::FunctorTraits<int (content::BrowserMainLoop::*)(), void>::Invoke<content::BrowserMainLoop*>(int (content::BrowserMainLoop::*)(), content::BrowserMainLoop*&&) + 125 43 libcontent.dylib 0x000000011f7b9724 int base::internal::InvokeHelper<false, int>::MakeItSo<int (content::BrowserMainLoop::* const&)(), content::BrowserMainLoop*>(int (content::BrowserMainLoop::* const&&&)(), content::BrowserMainLoop*&&) + 68 44 libcontent.dylib 0x000000011f7b96b3 int base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::RunImpl<int (content::BrowserMainLoop::* const&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&, 0ul>(int (content::BrowserMainLoop::* const&&&)(), std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > const&&&, std::__1::integer_sequence<unsigned long, 0ul>) + 99 45 libcontent.dylib 0x000000011f7b95bc base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::Run(base::internal::BindStateBase*) + 44 46 libcontent.dylib 0x0000000120bb022d base::RepeatingCallback<int ()>::Run() const & + 61 47 libcontent.dylib 0x0000000120bb0100 content::StartupTaskRunner::RunAllTasksNow() + 528 48 libcontent.dylib 0x000000011f7ad254 content::BrowserMainLoop::CreateStartupTasks() + 2260 49 libcontent.dylib 0x000000011f7bcdac content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 1980 50 libcontent.dylib 0x000000011f7a50ea content::BrowserMain(content::MainFunctionParams const&) + 554 51 libcontent.dylib 0x00000001223dbbdb content::RunNamedProcessTypeMain(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, content::MainFunctionParams const&, content::ContentMainDelegate*) + 603 52 libcontent.dylib 0x00000001223dd42b content::ContentMainRunnerImpl::Run() + 1323 53 libcontent.dylib 0x00000001223d41e5 content::ContentServiceManagerMainDelegate::RunEmbedderProcess() + 53 54 libembedder.dylib 0x000000012fc45ab0 service_manager::Main(service_manager::MainParams const&) + 2128 55 libcontent.dylib 0x00000001223db939 content::ContentMain(content::ContentMainParams const&) + 89 56 sync_integration_tests 0x000000010a6231f0 content::BrowserTestBase::SetUp() + 7632 57 sync_integration_tests 0x0000000102e98ec1 InProcessBrowserTest::SetUp() + 1969 58 sync_integration_tests 0x0000000102e69de3 SyncTest::SetUp() + 4803 59 sync_integration_tests 0x00000001006d303e void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 126 60 sync_integration_tests 0x00000001006ac2f2 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 114 61 sync_integration_tests 0x00000001006ac1d4 testing::Test::Run() + 116 [28/57] TwoClientPasswordsSyncTest.DISABLED_E2ETest_Delete (TIMED OUT) I don't see any issue immediately, may it be that tests just need more time on Mac?
,
Jan 12 2018
Any idea?
,
Jan 17 2018
Marc, could you try to repro it on you Mac, please?
,
Jan 17 2018
,
Jan 17 2018
As the resident Mac-machine-haver, I'll take a look :)
,
Jan 17 2018
Also, it seems that this time out is controlled via arguments to the tests suit (i.e. --test-launcher-timeout=120000 --ui-test-action-timeout=120000). Thus, if the required increase is not huge, we could probably increase them on the bot and file another bug to investigate. This will likely require asking infrastructure folks.
,
Jan 17 2018
,
Jan 17 2018
I think the relevant line in the log is: login_ui_test_utils::DismissSyncConfirmationDialog I've observed locally that the tests sometimes hang with the "You've signed in and turned on Sync" message showing. So apparently dismissing that dialog is flaky.
,
Jan 17 2018
It seems so. In the last builds on Mac [1] and Linux Dev [2] (both include the fix to issue 801521 - the typed URLs timeout increase), I observe tests timing out. On Linux - 30 tests time out, 24 have login_ui_test_utils::DismissSyncConfirmationDialog() in the stack trace. On Max - 29 tests time out, 17 have login_ui_test_utils::DismissSyncConfirmationDialog in the stack trace. [1] https://uberchromegw.corp.google.com/i/internal.client.kitchensync/builders/Mac%20Dev/builds/15551 [2] https://uberchromegw.corp.google.com/i/internal.client.kitchensync/builders/Linux%20Dev/builds/33057
,
Jan 17 2018
The problem seems to be in SigninViewControllerTestUtil::TryDismissSyncConfirmationDialog. It runs JS to detect if an element with the ID 'confirmButton' exists, and if it does, runs .click() on it. Sometimes that doesn't work, and then confirmation_closed_observer.WaitForConfirmationClosed() in DismissSyncConfirmationDialog waits forever. I haven't figured out why yet. First theory: Maybe the HTML element is loaded, but its JS onclick handler isn't attached yet?
,
Jan 17 2018
There was an attempt to fix it in https://chromium-review.googlesource.com/c/chromium/src/+/857467 by clicking the button in a separate js, but the CL was reverted in https://chromium-review.googlesource.com/c/chromium/src/+/870610.
,
Jan 17 2018
Their solution actually occasionally gets stuck locally. Checking "if (document.readyState === 'complete')" in the js before sending "ok" seems to work locally (together with their fix). I though that we could use 'interactive' instead of complete, because we don't need images (see https://stackoverflow.com/questions/978740/javascript-how-to-detect-if-document-has-loaded-ie-7-firefox-3), but it does not work locally.
,
Jan 17 2018
,
Jan 17 2018
,
Jan 18 2018
,
Jan 18 2018
The fix has been incorporated into https://chromium-review.googlesource.com/c/chromium/src/+/873070.
,
Jan 19 2018
These timeouts seem to be gone. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by vitaliii@chromium.org
, Jan 12 2018