New issue
Advanced search Search tips

Issue 869969 link

Starred by 1 user
Status: Assigned
Owner:
robliao@chromium.org
Cc:
mac-bugs-priority@chromium.org
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug



Sign in to add a comment

interactive_ui_tests regularly time out inside ui_test_utils::BringBrowserWindowToFront on Mac 10.10 tester

Project Member Reported by pbos@chromium.org, Aug 1 
See https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.10%20Tests

Example run:

https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8939379509084281136/+/steps/interactive_ui_tests_on_Mac-10.10/0/stdout

Stack trace @ fail:

BrowserTestBase received signal: Terminated: 15. Backtrace:
0   interactive_ui_tests                0x000000010367eedc base::debug::StackTrace::StackTrace(unsigned long) + 28
1   interactive_ui_tests                0x000000010307b4f5 content::(anonymous namespace)::DumpStackTraceSignalHandler(int) + 229
2   libsystem_platform.dylib            0x00007fff91244f1a _sigtramp + 26
3   interactive_ui_tests                0x0000000107f86977 (anonymous namespace)::kHeader + 5495
4   CoreFoundation                      0x00007fff8da7feb4 __CFRunLoopServiceMachPort + 212
5   CoreFoundation                      0x00007fff8da7f37b __CFRunLoopRun + 1371
6   CoreFoundation                      0x00007fff8da7ebd8 CFRunLoopRunSpecific + 296
7   HIToolbox                           0x00007fff8e4ff56f RunCurrentEventLoopInMode + 235
8   HIToolbox                           0x00007fff8e4ff2ea ReceiveNextEventCommon + 431
9   HIToolbox                           0x00007fff8e4ff12b _BlockUntilNextEventMatchingListInModeWithFilter + 71
10  AppKit                              0x00007fff835768ab _DPSNextEvent + 978
11  AppKit                              0x00007fff83575e58 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 346
12  interactive_ui_tests                0x0000000102b2a590 __71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 64
13  interactive_ui_tests                0x00000001035e01ca base::mac::CallWithEHFrame(void () block_pointer) + 10
14  interactive_ui_tests                0x0000000102b2a4c4 -[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 164
15  AppKit                              0x00007fff8356baf3 -[NSApplication run] + 594
16  interactive_ui_tests                0x00000001035efc9c base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 364
17  interactive_ui_tests                0x00000001035ee7de base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 110
18  interactive_ui_tests                0x0000000103612b95 base::RunLoop::Run() + 53
19  interactive_ui_tests                0x0000000100a486f5 ui_test_utils::BringBrowserWindowToFront(Browser const*) + 149
20  interactive_ui_tests                0x000000010092b29d extensions::(anonymous namespace)::BrowserActionInteractiveTest::SetUpOnMainThread() + 173
21  interactive_ui_tests                0x000000010307ad9c content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 460
22  interactive_ui_tests                0x0000000102b304d7 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 4375
23  interactive_ui_tests                0x0000000102b2f2be ChromeBrowserMainParts::PreMainMessageLoopRun() + 62
24  interactive_ui_tests                0x00000001016ac4c3 content::BrowserMainLoop::PreMainMessageLoopRun() + 67
25  interactive_ui_tests                0x0000000101aa8927 content::StartupTaskRunner::RunAllTasksNow() + 39
26  interactive_ui_tests                0x00000001016aac68 content::BrowserMainLoop::CreateStartupTasks() + 712
27  interactive_ui_tests                0x00000001016ae9e0 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 96
28  interactive_ui_tests                0x00000001016a8da4 content::BrowserMain(content::MainFunctionParams const&) + 180
29  interactive_ui_tests                0x0000000102aac968 content::ContentMainRunnerImpl::Run(bool) + 824
30  interactive_ui_tests                0x000000010473a631 service_manager::Main(service_manager::MainParams const&) + 2849
31  interactive_ui_tests                0x0000000102aab9b4 content::ContentMain(content::ContentMainParams const&) + 68
32  interactive_ui_tests                0x000000010307a9e9 content::BrowserTestBase::SetUp() + 2697
33  interactive_ui_tests                0x0000000102ad9e93 InProcessBrowserTest::SetUp() + 723
34  interactive_ui_tests                0x0000000100b3036d testing::Test::Run() + 109
35  interactive_ui_tests                0x0000000100b31000 testing::TestInfo::Run() + 320
36  interactive_ui_tests                0x0000000100b31577 testing::TestCase::Run() + 279
37  interactive_ui_tests                0x0000000100b3cba7 testing::internal::UnitTestImpl::RunAllTests() + 871
38  interactive_ui_tests                0x0000000100b3c81d testing::UnitTest::Run() + 109
39  interactive_ui_tests                0x00000001035b9246 base::TestSuite::Run() + 166
40  interactive_ui_tests                0x0000000100a49763 InteractiveUITestSuiteRunner::RunTestSuite(int, char**) + 51
41  interactive_ui_tests                0x0000000103081d58 content::LaunchTests(content::TestLauncherDelegate*, unsigned long, int, char**) + 552
42  interactive_ui_tests                0x0000000102ad23bd LaunchChromeTests(unsigned long, content::TestLauncherDelegate*, int, char**) + 333

Comment 1 by pbos@chromium.org, Aug 1 

I don't know which stack frames are missing (being inlined) between:
18  interactive_ui_tests                0x0000000103612b95 base::RunLoop::Run() + 53
19  interactive_ui_tests                0x0000000100a486f5 ui_test_utils::BringBrowserWindowToFront(Browser const*) + 149

But ui_test_utils::BringBrowserWindowToFront is either stuck inside: ShowAndFocusNativeWindow or BrowserActivationWaiter waiting for the browser to become the topmost window.

I think ShowAndFocusNativeWindow is blocking on Mac: https://cs.chromium.org/chromium/src/chrome/test/base/interactive_test_utils_mac.mm?type=cs&q=ShowAndFocusNativeWindow+file:mm&g=0&l=153

The BrowserActivationWaiter either returns on construction (if the browser window is already active), otherwise it runs a runloop that gets quitted inside BrowserActivationWaiter::OnBrowserSetLastActive

I guess RunLoop::RunUntilIdle() could also be racy in case it never ends up getting idle.

Comment 2 by pbos@chromium.org, Aug 1

Owner: robliao@chromium.org
Status: Assigned (was: Untriaged)
robliao@'s agreed to take a look, because nothing's been disabled still keeping it on the sheriffs' radar (as the tests are still actively flaking).

Comment 3 by engedy@chromium.org, Aug 3 

Still observing failures around 1/3 of the time.

Comment 4 by pkasting@chromium.org, Aug 11 

This still seems to be failing pretty regularly.  It doesn't fail every run, but every few runs; then when it does fail, dozens of tests fail with similar stacks.

This makes me wonder if there's some sort of bad state on the bot, e.g. a window that gets stuck open or something, and every time we get into that state, all the tests that use this start failing.  I know we've had issues like that on Windows before.

Rob, any chance of getting to this soon?

Comment 5 by xidac...@chromium.org, Aug 15

Labels: -Sheriff-Chromium

Comment 6 by s...@chromium.org, Aug 22 

These flakes are still commonly happening.
►

Sign in to add a comment