Issue 639616  has been merged into this issue.

 Issue 639615  has been merged into this issue.

 Issue 639547  has been merged into this issue.

Locally I reverted the menu changes in ToolbarActionView and AppMenu.

Using a release chromium os build I see the timeout for:

ToolbarActionViewInteractiveUITest.DoubleClickToolbarActionToClose

Looking into it, the test is never returning from its own ExtensionTestMessageListener on line 305: 
https://cs.chromium.org/chromium/src/chrome/browser/ui/views/toolbar/toolbar_action_view_interactive_uitest.cc?rcl=0&l=305

So the popup is never launching on the click.

The logs also contain reports of extensions failing to load, not sure if it's a red herring:

4888:8132:0819/144304:FATAL:dispatcher.cc(1098)] Check failed: false. 
Backtrace:
	base::debug::StackTrace::StackTrace [0x00000001419C1391+33]
	logging::LogMessage::~LogMessage [0x000000014193966C+76]
	extensions::Dispatcher::OnLoaded [0x0000000140F3748B+351]
	IPC::MessageT<ExtensionMsg_Loaded_Meta,std::tuple<std::vector<ExtensionMsg_Loaded_Params,std::allocator<ExtensionMsg_Loaded_Params> > >,void>::Dispatch<extensions::Dispatcher,extensions::Dispatcher,void,void (__cdecl extensions::Dispatcher::*)(std::vector [0x0000000140F30FBE+206]
	extensions::Dispatcher::OnControlMessageReceived [0x0000000140F368E8+352]
	content::RenderThreadImpl::OnControlMessageReceived [0x0000000143FFEE95+77]
	content::ChildThreadImpl::OnMessageReceived [0x0000000142E960AB+171]
	IPC::ChannelProxy::Context::OnDispatchMessage [0x00000001420DDAF4+40]
	base::debug::TaskAnnotator::RunTask [0x00000001419D65C8+328]
	blink::scheduler::TaskQueueManager::ProcessTaskFromWorkQueue [0x0000000142FE87A9+829]
	blink::scheduler::TaskQueueManager::DoWork [0x0000000142FE7B30+548]
	base::internal::Invoker<base::internal::BindState<void (__cdecl blink::scheduler::TaskQueueManager::*)(base::TimeTicks,bool) __ptr64,base::WeakPtr<blink::scheduler::TaskQueueManager>,base::TimeTicks,bool>,void __cdecl(void)>::Run [0x00000001440BF6EA+70]
	base::debug::TaskAnnotator::RunTask [0x00000001419D65C8+328]
	base::MessageLoop::RunTask [0x0000000141935F8C+876]
	base::MessageLoop::DoWork [0x000000014193504B+411]
	base::MessagePumpDefault::Run [0x00000001419D772F+431]
	base::RunLoop::Run [0x00000001419384BE+46]
	content::RendererMain [0x0000000143FB6A38+676]
	content::RunNamedProcessTypeMain [0x0000000140F8C5F5+265]
	content::ContentMainRunnerImpl::Run [0x0000000140F8C492+358]
	content::ContentMain [0x0000000140F8B4D4+48]
	content::LaunchTests [0x00000001413A594C+660]
	main [0x000000013FA90106+82]
	__scrt_common_main_seh [0x000000014436C8B0+292] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:255)
	BaseThreadInitThunk [0x0000000076BF5A4D+13]
	RtlUserThreadStart [0x0000000076D2B831+33]

@38 That would definitely be a problem (at least with some of the extension ones).  How did you get that to manifest?

It showed up in the logs of a trybot when I was testing a fix for different flakes:

Bot:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/265271

Log:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/265271/steps/interactive_ui_tests%20%28with%20patch%29%20on%20Windows-7-SP1/logs/stdio

Detected 10 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.CaptureLostDuringDrag/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUwsSBUZsYWtlIkhUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuQ2FwdHVyZUxvc3REdXJpbmdEcmFnLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 12 new flakes for test/step "MouseLeaveTest.ContextMenu". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJQsSBUZsYWtlIhpNb3VzZUxlYXZlVGVzdC5Db250ZXh0TWVudQw. This message was posted automatically by the chromium-try-flakes app.

Detected 3 new flakes for test/step "AutofillInteractiveTest.AutofillSelectViaTab". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNwsSBUZsYWtlIixBdXRvZmlsbEludGVyYWN0aXZlVGVzdC5BdXRvZmlsbFNlbGVjdFZpYVRhYgw. This message was posted automatically by the chromium-try-flakes app.

Detected 25 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteBeforeStartedDragging/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWQsSBUZsYWtlIk5UYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlQmVmb3JlU3RhcnRlZERyYWdnaW5nLzAM. This message was posted automatically by the chromium-try-flakes app.

I took a quick look at the test on linux and linux chromium os builds.

The following tests do not use menus:
AutofillInteractiveTest.AutofillSelectViaTab
BrowserActionInteractiveTest.BrowserClickClosesPopup1
BrowserActionInteractiveTest.BrowserClickClosesPopup2
FindInPageTest.CrashEscHandlers
LocationIconViewTest.HideOnSecondClick
TabDragging/DetachToBrowserTabDragControllerTest.DeleteBeforeStartedDragging/0
TabDragging/DetachToBrowserTabDragControllerTest.DragInSameWindow
TabDragging/DetachToBrowserTabDragControllerTest.CaptureLostDuringDrag/0
TabDragging/DetachToBrowserTabDragControllerTest.DeleteTabsWhileDetached
TabDragCaptureLostTest.ReleaseCaptureOnDrag
ToolbarActionViewInteractiveUITest.DoubleClickToolbarActionToClose

The following use the old style of menus:
PageActionImageViewInteractiveUITest.TestBasicPageActionFunctions
MouseLeaveTest.ContextMenu

I cannot properly reproduce ToolbarViewInteractiveUITest.TestAppMenuOpensOnDrag on my machine. This test should be using the asynchronous menus, however the failure in #1 is a timeout like all of the others. I would expect the statements checking the menu status to fail if impacted by the menu change.

Detected 20 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteTabWhileDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyVAsSBUZsYWtlIklUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlVGFiV2hpbGVEZXRhY2hlZC8wDA. This message was posted automatically by the chromium-try-flakes app.

 Issue 640318  has been merged into this issue.

Detected 11 new flakes for test/step "AutofillInteractiveTest.AutofillSelectViaTab". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNwsSBUZsYWtlIixBdXRvZmlsbEludGVyYWN0aXZlVGVzdC5BdXRvZmlsbFNlbGVjdFZpYVRhYgw. This message was posted automatically by the chromium-try-flakes app.

Detected 19 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindowFromMaximizedWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGV0YWNoVG9Pd25XaW5kb3dGcm9tTWF4aW1pemVkV2luZG93LzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 20 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DragAllToSeparateWindowAndCancel/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyXgsSBUZsYWtlIlNUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRHJhZ0FsbFRvU2VwYXJhdGVXaW5kb3dBbmRDYW5jZWwvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 21 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteTabWhileDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyVAsSBUZsYWtlIklUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlVGFiV2hpbGVEZXRhY2hlZC8wDA. This message was posted automatically by the chromium-try-flakes app.

This is super preventing me from landing patches with CQ and has been going on for a week.

Assigned => Random cc'd person, please have a look or assign to someone else.

Completely agree this needs to be fixed. Shouldn't the sheriffs be investigating this rather than picking random people that happened to be cc'd on the bug though?

 Issue 640729  has been merged into this issue.

Issue 641009 has been merged into this issue.

This win_rel_ng builder has been quite sad http://shortn/_wuSiSXGMK7: ~50% success lately

mmoss: you're trooper today, and chance you could get a screenshot of one of the win_rel_ng bots when the test is hung?

Detected 34 new flakes for test/step "MouseLeaveTest.ContextMenu". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJQsSBUZsYWtlIhpNb3VzZUxlYXZlVGVzdC5Db250ZXh0TWVudQw. This message was posted automatically by the chromium-try-flakes app.

Detected 34 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.CaptureLostDuringDrag/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUwsSBUZsYWtlIkhUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuQ2FwdHVyZUxvc3REdXJpbmdEcmFnLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 34 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteBeforeStartedDragging/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWQsSBUZsYWtlIk5UYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlQmVmb3JlU3RhcnRlZERyYWdnaW5nLzAM. This message was posted automatically by the chromium-try-flakes app.

I took a quick look; but I don't think I'll actually be able to help. Brain dump so far debugging the tests:

I get hangs running "interactive_ui_tests.exe --gtest_filter=FindInPageTest.CrashEscHandlers" locally.
That test notes: // Flaky because the test server fails to start? See:  http://crbug.com/96594 .
But it actually hangs somewhere while sending the escape key press event:
https://cs.chromium.org/chromium/src/chrome/browser/ui/views/find_bar_views_interactive_uitest.cc?rcl=0&l=122
  // This used to crash until bug 1303709 was fixed.
  ASSERT_TRUE(ui_test_utils::SendKeyPressSync(
      browser(), ui::VKEY_ESCAPE, false, false, false, false));
Thankfully, that's not actually a crbug from the future, but the probably unrelated very old internal bug http://b/1303709

I'm debugging through SendKeyPressToWindowSync and it's hanging on runner->Run():
https://cs.chromium.org/chromium/src/chrome/test/base/interactive_test_utils.cc?rcl=0&l=71
The most recent/suspicious change I see to the underlying files there is https://codereview.chromium.org/2049493004
But reverting that doesn't fix my local hangs... At this point I'm a little stuck.

==========================================================================
On the other side, looking for some recent culprit just from the bot logs:

I can't wring any useful info from the flakiness dashboard: (not sure why it doesn't have data for these tests...)
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=interactive_ui_tests&builder=chromium.win%3AWin7%20Tests%20(1)

So I spent some time looking at these bots' logs, in particular "Win7 Tests (1)":
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29?numbuilds=200
https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20%281%29?numbuilds=200
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%28dbg%29%281%29?numbuilds=200

I went back far (through #56320) on "Win7 Tests (1)" and found some earlier ExtensionDialogUiTest.TabFocusLoop failures:
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56347
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56349
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56352

But the real fun starts on build #56357 with many failing tests:
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56357

I looked backwards from the changes in that build 'git log b099e13e9cdd6d0380a4e915bc97f78436b73bbe':
But reverting the most suspicious CL I saw (https://codereview.chromium.org/2256983003) doesn't fix my local hangs.

CC'ing oshima for ui/base/test/ui_controls_internal_win.cc's InputDispatcher.

I found that manually pressing [Esc] while the test is waiting/hung actually causes the test to pass, so i wonder if the InputDispatcher is actually posting/observing the event correctly, or if it's a timing issue between dispatching the event and installing the hook or something similar.

Doing some more local debugging... It looks like we definitely call SetWindowsHookEx before calling SendInput, but for whatever reason, the KeyHook function isn't called until I manually press a key. Reverting https://codereview.chromium.org/2267123002 doesn't help.

I wonder if something about the test's use of GetCurrentThreadId() for SetWindowsHookEx limits the scope of the events monitored; from MSDN:

SetWindowsHookEx function
Installs an application-defined hook procedure into a hook chain. You would install a hook procedure to monitor the system for certain types of events. These events are associated either with a specific thread or with all threads in the same desktop as the calling thread.

Also, I wonder if the way we are calling GetMessage or PeekMessage has changed; also from MSDN:

KeyboardProc callback function
An application-defined or library-defined callback function used with the SetWindowsHookEx function. The system calls this function whenever an application calls the GetMessage or PeekMessage function and there is a keyboard message (WM_KEYUP or WM_KEYDOWN) to be processed.

AFAICT https://codereview.chromium.org/2250133004 (landed @ https://crrev.com/f15d4cc339737df653274e39726ae0d14d409b18
Cr-Commit-Position: refs/heads/master@{#412925} ) that I can find with a try run that has test failures like we are seeing. So, the failure is either because of that patch, or slightly before it. Unfortunately try runs are expiring in this range very soon, so we're going to lose data unless this is resoled very shortly.

FWIW I can't repro this locally with either release or debug builds. I'm on an 840 though, I think msw has an older machine. I tried with and without classic too.

Mike tried reverting https://codereview.chromium.org/2250133004 and it didn't help.

I've noticed that the translate bubble is showing a TON on canary for me. I wonder if that is some how related to this. Say for example we trigger showing the translate bubble in some of these tests for whatever reason: 641092

Since I've been getting reliable hangs locally, but haven't a clue as to the cause, I started a manual bisect.
The test passes on 246cf98c2d65b307d0219893f80b0f47e3146d30 / #412881 / Thu Aug 18 10:42:06 2016 -0700
I'll try to update again tonight (builds take an hour or so); suggestions for hashes/positions welcome.
When I next hit a failing revision, I'll try no-oping the translate bubble to see if that's related.

*Looks suspiciously at https://codereview.chromium.org/2259983002

For the error message "Cannot create windows on non-UI thread!" from singleton_hwnd.cc. I got the following stack track for triggering this:
        gfx::SingletonHwnd::SingletonHwnd [0x000007FEE8CF7115+213] (d:\src\chromium\src\ui\gfx\win\singleton_hwnd.cc:36)
        base::Singleton<gfx::SingletonHwnd,base::DefaultSingletonTraits<gfx::SingletonHwnd>,gfx::SingletonHwnd>::get [0x000007FEE8CF7667+59] (d:\src\chromium\sr
c\base\memory\singleton.h:250)
        gfx::SingletonHwndObserver::SingletonHwndObserver [0x000007FEE8CF7749+25] (d:\src\chromium\src\ui\gfx\win\singleton_hwnd_observer.cc:14)
        gfx::SysColorChangeObserver::SysColorChangeObserver [0x000007FEE8CEEB25+129] (d:\src\chromium\src\ui\gfx\sys_color_change_listener.cc:76)
        base::Singleton<gfx::SysColorChangeObserver,base::DefaultSingletonTraits<gfx::SysColorChangeObserver>,gfx::SysColorChangeObserver>::get [0x000007FEE8CEE
E9F+59] (d:\src\chromium\src\base\memory\singleton.h:250)
        gfx::ScopedSysColorChangeListener::ScopedSysColorChangeListener [0x000007FEE8CEEA7D+17] (d:\src\chromium\src\ui\gfx\sys_color_change_listener.cc:104)
        ui::NativeThemeWin::NativeThemeWin [0x000007FEF54B5710+132] (d:\src\chromium\src\ui\native_theme\native_theme_win.cc:288)
        ui::NativeThemeWin::instance [0x000007FEF54B9B0C+48] (d:\src\chromium\src\ui\native_theme\native_theme_win.cc:191)
        ui::NativeThemeAuraWin::GetPartSize [0x000007FEF54BA646+38] (d:\src\chromium\src\ui\native_theme\native_theme_aurawin.cc:78)
        content::WebThemeEngineImpl::getSize [0x000007FEE59D678B+91] (d:\src\chromium\src\content\child\webthemeengine_impl_default.cc:170)
        blink::ScrollbarThemeAura::scrollbarThickness [0x000007FEE49EE572+54] (d:\src\chromium\src\third_party\webkit\source\platform\scroll\scrollbarthemeaura.
cpp:149)
        blink::Scrollbar::Scrollbar [0x000007FEE49EB08F+271] (d:\src\chromium\src\third_party\webkit\source\platform\scroll\scrollbar.cpp:69)
        blink::Scrollbar::create [0x000007FEE492A781+77] (d:\src\chromium\src\third_party\webkit\source\platform\scroll\scrollbar.h:53)
        blink::FrameView::createScrollbar [0x000007FEE3ED46FD+121] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:523)
        blink::FrameView::setHasVerticalScrollbar [0x000007FEE3EDB063+59] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:3317)
        blink::FrameView::adjustScrollbarExistence [0x000007FEE3ED2B14+144] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:3634)
        blink::FrameView::updateScrollbars [0x000007FEE3EDD71A+234] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:3696)
        blink::FrameView::setScrollbarModes [0x000007FEE3EDB794+204] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:3370)
        blink::FrameView::layout [0x000007FEE3ED6ECD+1173] (d:\src\chromium\src\third_party\webkit\source\core\frame\frameview.cpp:1050)
        blink::Document::implicitClose [0x000007FEE3BA6E90+604] (d:\src\chromium\src\third_party\webkit\source\core\dom\document.cpp:2671)
        blink::FrameLoader::checkCompleted [0x000007FEE3F6FB04+208] (d:\src\chromium\src\third_party\webkit\source\core\loader\frameloader.cpp:639)
        blink::FrameLoader::finishedParsing [0x000007FEE3F70D39+173] (d:\src\chromium\src\third_party\webkit\source\core\loader\frameloader.cpp:557)
        blink::Document::finishedParsing [0x000007FEE3BA58B7+251] (d:\src\chromium\src\third_party\webkit\source\core\dom\document.cpp:4836)
        blink::HTMLTreeBuilder::finished [0x000007FEE3D03DF4+20] (d:\src\chromium\src\third_party\webkit\source\core\html\parser\htmltreebuilder.cpp:2804)
        blink::HTMLDocumentParser::end [0x000007FEE3CE984F+35] (d:\src\chromium\src\third_party\webkit\source\core\html\parser\htmldocumentparser.cpp:871)
        blink::HTMLDocumentParser::attemptToRunDeferredScriptsAndEnd [0x000007FEE3CE8C22+38] (d:\src\chromium\src\third_party\webkit\source\core\html\parser\htm
ldocumentparser.cpp:883)
        blink::HTMLDocumentParser::prepareToStopParsing [0x000007FEE3CEAD21+81] (d:\src\chromium\src\third_party\webkit\source\core\html\parser\htmldocumentpars
er.cpp:234)
        blink::HTMLDocumentParser::finish [0x000007FEE3CEA147+419] (d:\src\chromium\src\third_party\webkit\source\core\html\parser\htmldocumentparser.cpp:946)
        blink::DocumentWriter::end [0x000007FEE3F6AFED+117] (d:\src\chromium\src\third_party\webkit\source\core\loader\documentwriter.cpp:109)
        blink::DocumentLoader::finishedLoading [0x000007FEE3F66302+150] (d:\src\chromium\src\third_party\webkit\source\core\loader\documentloader.cpp:317)
        blink::DocumentLoader::maybeLoadEmpty [0x000007FEE3F66697+395] (d:\src\chromium\src\third_party\webkit\source\core\loader\documentloader.cpp:644)
        blink::DocumentLoader::startLoadingMainResource [0x000007FEE3F67318+68] (d:\src\chromium\src\third_party\webkit\source\core\loader\documentloader.cpp:65
9)
        blink::FrameLoader::startLoad [0x000007FEE3F739AC+868] (d:\src\chromium\src\third_party\webkit\source\core\loader\frameloader.cpp:1461)
        blink::FrameLoader::load [0x000007FEE3F71273+763] (d:\src\chromium\src\third_party\webkit\source\core\loader\frameloader.cpp:1016)
        blink::WebLocalFrameImpl::load [0x000007FEE47CC8D2+198] (d:\src\chromium\src\third_party\webkit\source\web\weblocalframeimpl.cpp:1843)
        content::RenderFrameImpl::NavigateInternal [0x000007FEE619FA1E+2566] (d:\src\chromium\src\content\renderer\render_frame_impl.cc:5643)
        content::RenderFrameImpl::OnNavigate [0x000007FEE61A370C+292] (d:\src\chromium\src\content\renderer\render_frame_impl.cc:1588)
        IPC::MessageT<FrameMsg_Navigate_Meta,std::tuple<content::CommonNavigationParams,content::StartNavigationParams,content::RequestNavigationParams>,void>::
Dispatch<content::RenderFrameImpl,content::RenderFrameImpl,void,void (__cdecl content::RenderFrameImpl: [0x000007FEE6195CD4+260] (d:\src\chromium\src\ipc\ipc_me
ssage_templates.h:121)
        content::RenderFrameImpl::OnMessageReceived [0x000007FEE61A1949+617] (d:\src\chromium\src\content\renderer\render_frame_impl.cc:1460)
        IPC::MessageRouter::RouteMessage [0x000007FEF426027C+80] (d:\src\chromium\src\ipc\message_router.cc:52)
        content::ChildThreadImpl::OnMessageReceived [0x000007FEE5988230+180] (d:\src\chromium\src\content\child\child_thread_impl.cc:767)
        IPC::ChannelProxy::Context::OnDispatchMessage [0x000007FEF424B4DC+40] (d:\src\chromium\src\ipc\ipc_channel_proxy.cc:315)
        base::debug::TaskAnnotator::RunTask [0x000007FEF5625368+328] (d:\src\chromium\src\base\debug\task_annotator.cc:56)
        blink::scheduler::TaskQueueManager::ProcessTaskFromWorkQueue [0x000007FEE49D572D+741] (d:\src\chromium\src\third_party\webkit\source\platform\scheduler\
base\task_queue_manager.cc:319)
        blink::scheduler::TaskQueueManager::DoWork [0x000007FEE49D4D9A+498] (d:\src\chromium\src\third_party\webkit\source\platform\scheduler\base\task_queue_ma
nager.cc:218)
        base::internal::InvokeHelper<1,void>::MakeItSo<void (__cdecl blink::scheduler::TaskQueueManager::*const & __ptr64)(base::TimeTicks,bool) __ptr64,base::W
eakPtr<blink::scheduler::TaskQueueManager> const & __ptr64,base::TimeTicks const & __ptr64,bool const & [0x000007FEE49D38ED+101] (d:\src\chromium\src\base\bind_
internal.h:306)
        base::debug::TaskAnnotator::RunTask [0x000007FEF5625368+328] (d:\src\chromium\src\base\debug\task_annotator.cc:56)
        base::MessageLoop::RunTask [0x000007FEF5649F69+889] (d:\src\chromium\src\base\message_loop\message_loop.cc:489)
        base::MessageLoop::DoWork [0x000007FEF564944B+411] (d:\src\chromium\src\base\message_loop\message_loop.cc:621)
        base::MessagePumpDefault::Run [0x000007FEF564B086+326] (d:\src\chromium\src\base\message_loop\message_pump_default.cc:35)
        base::RunLoop::Run [0x000007FEF56719EE+46] (d:\src\chromium\src\base\run_loop.cc:36)
        content::RendererMain [0x000007FEE61DAD28+704] (d:\src\chromium\src\content\renderer\renderer_main.cc:198)
        content::RunNamedProcessTypeMain [0x000007FEE6297582+178] (d:\src\chromium\src\content\app\content_main_runner.cc:435)
        content::ContentMainRunnerImpl::Run [0x000007FEE6297476+206] (d:\src\chromium\src\content\app\content_main_runner.cc:786)
        content::ContentMain [0x000007FEE6296810+48] (d:\src\chromium\src\content\app\content_main.cc:20)
        content::LaunchTests [0x000000014002CE6F+599] (d:\src\chromium\src\content\public\test\test_launcher.cc:523)
        main [0x000000013F98970A+82] (d:\src\chromium\src\chrome\test\base\interactive_ui_tests_main.cc:88)
        __scrt_common_main_seh [0x0000000140D1BDB8+292] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:255)
        BaseThreadInitThunk [0x00000000775559BD+13]
        RtlUserThreadStart [0x000000007778A2E1+33]

Not sure this is related.

It's not, that's a pretty common non-fatal error message.

My local build passes at 62440fd6ecb0bea661f072d68ee25fc047e8bd72 / #412928 / Thu Aug 18 13:58:54 2016 -0700
Next position between that and #412976 ( from https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56357 ) is #412952.

That trace from the renderer isn't fatal and I don't believe would impact these tests.

 Issue 641224  has been merged into this issue.

 Issue 641110  has been merged into this issue.

 Issue 641109  has been merged into this issue.

 Issue 641108  has been merged into this issue.

In response to #62, I was able to see a point where the first test in this report became more flaky here:
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#testType=interactive_ui_tests&showAllRuns=true&tests=BrowserActionInteractiveTest.BrowserClickClosesPopup%0A

I had to select "use all recorded runs" and scroll right until the teal color went away, roughly around http://test-results.appspot.com/revision_range?start=412966&end=412976

If I missed some context in the earlier comments please ignore me.

Well, I tried r412952 (pass) -> r412968 (pass) -> r412976 (pass, but that's odd [1]) -> r414288 (pass, ugh! [2])
[1] r412976 failed for: https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29/builds/56357 
[2] r414288 failed for me locally yesterday (debugged above), now it's passing over remote desktop...
So, I can't even repro anymore, and the problems I saw earlier might not be related... Back to square one :(

Thanks estaab, here are the three win bots on the flakiness dash with 'use all recorded runs':
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&testType=interactive_ui_tests&builder=chromium.win%3AWin%207%20Tests%20x64%20(1)
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&testType=interactive_ui_tests&builder=chromium.win%3AWin7%20Tests%20(1)
http://test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&testType=interactive_ui_tests&builder=chromium.win%3AWin7%20Tests%20(dbg)(1)

Here's the earliest build where I see each bot flaking a fair number of tests in one build:
r412998 https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20(1)/builds/16096
r412976 https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20(1)/builds/56357
r412997 https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20(dbg)(1)/builds/51851
*r412917 https://build.chromium.org/p/chromium.win/builders/Win%207%20Tests%20x64%20(1)/builds/16090 (not sure if this fits the pattern, it's earlier...)

It does seem like we should be looking backwards from r412976 (or r412917) no more than ~50-100 cls... roughly:
https://chromium.googlesource.com/chromium/src/+log/04831a189c07ec61a8b07bcce3d6efed4d3b9c74..b099e13e9cdd6d0380a4e915bc97f78436b73bbe?pretty=fuller

======================================================================================

On the flip side, we could land some print debugging code, like what I was using yesterday, to see where the test hangs.
I attached the two patches I used yesterday; I can land something similar, otherwise I'm stuck...
Assigning/cc'ing unlucky sheriffs, adding some labels...

Deleted: 639350_debug_0.patch 4.7 KB

639350_debug_0.patch 4.7 KB Download

Deleted: 639350_debug_1.patch 4.6 KB

639350_debug_1.patch 4.6 KB Download

Mike, https://codereview.chromium.org/2259983002 landed after https://codereview.chromium.org/2250133004, which also shows the flake on its try runs. I think the bug was introduced before 412925. You said you went to 412881 and that passed. So, it looks like the bad patch is between 412881 and 412925. http://test-results.appspot.com/revision_range?start=412881&end=412925

I support adding debugging code if it makes it easier to identify what's going on.

Yeah, the earlier try run that failed on https://codereview.chromium.org/2250133004 got revision 412893:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264190
So the suspect CL is earlier than r412893. We could look for earlier interactive_ui_tests failures on that trybot.

Scott, 412881 passing for me locally yesterday isn't necessarily indicative of anything, ToT passes for me now :(
I'll put up a debugging CL...

I manually looked back over 190 builds on the win_chromium_x64_rel_ng trybot (back to build 264000), here are some earlier failures I see:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264117 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264122 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264125 <- Many failures; more like what we see now...
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264132 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264138 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264152 <- Many failures; more like what we see now...
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264161 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264162 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264166 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264169 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264171 <- Many failures; more like what we see now...
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264175 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264177 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264178 <- ExtensionDialogUiTest.TabFocusLoop
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/264186 <- ExtensionDialogUiTest.TabFocusLoop

ExtensionDialogUiTest.TabFocusLoop seems like a separate issue and it's already disabled on Win ( see  Issue 583124  and  Issue 639072  ).

It's likely the flakes started recently before r412858 (ToT revision for win_chromium_x64_rel_ng build #264125); logs for r412800-r412858:
https://chromium.googlesource.com/chromium/src/+log/e4a15c1756fcd8b425696a7b30ceda35067796bc..7589625e80b551184e36527bf8979e5427ec9118/?pretty=fuller

WIP debug logging CL: https://codereview.chromium.org/2286853002/

Detected 35 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.CancelOnNewTabWhenDragging/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWAsSBUZsYWtlIk1UYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuQ2FuY2VsT25OZXdUYWJXaGVuRHJhZ2dpbmcvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 35 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DragAllToSeparateWindowAndCancel/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyXgsSBUZsYWtlIlNUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRHJhZ0FsbFRvU2VwYXJhdGVXaW5kb3dBbmRDYW5jZWwvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 11 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindowFromMaximizedWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGV0YWNoVG9Pd25XaW5kb3dGcm9tTWF4aW1pemVkV2luZG93LzAM. This message was posted automatically by the chromium-try-flakes app.

Scott and I have been sending tryjobs with some extra debug logging.
It looks like FindInPageTest.CrashEscHandlers is hanging when clicking the location bar:
https://codereview.chromium.org/2286853002/diff/40001/chrome/browser/ui/views/find_bar_views_interactive_uitest.cc#newcode127
  // Click on the location bar so that Find box loses focus.
  ASSERT_NO_FATAL_FAILURE(ui_test_utils::ClickOnView(browser(), VIEW_ID_OMNIBOX));

All the tests listed in the original comment call similar functions:
BrowserActionInteractiveTest.BrowserClickClosesPopup1 - ClickOnView
FindInPageTest.CrashEscHandlers - ClickOnView
PageActionImageViewInteractiveUITest.TestBasicPageActionFunctions - SendMouseMoveNotifyWhenDone/SendMouseEventsNotifyWhenDone
ToolbarViewInteractiveUITest.TestAppMenuOpensOnDrag - DoDragAndDrop
BrowserActionInteractiveTest.BrowserClickClosesPopup2 - ClickOnView
TabDragCaptureLostTest.ReleaseCaptureOnDrag - SendMouseMoveSync/SendMouseEventsSync
ToolbarActionViewInteractiveUITest.DoubleClickToolbarActionToClose - SendMouseMoveSync/SendMouseEventsSync
TabDragging/DetachToBrowserTabDragControllerTest.DragInSameWindow - SendMouseMoveSync/SendMouseEventsSync
TabDragging/DetachToBrowserTabDragControllerTest.DeleteTabsWhileDetached - SendMouseMoveSync/SendMouseEventsSync

AFAICT the mouse hook never see the event. It may be something else is on screen at that position. I'm adding more logging to see if that is the case.

Looks like we are getting an explorer window on top of the chrome window, so that when we generate the mouse click it goes there and the test hangs. No idea who is triggering bringing up the explorer window...

Detected 40 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteSourceDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUgsSBUZsYWtlIkdUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlU291cmNlRGV0YWNoZWQvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "AutofillInteractiveTest.AutofillViaDownArrow". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNwsSBUZsYWtlIixBdXRvZmlsbEludGVyYWN0aXZlVGVzdC5BdXRvZmlsbFZpYURvd25BcnJvdww. This message was posted automatically by the chromium-try-flakes app.

Detected 5 new flakes for test/step "AutofillInteractiveTest.AutofillSelectViaTab". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNwsSBUZsYWtlIixBdXRvZmlsbEludGVyYWN0aXZlVGVzdC5BdXRvZmlsbFNlbGVjdFZpYVRhYgw. This message was posted automatically by the chromium-try-flakes app.

Detected 5 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindowFromMaximizedWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyYgsSBUZsYWtlIldUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGV0YWNoVG9Pd25XaW5kb3dGcm9tTWF4aW1pemVkV2luZG93LzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 5 new flakes for test/step "PageActionImageViewInteractiveUITest.TestBasicPageActionFunctions". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyTAsSBUZsYWtlIkFQYWdlQWN0aW9uSW1hZ2VWaWV3SW50ZXJhY3RpdmVVSVRlc3QuVGVzdEJhc2ljUGFnZUFjdGlvbkZ1bmN0aW9ucww. This message was posted automatically by the chromium-try-flakes app.

I think I have a patch that fixs this.

Detected 3 new flakes for test/step "MouseLeaveTest.ContextMenu". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJQsSBUZsYWtlIhpNb3VzZUxlYXZlVGVzdC5Db250ZXh0TWVudQw. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.CaptureLostDuringDrag/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUwsSBUZsYWtlIkhUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuQ2FwdHVyZUxvc3REdXJpbmdEcmFnLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteBeforeStartedDragging/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWQsSBUZsYWtlIk5UYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlQmVmb3JlU3RhcnRlZERyYWdnaW5nLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteSourceDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUgsSBUZsYWtlIkdUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlU291cmNlRGV0YWNoZWQvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 4 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteSourceDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUgsSBUZsYWtlIkdUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlU291cmNlRGV0YWNoZWQvMAw. This message was posted automatically by the chromium-try-flakes app.

 Issue 641796  has been merged into this issue.

 Issue 641795  has been merged into this issue.

 Issue 641720  has been merged into this issue.

 Issue 641505  has been merged into this issue.

 Issue 641504  has been merged into this issue.

 Issue 641503  has been merged into this issue.

 Issue 641704  has been merged into this issue.

 Issue 641705  has been merged into this issue.

 Issue 641502  has been merged into this issue.

Detected 3 new flakes for test/step "MouseLeaveTest.ContextMenu". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyJQsSBUZsYWtlIhpNb3VzZUxlYXZlVGVzdC5Db250ZXh0TWVudQw. This message was posted automatically by the chromium-try-flakes app.

 Issue 639010  has been merged into this issue.

 Issue 638692  has been merged into this issue.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9a221a492c097da82c4f14019dad5dba14d9ac23

commit 9a221a492c097da82c4f14019dad5dba14d9ac23
Author: sky <sky@chromium.org>
Date: Mon Aug 29 16:49:34 2016

Ensures the first browser in each interactive_ui_test is in the foreground

It's entirely possible for an interactive_ui_test to do something that
triggers the suite to go to the background (such as clicking on a
desktop). By ensuring each browser is in the front at the start of the
test we avoid typical flake.

I should be able to remove some calls to BringBrowserWindowToFront
that are sprinkled throughout the code after this, but I'll do that
separately.

BUG= 639350 
TEST=test only fix

Review-Url: https://codereview.chromium.org/2285023003
Cr-Commit-Position: refs/heads/master@{#415018}

[modify] https://crrev.com/9a221a492c097da82c4f14019dad5dba14d9ac23/chrome/test/base/in_process_browser_test.cc
[modify] https://crrev.com/9a221a492c097da82c4f14019dad5dba14d9ac23/chrome/test/base/in_process_browser_test.h
[modify] https://crrev.com/9a221a492c097da82c4f14019dad5dba14d9ac23/chrome/test/base/interactive_ui_tests_main.cc

I'm hoping the patch I just landed fixes this. Will leave the bug open and see how the flake goes today.

Thanks for fixing this

Detected 5 new flakes for test/step "AutofillInteractiveTest.AutofillSelectViaTab". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyNwsSBUZsYWtlIixBdXRvZmlsbEludGVyYWN0aXZlVGVzdC5BdXRvZmlsbFNlbGVjdFZpYVRhYgw. This message was posted automatically by the chromium-try-flakes app.

Detected 12 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DragSingleTabToSeparateWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyWwsSBUZsYWtlIlBUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRHJhZ1NpbmdsZVRhYlRvU2VwYXJhdGVXaW5kb3cvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 12 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DragToSeparateWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUgsSBUZsYWtlIkdUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRHJhZ1RvU2VwYXJhdGVXaW5kb3cvMAw. This message was posted automatically by the chromium-try-flakes app.

Detected 12 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.PressEscapeWhileDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyVgsSBUZsYWtlIktUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuUHJlc3NFc2NhcGVXaGlsZURldGFjaGVkLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 13 new flakes for test/step "LocationIconViewTest.HideOnSecondClick". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMQsSBUZsYWtlIiZMb2NhdGlvbkljb25WaWV3VGVzdC5IaWRlT25TZWNvbmRDbGljaww. This message was posted automatically by the chromium-try-flakes app.

Failures referenced in comments 116-120 are before my patch landed.

Detected 6 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DetachToOwnWindow/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyTwsSBUZsYWtlIkRUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGV0YWNoVG9Pd25XaW5kb3cvMAw. This message was posted automatically by the chromium-try-flakes app.

 Issue 642123  has been merged into this issue.

Same for comment 122.

#56711 - #56719 are all green after the fix landed in #56711. Things are looking good given that the failures were much more frequent before the fix:
https://build.chromium.org/p/chromium.win/builders/Win7%20Tests%20%281%29?numbuilds=100

Detected 11 new flakes for test/step "FindInPageTest.CrashEscHandlers". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyKgsSBUZsYWtlIh9GaW5kSW5QYWdlVGVzdC5DcmFzaEVzY0hhbmRsZXJzDA. This message was posted automatically by the chromium-try-flakes app.

Same for comment 126.

Detected 12 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.CaptureLostDuringDrag/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyUwsSBUZsYWtlIkhUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuQ2FwdHVyZUxvc3REdXJpbmdEcmFnLzAM. This message was posted automatically by the chromium-try-flakes app.

Detected 8 new flakes for test/step "TabDragging/DetachToBrowserTabDragControllerTest.DeleteTabWhileDetached/0". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyVAsSBUZsYWtlIklUYWJEcmFnZ2luZy9EZXRhY2hUb0Jyb3dzZXJUYWJEcmFnQ29udHJvbGxlclRlc3QuRGVsZXRlVGFiV2hpbGVEZXRhY2hlZC8wDA. This message was posted automatically by the chromium-try-flakes app.

Sure does take a while for the flake detector to catch up. Same comment for 129 and 130.

AFAICT since https://codereview.chromium.org/2285023003 landed yesterday there hasn't been any more flake. Marking fixed.