tapted@, since you're the author of the test, could you take look?
Since the test is introduced on Jun 2015[1], you may not be related. Please help to route appropriate owner.

[1] https://codereview.chromium.org/1186803003 Mac: Give packaged app windows their own window cycle list.

tapted@, since you're the author of the test, could you take look?

Also failed on Mac10.10 Test bot
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac10.10%20Tests/34392

[ RUN      ] AppShimMenuControllerUITest.WindowCycling
[4095:5127:0821/230718.604574:WARNING:notification_platform_bridge_mac.mm(510)] AlertNotificationService: XPC connection invalidated.
[23:07:18.655] vtDecompressionDuctCreate signalled err=-8973 (err) (Could not select and open decoder instance) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTDecompressionSession.c line 1181
<<<< VTVideoEncoderSelection >>>> VTSelectAndCreateVideoEncoderInstanceInternal: no video encoder found for 'avc1'
[23:07:18.741] VTSelectAndCreateVideoEncoderInstanceInternal signalled err=-12908 (err) (Video encoder not available) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTVideoEncoderSelection.c line 1245
[23:07:18.742] VTCompressionSessionCreate signalled err=-12908 (err) (Could not select and open encoder instance) at /SourceCache/CoreMedia_frameworks/CoreMedia-1562.240/Sources/VideoToolbox/VTCompressionSession.c line 946
[4095:5891:0821/230719.479739:ERROR:mach_port_broker.mm(175)] Unknown process 4105 is sending Mach IPC messages!
[4095:779:0821/230719.861525:FATAL:web_contents_observer_sanity_checker.cc(317)] Check failed: is_loading_.
0   interactive_ui_tests                0x0000000104efacbc base::debug::StackTrace::StackTrace(unsigned long) + 28
1   interactive_ui_tests                0x0000000104e55eb1 logging::LogMessage::~LogMessage() + 225
2   interactive_ui_tests                0x00000001048fad54 content::WebContentsObserverSanityChecker::DidStopLoading() + 68
3   interactive_ui_tests                0x00000001033d71c1 content::WebContentsImpl::LoadingStateChanged(bool, bool, content::LoadNotificationDetails*) + 1057
4   interactive_ui_tests                0x00000001033deb71 content::WebContentsImpl::DidStopLoading() + 161
5   interactive_ui_tests                0x00000001030e00e7 content::FrameTreeNode::~FrameTreeNode() + 1543
6   interactive_ui_tests                0x00000001030e07dd content::FrameTreeNode::ResetForNewProcess() + 109
7   interactive_ui_tests                0x000000010310d221 content::RenderFrameHostImpl::OnRenderProcessGone(int, int) + 81
8   interactive_ui_tests                0x000000010310d0f4 bool IPC::MessageT<FrameHostMsg_RenderProcessGone_Meta, std::__1::tuple<int, int>, void>::Dispatch<content::RenderFrameHostImpl, content::RenderFrameHostImpl, void, void (content::RenderFrameHostImpl::*)(int, int)>(IPC::Message const*, content::RenderFrameHostImpl*, content::RenderFrameHostImpl*, void*, void (content::RenderFrameHostImpl::*)(int, int)) + 116
9   interactive_ui_tests                0x000000010310c16d content::RenderFrameHostImpl::OnMessageReceived(IPC::Message const&) + 77
10  interactive_ui_tests                0x00000001032aef00 content::RenderProcessHostImpl::ProcessDied(bool, content::ChildProcessTerminationInfo*) + 1024
11  interactive_ui_tests                0x00000001032aea50 content::RenderProcessHostImpl::FastShutdownIfPossible(unsigned long, bool) + 288
12  interactive_ui_tests                0x000000010703f432 TabStripModel::CloseWebContentses(base::span<content::WebContents* const, 18446744073709551615ul>, unsigned int) + 354
13  interactive_ui_tests                0x000000010703c62f TabStripModel::InternalCloseTabs(base::span<content::WebContents* const, 18446744073709551615ul>, unsigned int) + 479
14  interactive_ui_tests                0x000000010703c41f TabStripModel::CloseAllTabs() + 255
15  interactive_ui_tests                0x000000010729b21c BrowserView::CanClose() + 124
16  interactive_ui_tests                0x0000000104e10d22 views::Widget::Close() + 50
17  interactive_ui_tests                0x00000001021a3086 virtual thunk to (anonymous namespace)::AppShimMenuControllerUITest::TearDownOnMainThread() + 70
18  interactive_ui_tests                0x00000001048ec254 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() + 500
19  interactive_ui_tests                0x000000010437f227 ChromeBrowserMainParts::PreMainMessageLoopRunImpl() + 4375
20  interactive_ui_tests                0x000000010437e00e ChromeBrowserMainParts::PreMainMessageLoopRun() + 62
21  interactive_ui_tests                0x0000000102fa0493 content::BrowserMainLoop::PreMainMessageLoopRun() + 67
22  interactive_ui_tests                0x00000001033a9817 content::StartupTaskRunner::RunAllTasksNow() + 39
23  interactive_ui_tests                0x0000000102f9ebc8 content::BrowserMainLoop::CreateStartupTasks() + 712
24  interactive_ui_tests                0x0000000102fa29b0 content::BrowserMainRunnerImpl::Initialize(content::MainFunctionParams const&) + 96
25  interactive_ui_tests                0x0000000102f9cce4 content::BrowserMain(content::MainFunctionParams const&) + 180
26  interactive_ui_tests                0x0000000104300ec0 content::ContentMainRunnerImpl::Run(bool) + 592
27  interactive_ui_tests                0x000000010630e331 service_manager::Main(service_manager::MainParams const&) + 2849
28  interactive_ui_tests                0x00000001043000c4 content::ContentMain(content::ContentMainParams const&) + 68
29  interactive_ui_tests                0x00000001048ebe79 content::BrowserTestBase::SetUp() + 2697
30  interactive_ui_tests                0x000000010432e2f3 InProcessBrowserTest::SetUp() + 723
31  interactive_ui_tests                0x0000000102304b3d testing::Test::Run() + 109
32  interactive_ui_tests                0x00000001023057d0 testing::TestInfo::Run() + 320
33  interactive_ui_tests                0x0000000102305d47 testing::TestCase::Run() + 279
34  interactive_ui_tests                0x0000000102311337 testing::internal::UnitTestImpl::RunAllTests() + 871
35  interactive_ui_tests                0x0000000102310fad testing::UnitTest::Run() + 109
36  interactive_ui_tests                0x0000000104e303f6 base::TestSuite::Run() + 166
37  interactive_ui_tests                0x000000010221f6b3 InteractiveUITestSuiteRunner::RunTestSuite(int, char**) + 51
38  interactive_ui_tests                0x00000001048f36c8 content::LaunchTests(content::TestLauncherDelegate*, unsigned long, int, char**) + 552
39  interactive_ui_tests                0x00000001043267cd LaunchChromeTests(unsigned long, content::TestLauncherDelegate*, int, char**) + 333
40  interactive_ui_tests                0x000000010221f61c main + 92
41  libdyld.dylib                       0x00007fff8e7dd5c9 start + 1

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7fd142a74af9c3d83f76ea6daf05657211e3026a

commit 7fd142a74af9c3d83f76ea6daf05657211e3026a
Author: Guido Urdaneta <guidou@chromium.org>
Date: Wed Aug 22 13:59:50 2018

Disable flaky AppShimMenuControllerUITest.WindowCycling

Bug:  876623 
Change-Id: I086440a188e915894393805a9973d1419efc3d06
TBR: tapted@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1185010
Reviewed-by: Guido Urdaneta <guidou@chromium.org>
Commit-Queue: Guido Urdaneta <guidou@chromium.org>
Cr-Commit-Position: refs/heads/master@{#585002}
[modify] https://crrev.com/7fd142a74af9c3d83f76ea6daf05657211e3026a/chrome/browser/ui/cocoa/apps/app_shim_menu_controller_mac_interactive_uitest.mm

Searching for web_contents_observer_sanity_checker in crbug shows a bunch of tests disabled "for flakiness" in the last 2 days, which are hitting this same dcheck.

It seems related to changes being done to the NTP.

This specific issue can probably be resolved by loading about:blank when it does

    Browser* browser_new = new Browser(Browser::CreateParams(profile(), true));
    chrome::AddTabAt(browser_new, GURL(), -1, true);

but I suspect there's a systemic issue that should be fixed.

Ah, I finally see why seemingly random tests are failing with the culprit being NTP changes...these tests all load the NTP (or open a new tab that defaults to the NTP).


The core issue is described in https://crbug.com/786313

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2976ea78bbec3db9187f8106848776d180310807

commit 2976ea78bbec3db9187f8106848776d180310807
Author: Trent Apted <tapted@chromium.org>
Date: Mon Sep 03 05:44:18 2018

Remove chrome://flags#app-window-cycling.

Mostly this is a revert of r876623.

chrome://flags#app-window-cycling never worked right and was never
enabled by default. Remove it, and the supporting code.

In r585002 its test was disabled for unrelated reasons (NTP changes),
but there's no point keeping this around. Window cycling will eventually
be fixed properly as part of https://crbug.com/859152, and this will
probably just get in the way.

Bug: 278408, 484737,  876623 
Change-Id: I6d1e453b1c5d258191913bce211785e818ee3247
Reviewed-on: https://chromium-review.googlesource.com/1198704
Reviewed-by: Dominick Ng <dominickn@chromium.org>
Commit-Queue: Trent Apted <tapted@chromium.org>
Cr-Commit-Position: refs/heads/master@{#588332}
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/browser/about_flags.cc
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/browser/flag_descriptions.cc
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/browser/flag_descriptions.h
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/browser/ui/cocoa/apps/app_shim_menu_controller_mac.mm
[delete] https://crrev.com/234b598da7ce994cabf3c144431d7a82b087d305/chrome/browser/ui/cocoa/apps/app_shim_menu_controller_mac_interactive_uitest.mm
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/common/chrome_switches.cc
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/common/chrome_switches.h
[modify] https://crrev.com/2976ea78bbec3db9187f8106848776d180310807/chrome/test/BUILD.gn