This appeared in 8 out of last 13 builds, so it's a flake. The last three are green, so it maybe even went away.

I compared the failing and passing runs. You pasted the failing one, so here's the passing one:

https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/29616/steps/mash_browser_tests/logs/stdio

============================================================================

[1118/022611:ERROR:gpu_service_internal.cc(87)] Not implemented reached in virtual void ui::GpuServiceInternal::DidCreateOffscreenContext(const GURL &)
[1118/022613:ERROR:gpu_service_internal.cc(92)] Not implemented reached in virtual void ui::GpuServiceInternal::DidDestroyChannel(int)
[1118/022613:ERROR:gpu_service_internal.cc(96)] Not implemented reached in virtual void ui::GpuServiceInternal::DidDestroyOffscreenContext(const GURL &)
[1118/022613:ERROR:wm_shell_mus.cc(405)] Not implemented reached in virtual void ash::mus::WmShellMus::RemoveDisplayObserver(ash::WmDisplayObserver *)
[1/1] BrowserTest.Title (10237 ms)
SUCCESS: all tests passed.
[1118/022614:ERROR:native_widget_mus.cc(859)] Not implemented reached in virtual void views::NativeWidgetMus::ViewRemoved(views::View *)
[1118/022614:ERROR:session.cc(20)] Restarting service: quick_launch
[1118/022614:ERROR:session.cc(20)] Restarting service: ash
Received signal 11 SEGV_MAPERR fffffffffffff352
#0 0x000002cd6db7 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#1 0x7f79b0cdbcb0 <unknown>
#2 0x000001b3bd39 gpu::GpuChannelHost::Send()
#3 0x000001b3bf59 gpu::GpuChannelHost::OrderingBarrier()
...
#24 0x000002ccb6c6 RunMashBrowserTests()

============================================================================

In both cases, we get signal 11 (i.e. segfault).

In the passing one, we get it after BrowserTest.Title has already successfully finished. In the failing one, we get it slightly earlier - before RemoveDisplayObserver(), and before the test is finished, and thus the test is not successful.

So we always get a segfault, the only difference is the timing. One of the recent CLs uncovered the problem, but didn't cause it. The segfault is the problem, and the four NOTREACHED() instances also don't seem like something that should appear in a passing test... though I don't know yet if it's related.

The very old https://build.chromium.org/p/chromium.chromiumos/builders/Linux%20ChromiumOS%20Tests%20%281%29/builds/29421/steps/mash_browser_tests/logs/stdio already contains the 4 NOTREACHED()s but not the segfault, so they're not related.

Build 29510 only has NOTREACHED()
Build 29511 only has segfault
Build 29512 has both, and this state persists until today, even though the tests mostly pass

And actually, 29511 crashed with the same stack-trace as in comment #0. It's still the same problem.

Re #0: "The previous mash failures were a different stack trace." -> I am inclined to believe that this is just because the test was killed at a different time, as I mentioned in #1.

So, 29511 contained a single CL touching mash/, which is https://codereview.chromium.org/2503063003.

It looks like it's just replacing a string literal with a constant, but the stack trace begins in DidReceiveCompositorFrameAck inside src/services/ui/, which is in the core of this CL.

Assigning to ben@, the author, PTAL. Even if this is a red herring, you're an owner of both mash/ and services/ui, so you could at least have an idea what's going on.

Removing Sheriff-Chromium in the meantime.

this crash stack has actually been showing up for a while, though not consistently