sky@: can you please help triage this, since you've looked at previous flakiness in mash_browser_tests in  issue 626869 ?  Looks like the BrowserTest.Title test started occasionally crashing around 8/17, only on linux_chromium_chromeos_ozone_rel_ng.

The stack is:
../../third_party/tcmalloc/chromium/src/tcmalloc.cc:289] Attempt to free invalid pointer 0xa14614dd40 
Received signal 11 SEGV_MAPERR 000000000039
#0 0x000002e8bbd7 base::debug::(anonymous namespace)::StackDumpSignalHandler()
#1 0x7f52dcb98cb0 <unknown>
#2 0x000001838330 <unknown>
#3 0x00000183bf66 tcmalloc::Log()
#4 0x000001842c62 (anonymous namespace)::do_free_with_callback()
#5 0x7f52d7fc4550 SkMallocPixelRef::~SkMallocPixelRef()
#6 0x7f52d7fc4579 SkMallocPixelRef::~SkMallocPixelRef()
#7 0x7f52d7f3dbab SkBitmap::freePixels()
#8 0x7f52d7f3da99 SkBitmap::~SkBitmap()
#9 0x7f52d7f7aa8b SkNoPixelsBitmapDevice::~SkNoPixelsBitmapDevice()
#10 0x7f52d7f70ec6 DeviceCM::~DeviceCM()
#11 0x7f52d7f6e63d SkCanvas::internalRestore()
#12 0x7f52d7f6e414 SkCanvas::~SkCanvas()
#13 0x7f52d7f313b9 skia::AnalysisCanvas::~AnalysisCanvas()
#14 0x7f52d82183b0 gfx::Canvas::~Canvas()
#15 0x7f52d821aa13 gfx::CanvasImageSource::GetImageForScale()
#16 0x7f52d820dda3 gfx::internal::ImageSkiaStorage::FindRepresentation()
#17 0x7f52d820e920 gfx::ImageSkia::GetRepresentation()
#18 0x7f52d8219005 gfx::Canvas::DrawImageInt()
#19 0x7f52d7e9cd12 ash::FrameCaptionButton::PaintCentered()
#20 0x7f52d858896d views::View::Paint()
#21 0x7f52d858a186 views::View::PaintChildren()
#22 0x7f52d858898b views::View::Paint()
#23 0x7f52d858a186 views::View::PaintChildren()
#24 0x7f52d858898b views::View::Paint()
#25 0x7f52d858a186 views::View::PaintChildren()
#26 0x7f52d858898b views::View::Paint()
#27 0x7f52d858a186 views::View::PaintChildren()
#28 0x7f52d858898b views::View::Paint()
#29 0x7f52d858a186 views::View::PaintChildren()
#30 0x7f52d858898b views::View::Paint()
#31 0x7f52d8443f90 ui::Layer::PaintContentsToDisplayList()
#32 0x7f52d84440ed ui::Layer::PaintContentsToDisplayList()
#33 0x7f52d847538b cc::PictureLayer::Update()
#34 0x7f52d84e5b59 cc::LayerTree::UpdateLayers()
#35 0x7f52d84e98ed cc::LayerTreeHost::DoUpdateLayers()
#36 0x7f52d84e94e8 cc::LayerTreeHost::UpdateLayers()
#37 0x7f52d8520dc2 cc::SingleThreadProxy::DoBeginMainFrame()
#38 0x7f52d8521a7f cc::SingleThreadProxy::BeginMainFrame()
#39 0x7f52d7d0d746 base::debug::TaskAnnotator::RunTask()
#40 0x7f52d7d1dcb5 base::MessageLoop::RunTask()
#41 0x7f52d7d1dfd8 base::MessageLoop::DeferOrRunPendingTask()
#42 0x7f52d7d1e32b base::MessageLoop::DoWork()
#43 0x7f52d7d1f77c base::MessagePumpDefault::Run()
#44 0x7f52d7d1d7fd base::MessageLoop::RunHandler()
#45 0x7f52d7d2ef30 base::RunLoop::Run()
#46 0x7f52d843048f shell::ServiceRunner::Run()
#47 0x7f52d843052c shell::ServiceRunner::Run()
#48 0x7f52d7c0ae0a ServiceMain
#49 0x00000281ef0c shell::RunNativeApplication()
#50 0x00000281c360 shell::(anonymous namespace)::RunNativeLibrary()
#51 0x000001b9df0a _ZN4base8internal7InvokerINS0_9BindStateIPFvPN7content15RenderFrameHostEN4mojo16InterfaceRequestIN5blink5mojom19PresentationServiceEEEEJNS0_17UnretainedWrapperINS3_19RenderFrameHostImplEEEEEEFvSB_EE3RunEPNS0_13BindStateBaseEOSB_
#52 0x00000281de2a shell::ChildProcessMainWithCallback()
#53 0x00000281c306 shell::ChildProcessMain()
#54 0x000002e7f438 RunMashBrowserTests()
#55 0x000002e7f133 main
#56 0x7f52d99437ed __libc_start_main
#57 0x0000005d4a1d <unknown>

There's a bunch more info and errors in the logs, e.g. https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_ozone_rel_ng/builds/220943/steps/mash_browser_tests%20%28with%20patch%29/logs/stdio

I believe Sadrul has been looking at this. I'm not sure that crash is relevant to the failure as last I looked it was in sysui, which doesn't effect the overall status of the test.

That looks like a crash in mojo:ash, and ash crashing would explain the browser-test failing to complete too.

I can repro the failure locally in non-component builds ... and, this is difficult to explain, but this seems to be an issue with tcmalloc. If I turn that off (e.g. is_asan = true, or use_allocator = 'none'), then the ash crashes go away, and mash_browser_tests stops becoming flaky.

+wfh@ as tcmalloc owner: any advice on how I can debug possible issues with tcmalloc?

 Issue 594674  seems to be relevant to this.

Yeah it's the same bug. Unfortunately it's fundamental flaw in mojo_runner that it can't work with tcmalloc in non-component builds. The workaround for now is to use component builds.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6d9ba42aa4635a02ddc20e9d155a055c76c510e5

commit 6d9ba42aa4635a02ddc20e9d155a055c76c510e5
Author: sadrul <sadrul@chromium.org>
Date: Mon Aug 22 16:29:38 2016

ash/mus: Fix a crash during tear down.

WindowManagerObserver can operate on the WMShell instance when the
WindowTreeClient is destroyed (e.g. AcceleratorRegistrarImpl). So
notify the observers before destroying the WMShell instance.

BUG= 639039 

Review-Url: https://codereview.chromium.org/2269473003
Cr-Commit-Position: refs/heads/master@{#413453}

[modify] https://crrev.com/6d9ba42aa4635a02ddc20e9d155a055c76c510e5/ash/mus/window_manager.cc

I have filed  issue 639882  to convert the FYI builder to do ozone + component builds.

Ken is working on a fix for 594674. It doesn't sound too involved, should know soonish.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/6be9df9727afb4e827c24acc708abc1a5f49b92c

commit 6be9df9727afb4e827c24acc708abc1a5f49b92c
Author: sadrul <sadrul@chromium.org>
Date: Mon Sep 12 22:24:19 2016

mash: Re-enable mash_browser_tests in ozone builder.

The tests needed to be removed from the ozone builders
because of the flak, but that has been addressed in
crrev.com/415329 It should now be safe to turn the tests
back on this builder.

BUG= 639039 

Review-Url: https://codereview.chromium.org/2257873005
Cr-Commit-Position: refs/heads/master@{#418077}

[modify] https://crrev.com/6be9df9727afb4e827c24acc708abc1a5f49b92c/testing/buildbot/chromium.chromiumos.json