Now a lot of tests are failing, and I failed to find the culprit.

+satorux@

(Thanks, tzik@-san!)

I picked a test stacktrace https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8939979537633524080/+/steps/components_unittests/0/logs/UIDevToolsTest.WindowWidgetViewGetMatchedStylesForNode/0 and symbolized by a tool that can be downloaded the Swarm report site.

    #0 0x000000efc051 in
    #1 0x000011f49fac in ?? ??:0
    #2 0x000011f48d7d in ?? ??:0
    #3 0x7f82137fa330 in
    #4 0x000011c64b62 in ?? ??:0
#5 0x7f81ff2d0078 <unknown>
#6 0x7f81ff310a6d <unknown>
#7 0x7f81ff2f687a <unknown>
#8 0x7f81ff2f6802 <unknown>
#9 0x7f81ff312ed0 <unknown>
#10 0x7f81ff2e1d47 <unknown>
#11 0x7f81ff593c00 <unknown>
    #5 0x7f8213c1c1da in
    #6 0x7f8213c1c2c3 in ?? ??:0
    #7 0x7f8213c20d00 in ?? ??:0
    #8 0x7f8213c1c094 in ?? ??:0
    #9 0x7f8213c2044b in ?? ??:0
    #10 0x7f8213a0902b in
    #11 0x7f8213c1c094 in ?? ??:0
    #12 0x7f8213a0962d in ?? ??:0
    #13 0x7f8213a090c1 in ?? ??:0
    #14 0x000000f25298 in ?? ??:0
#22 0x7f8207b7c2df <unknown>
#23 0x7f820791f481 <unknown>
#24 0x7f82077df804 <unknown>
    #15 0x0000157493b9 in gl::GLContextOSMesa::Initialize(gl::GLSurface*, gl::GLContextAttribs const&) ./../../ui/gl/gl_context_osmesa.cc:46:14
    #16 0x000015747f75 in gl::InitializeGLContext(scoped_refptr<gl::GLContext>, gl::GLSurface*, gl::GLContextAttribs const&) ./../../ui/gl/gl_context.cc:401:17
    #17 0x00000b1e0033 in ui::GLOzoneOSMesa::CreateGLContext(gl::GLShareGroup*, gl::GLSurface*, gl::GLContextAttribs const&) ./../../ui/ozone/common/gl_ozone_osmesa.cc:60:10
    #18 0x00001691543c in gl::init::CreateGLContext(gl::GLShareGroup*, gl::GLSurface*, gl::GLContextAttribs const&) ./../../ui/gl/init/gl_factory_ozone.cc:39:26
    #19 0x0000220427c3 in gpu::InProcessCommandBuffer::InitializeOnGpuThread(gpu::InProcessCommandBuffer::InitializeOnGpuThreadParams const&) ./../../gpu/ipc/in_process_command_buffer.cc:400:20
    #20 0x000022057173 in Run ./../../base/callback.h:99:12
    #21 0x000022057173 in operator() ./../../gpu/ipc/in_process_command_buffer.cc:92:0
    #22 0x000022057173 in Invoke<(lambda at ../../gpu/ipc/in_process_command_buffer.cc:90:18), base::OnceCallback<gpu::ContextResult ()>, gpu::ContextResult *, base::WaitableEvent *> ./../../base/bind_internal.h:403:0
    #23 0x000022057173 in MakeItSo<(lambda at ../../gpu/ipc/in_process_command_buffer.cc:90:18), base::OnceCallback<gpu::ContextResult ()>, gpu::ContextResult *, base::WaitableEvent *> ./../../base/bind_internal.h:616:0
    #24 0x000022057173 in RunImpl<(lambda at ../../gpu/ipc/in_process_command_buffer.cc:90:18), std::__1::tuple<base::OnceCallback<gpu::ContextResult ()>, gpu::ContextResult *, base::WaitableEvent *>, 0, 1, 2> ./../../base/bind_internal.h:689:0
    #25 0x000022057173 in base::internal::Invoker<base::internal::BindState<base::OnceCallback<void ()> gpu::(anonymous namespace)::WrapTaskWithResult<gpu::ContextResult>(base::OnceCallback<gpu::ContextResult ()>, gpu::ContextResult*, base::WaitableEvent*)::'lambda'(base::OnceCallback<gpu::ContextResult ()>, gpu::ContextResult*, base::WaitableEvent*), base::On\
ceCallback<gpu::ContextResult ()>, gpu::ContextResult*, base::WaitableEvent*>, void ()>::RunOnce(base::internal::BindStateBase*) ./../../base/bind_internal.h:658:0
    #26 0x000011fa8545 in Run ./../../base/callback.h:99:12
    #27 0x000011fa8545 in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) ./../../base/debug/task_annotator.cc:101:0
    #28 0x000011d391e9 in base::MessageLoop::RunTask(base::PendingTask*) ./../../base/message_loop/message_loop.cc:421:46
    #29 0x000011d3a366 in base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) ./../../base/message_loop/message_loop.cc:432:5
    #30 0x000011d3ac17 in base::MessageLoop::DoWork() ./../../base/message_loop/message_loop.cc:480:16
    #31 0x000011d44d86 in base::MessagePumpDefault::Run(base::MessagePump::Delegate*) ./../../base/message_loop/message_pump_default.cc:37:31
    #32 0x000011d37da5 in base::MessageLoop::Run(bool) ./../../base/message_loop/message_loop.cc:373:12
    #33 0x000011dc4efe in base::RunLoop::Run() ./../../base/run_loop.cc:102:14
    #34 0x000011eabfa8 in base::Thread::Run(base::RunLoop*) ./../../base/threading/thread.cc:255:13
    #35 0x000011eac960 in base::Thread::ThreadMain() ./../../base/threading/thread.cc:337:3
    #36 0x000011f838b3 in base::(anonymous namespace)::ThreadFunc(void*) ./../../base/threading/platform_thread_posix.cc:76:13
    #37 0x7f82137f2184 in ?? ??:0
    #38 0x7f820f0b703d in

It looks like https://chromium-review.googlesource.com/c/chromium/src/+/1150662 is the culprit. I'll revert this.

Revert landed in https://crrev.com/c/1150062. Linux Chromium OS ASan LSan Tests (1) hasn't caught up yet though. 

Wait, nevermind, the bot has caught up and the revert was included in this run:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Chromium%20OS%20ASan%20LSan%20Tests%20%281%29/28398

That didn't fix the problem. https://crrev.com/c/1150662 wasn't the issue, so I'll reland it. 

Here is the first failing run, it must be something else from that:

https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Chromium%20OS%20ASan%20LSan%20Tests%20%281%29/28394

I don't see any other CLs in the suspect change list that are obviously the culprit. The failure is CrOS only so probably a change to CrOS code. There are a couple CLs that touch CrOS code tho.

I tried running some of the failing tests with ASan at ToT (and ToT plus https://crrev.com/c/1150662) and they're not failing, so maybe the problem was fixed? I'm not going to look at this any further, it's not https://crrev.com/c/1150662 and I can't reproduce.