New issue
Advanced search Search tips

Issue 817758 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 1
Type: Bug



Sign in to add a comment

Null-dereference READ in viz::GLRenderer::GLRenderer

Project Member Reported by ClusterFuzz, Mar 1 2018

Issue description

Detailed report: https://clusterfuzz.com/testcase?key=4826328319393792

Fuzzer: dstockwell-css-fuzzer
Job Type: linux_asan_content_shell_drt
Platform Id: linux

Crash Type: Null-dereference READ
Crash Address: 0x0000000005e1
Crash State:
  viz::GLRenderer::GLRenderer
  viz::Display::Initialize
  viz::TestLayerTreeFrameSink::BindToClient
  
Sanitizer: address (ASAN)

Reproducer Testcase: https://clusterfuzz.com/download?testcase_id=4826328319393792

Issue filed automatically.

See https://github.com/google/clusterfuzz-tools for more information.

Note: This crash might not be reproducible with the provided testcase. That said, for the past 14 days we've been seeing this crash frequently. If you are unable to reproduce this, please try a speculative fix based on the crash stacktrace in the report. The fix can be verified by looking at the crash statistics in the report, a day after the fix is deployed. We will auto-close the bug if the crash is not seen for 14 days.
 
Project Member

Comment 1 by ClusterFuzz, Mar 1 2018

Components: Internals>Compositing
Labels: Test-Predator-Auto-Components
Automatically applying components based on crash stacktrace and information from OWNERS files.

If this is incorrect, please apply the Test-Predator-Wrong-Components label.

Comment 2 by piman@chromium.org, Mar 2 2018

Owner: kylec...@chromium.org
Status: Assigned (was: Untriaged)
Project Member

Comment 3 by ClusterFuzz, Mar 15 2018

Labels: OS-Mac
There is a failure allocating memory for the command buffer ring buffer in the renderer.

[1:1:0100/000000.092927:ERROR:cmd_buffer_helper.cc(139)] ContextResult::kFatalFailure: CommandBufferHelper::AllocateRingBuffer() failed

It looks like a failure to allocate SharedMemory. I can't reproduce locally just running the test. If I introduce a similar error, with the third call to CommandBufferHelper::AllocateRingBuffer() failing I get that stack trace.

==1==ERROR: AddressSanitizer: SEGV on unknown address 0x0000000001d1 (pc 0x00000de69c28 bp 0x7ffdec2d2b10 sp 0x7ffdec2d2a60 T0)
==1==The signal is caused by a READ memory access.
==1==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
==1==WARNING: invalid path to external symbolizer!
==1==WARNING: Failed to use and restart external symbolizer!
    #0 0xde69c27 in viz::GLRenderer::GLRenderer(viz::RendererSettings const*, viz::OutputSurface*, cc::DisplayResourceProvider*, scoped_refptr<base::SingleThreadTaskRunner>) ./../../components/viz/service/display/gl_renderer.cc:0:29
    #1 0xde15a63 in make_unique<viz::GLRenderer, const viz::RendererSettings *, viz::OutputSurface *, cc::DisplayResourceProvider *, scoped_refptr<base::SingleThreadTaskRunner> &> ./../../buildtools/third_party/libc++/trunk/include/memory:3079:32
    #2 0xde15a63 in viz::Display::InitializeRenderer() ./../../components/viz/service/display/display.cc:211:0
    #3 0xde15421 in viz::Display::Initialize(viz::DisplayClient*, viz::SurfaceManager*) ./../../components/viz/service/display/display.cc:96:3
    #4 0x161c4dae in viz::TestLayerTreeFrameSink::BindToClient(cc::LayerTreeFrameSinkClient*) ./../../components/viz/test/test_layer_tree_frame_sink.cc:131:13
    #5 0xcac1b32 in cc::LayerTreeHostImpl::InitializeRenderer(cc::LayerTreeFrameSink*) ./../../cc/trees/layer_tree_host_impl.cc:2802:31
    #6 0xcd00050 in cc::SingleThreadProxy::SetLayerTreeFrameSink(cc::LayerTreeFrameSink*) ./../../cc/trees/single_thread_proxy.cc:142:27
    #7 0xca78a7a in cc::LayerTreeHost::SetLayerTreeFrameSink(std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >) ./../../cc/trees/layer_tree_host.cc:459:11
    #8 0x13d548b0 in content::RenderWidgetCompositor::SetLayerTreeFrameSink(std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >) ./../../content/renderer/gpu/render_widget_compositor.cc:1009:21
    #9 0x13d59b12 in Invoke<const base::WeakPtr<content::RenderWidgetCompositor> &, std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> > > ./../../base/bind_internal.h:447:12
    #10 0x13d59b12 in MakeItSo<void (content::RenderWidgetCompositor::*const &)(std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >), const base::WeakPtr<content::RenderWidgetCompositor> &, std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> > > ./../../base/bind_internal.h:550:0
    #11 0x13d59b12 in RunImpl<void (content::RenderWidgetCompositor::*const &)(std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >), const std::__1::tuple<base::WeakPtr<content::RenderWidgetCompositor> > &, 0> ./../../base/bind_internal.h:604:0
    #12 0x13d59b12 in base::internal::Invoker<base::internal::BindState<void (content::RenderWidgetCompositor::*)(std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >), base::WeakPtr<content::RenderWidgetCompositor> >, void (std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >)>::Run(base::internal::BindStateBase*, std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >&&) ./../../base/bind_internal.h:586:0
    #13 0x13ebc06b in Run ./../../base/callback.h:124:12
    #14 0x13ebc06b in content::RenderThreadImpl::RequestNewLayerTreeFrameSink(int, scoped_refptr<content::FrameSwapMessageQueue>, GURL const&, base::RepeatingCallback<void (std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >)> const&, mojo::InterfaceRequest<content::mojom::RenderFrameMetadataObserverClient>, mojo::InterfacePtr<content::mojom::RenderFrameMetadataObserver>) ./../../content/renderer/render_thread_impl.cc:2185:0
    #15 0x14dd250e in content::RenderWidget::RequestNewLayerTreeFrameSink(base::RepeatingCallback<void (std::__1::unique_ptr<cc::LayerTreeFrameSink, std::__1::default_delete<cc::LayerTreeFrameSink> >)> const&) ./../../content/renderer/render_widget.cc:1058:32
    #16 0x13d57157 in content::RenderWidgetCompositor::RequestNewLayerTreeFrameSink() ./../../content/renderer/gpu/render_widget_compositor.cc:1230:14
    #17 0xcd06f54 in RequestNewLayerTreeFrameSink ./../../cc/trees/single_thread_proxy.cc:122:21
    #18 0xcd06f54 in cc::SingleThreadProxy::CompositeImmediately(base::TimeTicks, bool) ./../../cc/trees/single_thread_proxy.cc:505:0
    #19 0x13d5461a in content::RenderWidgetCompositor::SynchronouslyComposite(bool, std::__1::unique_ptr<cc::SwapPromise, std::__1::default_delete<cc::SwapPromise> >) ./../../content/renderer/gpu/render_widget_compositor.cc:1096:21
    #20 0x13d55393 in content::RenderWidgetCompositor::SynchronouslyCompositeNoRasterForTesting() ./../../content/renderer/gpu/render_widget_compositor.cc:1065:3
    #21 0x1614f49f in test_runner::WebWidgetTestClient::AnimateNow() ./../../content/shell/test_runner/web_widget_test_client.cc:53:15
    #22 0x9a2a677 in Run ./../../base/callback.h:95:12
    #23 0x9a2a677 in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) ./../../base/debug/task_annotator.cc:61:0
    #24 0x76af365 in blink::scheduler::internal::ThreadControllerImpl::DoWork(blink::scheduler::internal::SequencedTaskSource::WorkType) ./../../third_party/WebKit/Source/platform/scheduler/base/thread_controller_impl.cc:162:21
    #25 0x9a2a677 in Run ./../../base/callback.h:95:12
    #26 0x9a2a677 in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) ./../../base/debug/task_annotator.cc:61:0
    #27 0x9a9eb9e in base::MessageLoop::RunTask(base::PendingTask*) ./../../base/message_loop/message_loop.cc:391:25
    #28 0x9aa00bf in DeferOrRunPendingTask ./../../base/message_loop/message_loop.cc:403:5
    #29 0x9aa00bf in base::MessageLoop::DoWork() ./../../base/message_loop/message_loop.cc:447:0
    #30 0x9aa8c8f in base::MessagePumpDefault::Run(base::MessagePump::Delegate*) ./../../base/message_loop/message_pump_default.cc:37:31
    #31 0x9b23e4b in base::RunLoop::Run() ./../../base/run_loop.cc:130:14
    #32 0x14e9c300 in content::RendererMain(content::MainFunctionParams const&) ./../../content/renderer/renderer_main.cc:247:23
    #33 0x78c445a in content::RunZygote(content::ContentMainDelegate*) ./../../content/app/content_main_runner.cc:352:14
    #34 0x78c7dd7 in content::ContentMainRunnerImpl::Run() ./../../content/app/content_main_runner.cc:703:12
    #35 0xe3f6f4c in service_manager::Main(service_manager::MainParams const&) ./../../services/service_manager/embedder/main.cc:453:29
    #36 0x52dc1d2 in content::ContentMain(content::ContentMainParams const&) ./../../content/app/content_main.cc:19:10
    #37 0x2fbb7a8 in main ./../../content/shell/app/shell_main.cc:48:10
    #38 0x7fdf117a62b0 in __libc_start_main ??:0:0
Cc: mmoroz@chromium.org
+mmoroz

Is there a proper way to fatally fail that ClusterFuzz will ignore? If we can't allocate SharedMemory for the command buffer the test should just crash there. Will ClusterFuzz pick up that crash as a different bug?

Comment 6 by mmoroz@chromium.org, Mar 23 2018

So this is Out of Memory crash happening on ClusterFuzz? If it is not a valid bug, please close it as WontFix and add ClusterFuzz-Ignore label.
The crash is caused by some error with allocating SharedMemory. It could be file handle related. It's hard to say without being able to reproduce the real reason for the crash. We aren't bothering to check if BindToCurrentThread() was successful here:

https://cs.chromium.org/chromium/src/content/test/layouttest_support.cc?l=394&rcl=2636fba313a00683509bc25cf4e79ad1f917ee49

Later on we use something uninitialized as a result. The test can't use software compositing so there's not much to do but crash at that point.

I guess ClusterFuzz would find that as a separate crash and I should mark that as WontFix/ClusterFuzz-Ignore?

Comment 8 by mmoroz@chromium.org, Mar 23 2018

If another crash has another stacktrace, yes, it will be a separate one.
Project Member

Comment 9 by bugdroid1@chromium.org, Mar 27 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8f76378d1ae6d9d6ce6faddc3cc7c91c6ea9e3fa

commit 8f76378d1ae6d9d6ce6faddc3cc7c91c6ea9e3fa
Author: kylechar <kylechar@chromium.org>
Date: Tue Mar 27 12:13:25 2018

Check context result in layout tests.

This CL adds a check when initializing the ContextProvider for layout
tests. We should retry the on transient failures but just crash for
fatal failures. There is no compositing mode fallback in layout tests
for simplicity, so we can't do much other than crash here.

The reason for this change is that ClusterFuzz was picking up access to
some uninitialized data in some vaguely related code. This happens
because there is a fatal failure in CreateDisplayOutputSurface() that is
ignored. The crash stack should now pinpoint exactly where the failure
was and make problems easier to diagnose.

Bug:  817758 
Change-Id: I766c0526e0e2c01d16be922f78de1b89365ae401
Reviewed-on: https://chromium-review.googlesource.com/980653
Reviewed-by: Antoine Labour <piman@chromium.org>
Commit-Queue: kylechar <kylechar@chromium.org>
Cr-Commit-Position: refs/heads/master@{#546085}
[modify] https://crrev.com/8f76378d1ae6d9d6ce6faddc3cc7c91c6ea9e3fa/content/test/layouttest_support.cc

Status: Fixed (was: Assigned)
The crash rate dropped to zero after that CL landed so this issue is fixed.

I expect there will be a new ClusterFuzz crash with call stack ending in LayoutTestDependenciesImpl::CreateDisplayOutputSurface(). If someone is reading this after seeing that crash, check for the same failure allocating a ring buffer with a log message like this:

[1:1:0100/000000.092927:ERROR:cmd_buffer_helper.cc(139)] ContextResult::kFatalFailure: CommandBufferHelper::AllocateRingBuffer() failed

The crash is expected if we can't allocate a ring buffer. Why we can't allocate the ring buffer might be worth looking into?

Sign in to add a comment