Project: chromium Issues People Development process History Sign in
New issue
Advanced search Search tips
Issue 356453 Assertion failures on debug GPU bots from r259368
Starred by 2 users Project Member Reported by kbr@chromium.org, Mar 25, 2014 Back to list
Status: Fixed
Owner: danakj@chromium.org
Closed: Apr 2014
Cc: bajones@chromium.org, piman@chromium.org, nduca@chromium.org, zmo@chromium.org, jbau...@chromium.org
Components:
OS: Linux
Pri: 1
Type: Bug


Sign in to add a comment
http://src.chromium.org/viewvc/chrome?revision=259376&view=revision caused assertion failures on the debug GPU bots:

http://build.chromium.org/p/chromium.gpu/builders/Linux%20Debug%20%28NVIDIA%29/builds/26444

http://build.chromium.org/p/chromium.gpu.fyi/builders/Linux%20Debug%20%28NVIDIA%29/builds/735

Here's the stack trace from the log:

INFO:root:Running file://webgl.html?query=kill_after_notification
[16:16:0325/163923:ERROR:trap.cc(86)] Existing signal handler when trying to install SIGSYS
[7647:7647:0325/163923:INFO:CONSOLE(24)] "Harness injected.", source:  (24)
[26:26:0325/163923:ERROR:trap.cc(86)] Existing signal handler when trying to install SIGSYS
Received signal 11 SEGV_MAPERR 000000000000
#0 0x7f2820732f5f base::debug::StackTrace::StackTrace()
#1 0x7f282073285d base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7f2812f51cb0 <unknown>
#3 0x7f2815eaaa90 content::GpuChildThread::OnCrash()
#4 0x7f2815eaad8e IPC::Message::Dispatch<>()
#5 0x7f2815eaa1e5 content::GpuChildThread::OnControlMessageReceived()
#6 0x7f2815c0839b content::ChildThread::OnMessageReceived()
#7 0x7f281eafb7bf IPC::ChannelProxy::Context::OnDispatchMessage()
#8 0x7f281eb012b8 base::internal::RunnableAdapter<>::Run()
#9 0x7f281eb00c20 base::internal::InvokeHelper<>::MakeItSo()
#10 0x7f281eb0029b base::internal::Invoker<>::Run()
#11 0x7f2820726179 base::Callback<>::Run()
#12 0x7f28207a181c base::MessageLoop::RunTask()
#13 0x7f28207a1940 base::MessageLoop::DeferOrRunPendingTask()
#14 0x7f28207a1e65 base::MessageLoop::DoWork()
#15 0x7f28207a8b94 base::MessagePumpDefault::Run()
#16 0x7f28207a136b base::MessageLoop::RunHandler()
#17 0x7f28207e16f4 base::RunLoop::Run()
#18 0x7f28207a0a5e base::MessageLoop::Run()
#19 0x7f2815ea8c51 content::GpuMain()
#20 0x7f28155fbef8 content::RunNamedProcessTypeMain()
#21 0x7f28155fd0b7 content::ContentMainRunnerImpl::Run()
#22 0x7f28155fb45b content::ContentMain()
#23 0x7f2823e4c898 ChromeMain
#24 0x7f2823e4c83c main
#25 0x7f281096c76d __libc_start_main
#26 0x7f2823e4c749 <unknown>
  r8: 0000000000000000  r9: 00007fff16f6e3a0 r10: 0101010101010101 r11: 0000000000000246
 r12: 0000000000000000 r13: 00007f28179c6c68 r14: 0000000000000001 r15: 0000000000000000
  di: 00007fff16f6e0f8  si: 00007fff16f6e0fc  bp: 00007fff16f6e4b0  bx: 0000000000000000
  dx: 00000000ffffffff  ax: 0000000000000000  cx: 00007fff16f6e2cc  sp: 00007fff16f6e300
  ip: 00007f2815eaaa90 efl: 0000000000010246 cgf: 0000000000000033 erf: 0000000000000006
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
[26:33:0325/163924:ERROR:command_buffer_proxy_impl.cc(160)] Could not send GpuCommandBufferMsg_Initialize.
[26:33:0325/163924:ERROR:webgraphicscontext3d_command_buffer_impl.cc(382)] CommandBufferProxy::Initialize failed.
[26:33:0325/163924:ERROR:webgraphicscontext3d_command_buffer_impl.cc(401)] Failed to initialize command buffer.
[26:33:0325/163924:ERROR:webgraphicscontext3d_command_buffer_impl.cc(457)] Failed to initialize context.
[7647:7647:0325/163924:ERROR:gpu_process_transport_factory.cc(460)] Lost UI shared context.
[7647:7647:0325/163924:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://127.0.0.1:54881/webgl.html?query=kill_after_notification (0)
[7747:7747:0325/163924:ERROR:trap.cc(86)] Existing signal handler when trying to install SIGSYS
[7750:7750:0325/163924:ERROR:trap.cc(86)] Existing signal handler when trying to install SIGSYS
[16:16:0325/163924:ERROR:webgraphicscontext3d_command_buffer_impl.cc(462)] Context dead on arrival. Last error: 5
NVIDIA: could not open the device file /dev/nvidia0 (Operation not permitted).
NVIDIA: could not open the device file /dev/nvidia0 (Operation not permitted).
[16:23:0325/163924:ERROR:delegating_renderer.cc(175)] Not implemented reached in virtual void cc::DelegatingRenderer::SendManagedMemoryStats(size_t, size_t, size_t)
[7647:7647:0325/163924:FATAL:resource_provider.cc(1132)] Check failed: texture_id_. 
#0 0x7fb923641f5f base::debug::StackTrace::StackTrace()
#1 0x7fb923699fe1 logging::LogMessage::~LogMessage()
#2 0x7fb9217b9092 cc::ResourceProvider::ScopedReadLockGL::ScopedReadLockGL()
#3 0x7fb9217b9142 cc::ResourceProvider::ScopedSamplerGL::ScopedSamplerGL()
#4 0x7fb9217253dd cc::GLRenderer::DrawContentQuad()
#5 0x7fb921724dc3 cc::GLRenderer::DrawTileQuad()
#6 0x7fb92171f525 cc::GLRenderer::DoDrawQuad()
#7 0x7fb9217110cc cc::DirectRenderer::DrawRenderPass()
#8 0x7fb92171055c cc::DirectRenderer::DrawFrame()
#9 0x7fb921845b06 cc::LayerTreeHostImpl::DrawLayers()
#10 0x7fb92186e209 cc::SingleThreadProxy::DoComposite()
#11 0x7fb92186dc20 cc::SingleThreadProxy::CommitAndComposite()
#12 0x7fb92186d45b cc::SingleThreadProxy::CompositeImmediately()
#13 0x7fb92181ff4a cc::LayerTreeHost::Composite()
#14 0x7fb912ebb6fc ui::Compositor::Draw()
#15 0x7fb912ec14fc base::internal::RunnableAdapter<>::Run()
#16 0x7fb912ec102e base::internal::InvokeHelper<>::MakeItSo()
#17 0x7fb912ec033d base::internal::Invoker<>::Run()
#18 0x7fb923635179 base::Callback<>::Run()
#19 0x7fb9236b081c base::MessageLoop::RunTask()
#20 0x7fb9236b0940 base::MessageLoop::DeferOrRunPendingTask()
#21 0x7fb9236b0e65 base::MessageLoop::DoWork()
#22 0x7fb923615728 base::MessagePumpGlib::RunWithDispatcher()
#23 0x7fb923615b56 base::MessagePumpGlib::Run()
#24 0x7fb9236b036b base::MessageLoop::RunHandler()
#25 0x7fb9236f06f4 base::RunLoop::Run()
#26 0x7fb9273f32f2 ChromeBrowserMainParts::MainMessageLoopRun()
#27 0x7fb91854fc0a content::BrowserMainLoop::RunMainMessageLoopParts()
#28 0x7fb918557c02 content::BrowserMainRunnerImpl::Run()
#29 0x7fb91854b9df content::BrowserMain()
#30 0x7fb91850aef8 content::RunNamedProcessTypeMain()
#31 0x7fb91850c0b7 content::ContentMainRunnerImpl::Run()
#32 0x7fb91850a45b content::ContentMain()
#33 0x7fb926d5b898 ChromeMain
#34 0x7fb926d5b83c main
#35 0x7fb91387b76d __libc_start_main
#36 0x7fb926d5b749 <unknown>

If this isn't easy to fix please revert and re-land.

Apologies the GPU try bots didn't catch this. They build Release mode, not Debug. Issue 350201 tracks getting more assertions on the GPU bots.

 
Comment 1 by kbr@chromium.org, Mar 25, 2014
Blocking: chromium:344962
Comment 2 by danakj@chromium.org, Mar 26, 2014
Verified locally that I can reproduce this on trunk. Reverting my CL does not fix it. And I can't reproduce this on r259251 with my patch applied.
Comment 3 by danakj@chromium.org, Mar 26, 2014
Cc: jbau...@chromium.org
Local bisect says the crashing started here:

commit 65a33ce4b6feb129f23aac8661d4d9725d2c802e
Author: jbauman@chromium.org <jbauman@chromium.org@0039d316-1c4b-4281-b951-d872f2087c98>
Date:   Tue Mar 25 22:37:09 2014 +0000

    Enable delegated rendering with software compositor.
    
    When the --enable-delegated-renderer command-line flag is set, the delegated renderer should be used for both software and hardware.
    
    BUG= 327220 
    
    Review URL: https://codereview.chromium.org/202023005
    
    git-svn-id: svn://svn.chromium.org/chrome/trunk/src@259368 0039d316-1c4b-4281-b951-d872f2087c98

Comment 4 by danakj@chromium.org, Mar 26, 2014
Cc: piman@chromium.org
Labels: Cr-Internals-Compositing-Software
How a CL for software compositing can break WebGL tests remains to be seen.
Comment 5 by danakj@chromium.org, Mar 26, 2014
I believe the issue is when the GPU process crashes, the renderer is immediately dropping into software compositing, instead of retrying until fallback is true.

The crash happens because we have a software frame in the browser while the browser is in GL mode. This should be addressed by https://codereview.chromium.org/210413003/.

However, jbauman@ we should have some kind of visibility when this sort of thing happens. If we just drop the frames as per https://codereview.chromium.org/210413003/, will we hide bad situations where the renderer is in software compositing and we don't expect that in our tests?
Comment 6 by danakj@chromium.org, Mar 26, 2014
Blocking: chromium:327220
Comment 7 by danakj@chromium.org, Mar 26, 2014
Blocking: -chromium:344962
Comment 8 by danakj@chromium.org, Mar 26, 2014
This fixes the Gpu tests: https://codereview.chromium.org/212103005/
Comment 9 by jbau...@chromium.org, Mar 26, 2014
Thanks for debugging this, Dana. Should we have a way to kill the renderer if it sends a delegated frame that is invalid? That would keep failures like this more visible.
Comment 10 by kbr@chromium.org, Mar 26, 2014
Summary: Assertion failures on debug GPU bots from r259368 (was: Assertion failures on debug GPU bots from r259376)
Yes, Dana, thank you very much for debugging this. Sorry for mistakenly blaming your CL. Since the failures were intermittent, the bots didn't fail on the build immediately following r259368.

Updated the synopsis.

Project Member Comment 11 by bugdroid1@chromium.org, Mar 26, 2014
------------------------------------------------------------------
r259668 | danakj@chromium.org | 2014-03-26T19:59:33.065198Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/content/renderer/render_widget.cc?r1=259668&r2=259667&pathrev=259668

Retry gpu compositing when fallback is not true.

If the GPU process goes down, and the renderer tries to recreate its
context too soon, it will fail. We should retry using GPU compositing
until fallback is given as true from the compositor to allow fallback
into software compositing.

R=jbauman@chromium.org, piman@chromium.org
BUG= 356453 , 327220 

Review URL: https://codereview.chromium.org/212103005
-----------------------------------------------------------------
Status: Fixed
Comment 13 by kbr@chromium.org, Apr 22, 2014
Blocking: chromium:365904
Sign in to add a comment