New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 607048 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Long OOO (go/where-is-mgiuca)
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 1
Type: Bug-Regression



Sign in to add a comment

GPU-context-loss-related crashes and failures on builders

Project Member Reported by mgiuca@chromium.org, Apr 27 2016

Issue description

Build is broken:
content_browsertests on Ubuntu-12.04

Revision range:
chromium 389939 : 389949

Failing builders:
Linux Tests (dbg)(1): https://build.chromium.org/p/chromium.linux/builders/Linux%20Tests%20(dbg)(1)

Received signal 11 SEGV_MAPERR 000000000000
#0 0x7f9bee4e756e base::debug::StackTrace::StackTrace()
#1 0x7f9bee4e70af base::debug::(anonymous namespace)::StackDumpSignalHandler()
#2 0x7f9be0d80cb0 <unknown>
#3 0x7f9bef1be5b3 content::GpuChildThread::OnCrash()
#4 0x7f9bef1cf60d _ZN4base20DispatchToMethodImplIPN7content14GpuChildThreadEMS2_FvvEJEJEEEvRKT_T0_RKSt5tupleIJDpT1_EENS_13IndexSequenceIJXspT2_EEEE
#5 0x7f9bef1cf585 _ZN4base16DispatchToMethodIPN7content14GpuChildThreadEMS2_FvvEJEEEvRKT_T0_RKSt5tupleIJDpT1_EE
#6 0x7f9bef1cf52f _ZN3IPC16DispatchToMethodIN7content14GpuChildThreadEMS2_FvvEvSt5tupleIJEEEEvPT_T0_PT1_RKT2_
#7 0x7f9bef1c221f _ZN3IPC8MessageTI17GpuMsg_Crash_MetaSt5tupleIJEEvE8DispatchIN7content14GpuChildThreadES7_vMS7_FvvEEEbPKNS_7MessageEPT_PT0_PT1_T2_
#8 0x7f9bef1bd6b1 content::GpuChildThread::OnControlMessageReceived()
#9 0x7f9bef2846c8 content::ChildThreadImpl::OnMessageReceived()
#10 0x7f9bef1be98d content::GpuChildThread::OnMessageReceived()
#11 0x7f9bed2cd8d8 IPC::ChannelProxy::Context::OnDispatchMessage()
#12 0x7f9bed2d2076 _ZN4base8internal15RunnableAdapterIMN3IPC12ChannelProxy7ContextEFvRKNS2_7MessageEEE3RunIS4_JS7_EEEvRK13scoped_refptrIT_EDpOT0_
#13 0x7f9bed2d1f8e _ZN4base8internal12InvokeHelperILb0EvNS0_15RunnableAdapterIMN3IPC12ChannelProxy7ContextEFvRKNS3_7MessageEEEEE8MakeItSoIJRK13scoped_refptrIS5_ES8_EEEvSB_DpOT_
#14 0x7f9bed2d1f3d _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0ELm1EEEENS0_9BindStateINS0_15RunnableAdapterIMN3IPC12ChannelProxy7ContextEFvRKNS6_7MessageEEEEFvPS8_SB_EJSF_SB_EEENS0_12InvokeHelperILb0EvSE_EEFvvEE3RunEPNS0_13BindStateBaseE
#15 0x7f9bee4c782e base::Callback<>::Run()
#16 0x7f9bee4ecfbe base::debug::TaskAnnotator::RunTask()
#17 0x7f9bee5652fc base::MessageLoop::RunTask()
#18 0x7f9bee565598 base::MessageLoop::DeferOrRunPendingTask()
#19 0x7f9bee565762 base::MessageLoop::DoWork()
#20 0x7f9bee577c5c base::MessagePumpGlib::HandleDispatch()
#21 0x7f9bee578421 base::(anonymous namespace)::WorkSourceDispatch()
#22 0x7f9be3d4cd13 g_main_context_dispatch
#23 0x7f9be3d4d060 <unknown>
#24 0x7f9be3d4d124 g_main_context_iteration
#25 0x7f9bee577d5f base::MessagePumpGlib::Run()
#26 0x7f9bee564cef base::MessageLoop::RunHandler()
#27 0x7f9bee60c6c4 base::RunLoop::Run()
#28 0x7f9bee563d64 base::MessageLoop::Run()
#29 0x7f9bef1d676b content::GpuMain()
#30 0x7f9bf1687336 content::RunNamedProcessTypeMain()
#31 0x7f9bf1688fa9 content::ContentMainRunnerImpl::Run()
#32 0x7f9bf16867a2 content::ContentMain()
#33 0x000000b3eb8e content::LaunchTests()
#34 0x000000a12918 main
#35 0x7f9be09d476d __libc_start_main
#36 0x0000005d5f65 <unknown>
  r8: 2fc038b16143b700  r9: 0000000000000000 r10: 00007f9be0ae4cd0 r11: 0000000000000246
 r12: 0000000000000021 r13: 0000305fce083150 r14: 0000000000000000 r15: 0000305fcde5cf20
  di: 00007ffe8947569c  si: 00007ffe89475698  bp: 00007ffe89475a70  bx: 0000305fcde31e80
  dx: 0000000000000001  ax: 0000000000000000  cx: 2fc038b16143b700  sp: 00007ffe894758c0
  ip: 00007f9bef1be5b3 efl: 0000000000010246 cgf: 0000000000000033 erf: 0000000000000006
 trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
[10229:10229:0426/192148:16323892251:FATAL:program_binding.h(95)] Check failed: IsContextLost(context_provider->ContextGL()).
#0 0x7f14738c156e base::debug::StackTrace::StackTrace()
#1 0x7f14739222ef logging::LogMessage::~LogMessage()
#2 0x7f14715f8ab4 cc::ProgramBinding<>::Initialize()
#3 0x7f14715e8c11 cc::ProgramBinding<>::Initialize()
#4 0x7f14715db068 cc::GLRenderer::GetSolidColorProgram()
#5 0x7f14715d156f cc::GLRenderer::DrawSolidColorQuad()
#6 0x7f14715cd3a1 cc::GLRenderer::DoDrawQuad()
#7 0x7f14715b96a5 cc::DirectRenderer::DrawRenderPass()
#8 0x7f14715b8453 cc::DirectRenderer::DrawRenderPassAndExecuteCopyRequests()
#9 0x7f14715b82fe cc::DirectRenderer::DrawFrame()
#10 0x7f146efa35b9 cc::Display::DrawAndSwap()
#11 0x7f146efb7e72 cc::DisplayScheduler::DrawAndSwap()
#12 0x7f146efb75a2 cc::DisplayScheduler::AttemptDrawAndSwap()
#13 0x7f146efb6fcc cc::DisplayScheduler::OnBeginFrameDeadline()
#14 0x7f146efbbfb9 _ZN4base8internal15RunnableAdapterIMN2cc16DisplaySchedulerEFvvEE3RunIJEEEvPS3_DpOT_
#15 0x7f146efbbec0 _ZN4base8internal12InvokeHelperILb1EvNS0_15RunnableAdapterIMN2cc16DisplaySchedulerEFvvEEEE8MakeItSoINS_7WeakPtrIS4_EEJEEEvS7_T_DpOT0_
#16 0x7f146efbbe6d _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMN2cc16DisplaySchedulerEFvvEEEFvPS7_EJNS_7WeakPtrIS7_EEEEENS0_12InvokeHelperILb1EvSA_EEFvvEE3RunEPNS0_13BindStateBaseE
#17 0x7f146efc321e base::Callback<>::Run()
#18 0x7f146efc2f79 base::CancelableCallback<>::Forward()
#19 0x7f146efbbfb9 _ZN4base8internal15RunnableAdapterIMN2cc16DisplaySchedulerEFvvEE3RunIJEEEvPS3_DpOT_
#20 0x7f146efbbec0 _ZN4base8internal12InvokeHelperILb1EvNS0_15RunnableAdapterIMN2cc16DisplaySchedulerEFvvEEEE8MakeItSoINS_7WeakPtrIS4_EEJEEEvS7_T_DpOT0_
#21 0x7f146efc319d _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMNS_18CancelableCallbackIFvvEEEKFvvEEEFvPKS8_EJNS_7WeakPtrIS8_EEEEENS0_12InvokeHelperILb1EvSB_EES7_E3RunEPNS0_13BindStateBaseE
#22 0x7f14738a182e base::Callback<>::Run()
#23 0x7f14738c6fbe base::debug::TaskAnnotator::RunTask()
#24 0x7f147393f2fc base::MessageLoop::RunTask()
#25 0x7f147393f598 base::MessageLoop::DeferOrRunPendingTask()
#26 0x7f147393f9c6 base::MessageLoop::DoDelayedWork()
#27 0x7f1473951caa base::MessagePumpGlib::HandleDispatch()
#28 0x7f1473952421 base::(anonymous namespace)::WorkSourceDispatch()
#29 0x7f1469126d13 g_main_context_dispatch
#30 0x7f1469127060 <unknown>
#31 0x7f1469127124 g_main_context_iteration
#32 0x7f1473951d5f base::MessagePumpGlib::Run()
#33 0x7f147393ecef base::MessageLoop::RunHandler()
#34 0x7f14739e66c4 base::RunLoop::Run()
#35 0x00000072fe08 content::BrowserGpuChannelHostFactoryTest__Test::RunTestOnMainThread()
#36 0x000000a026dd content::ContentBrowserTest::RunTestOnMainThreadLoop()
#37 0x0000009ecc67 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop()
#38 0x000000858bfd _ZN4base8internal15RunnableAdapterIMN7content24ServiceWorkerBrowserTestEFvvEE3RunIS3_JEEEvRK13scoped_refptrIT_EDpOT0_
#39 0x0000009ee1a9 _ZN4base8internal12InvokeHelperILb0EvNS0_15RunnableAdapterIMN7content15BrowserTestBaseEFvvEEEE8MakeItSoIJRK13scoped_refptrIS4_EEEEvS7_DpOT_
#40 0x0000009ee170 _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMN7content15BrowserTestBaseEFvvEEEFvPS7_EJSB_EEENS0_12InvokeHelperILb0EvSA_EEFvvEE3RunEPNS0_13BindStateBaseE
#41 0x00000062e88e base::Callback<>::Run()
#42 0x000001aeae25 content::ShellBrowserMainParts::PreMainMessageLoopRun()
#43 0x7f1474ed2c14 content::BrowserMainLoop::PreMainMessageLoopRun()
#44 0x7f14745b7e29 _ZN4base8internal15RunnableAdapterIMN7content17GpuWatchdogThreadEFvvEE3RunIJEEEvPS3_DpOT_
#45 0x7f1474edae29 _ZN4base8internal12InvokeHelperILb0EiNS0_15RunnableAdapterIMN7content15BrowserMainLoopEFivEEEE8MakeItSoIJPS4_EEEiS7_DpOT_
#46 0x7f1474edaded _ZN4base8internal7InvokerINS_13IndexSequenceIJLm0EEEENS0_9BindStateINS0_15RunnableAdapterIMN7content15BrowserMainLoopEFivEEEFiPS7_EJNS0_17UnretainedWrapperIS7_EEEEENS0_12InvokeHelperILb0EiSA_EEFivEE3RunEPNS0_13BindStateBaseE
#47 0x7f1474608bde base::Callback<>::Run()
#48 0x7f14758ead0b content::StartupTaskRunner::RunAllTasksNow()
#49 0x7f1474ed0d95 content::BrowserMainLoop::CreateStartupTasks()
#50 0x7f1474edce6d content::BrowserMainRunnerImpl::Initialize()
#51 0x000001aa06a4 ShellBrowserMain()
#52 0x000001a821f6 content::ShellMainDelegate::RunProcess()
#53 0x7f1476a612eb content::RunNamedProcessTypeMain()
#54 0x7f1476a62fa9 content::ContentMainRunnerImpl::Run()
#55 0x7f1476a607a2 content::ContentMain()
#56 0x0000009ec9dc content::BrowserTestBase::SetUp()
#57 0x000000a0255d content::ContentBrowserTest::SetUp()
#58 0x000000be6b8a _ZN7testing8internal12InvokeHelperIvNSt3tr15tupleIJEEEE12InvokeMethodIN5media26MockVideoDecodeAcceleratorEMS8_FvvEEEvPT_T0_RKS4_
#59 0x000001b54c2e testing::internal::HandleExceptionsInMethodIfSupported<>()
#60 0x000001b49af3 testing::Test::Run()
#61 0x000001b4a2e8 testing::TestInfo::Run()
 
Project Member

Comment 1 by bugdroid1@chromium.org, Apr 27 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2cc4cad984106eeb0309f2af08a35cad6c53a59f

commit 2cc4cad984106eeb0309f2af08a35cad6c53a59f
Author: mgiuca <mgiuca@chromium.org>
Date: Wed Apr 27 03:32:54 2016

Revert of Report lost context from GLES2Implementation based on share group state. (patchset #3 id:40001 of https://codereview.chromium.org/1916303003/ )

Reason for revert:
Seems to be crashing tests. See two attached bugs:

1. BrowserGpuChannelHostFactoryTest crashes in GPU-related code. DCHECK(IsContextLost(context_provider->ContextGL())) is failing in the second stack trace, which leads me to this CL.  https://crbug.com/607048 .
2. virtual/gpu/fast/canvas/canvas-lost-gpu-context.html LayoutTest is failing with "Graphics context loss did not destroy canvas contents." The mention of context loss also leads me to this CL.  https://crbug.com/607049 .

Speculatively reverting.

BUG= 607048 , 607049 

Original issue's description:
> Report lost context from GLES2Implementation based on share group state.
>
> When any context in a gles2::ShareGroup becomes lost, all contexts in
> the same group should report that they are lost immediately. This avoids
> races such as the following:
>
> IPC thread hears of the loss of Context 1 and 2.
> IPC thread posts to thread B for Context 1.
> Context 1 hears it is lost on thread B. Posts to thread A to recreate.
> IPC thread posts to thread A for Context 2.
> Context 1 is recreated on thread A. Puts itself in same share group as 2.
> Context 2 hears it is lost on thread A, makes a new share group.
>
> This can lead to racey cases where share groups get split apart.
>
> Resolve this by storing a |lost_| state on gles2::ShareGroup. When
> GLES2Implementation hears of OnGpuControlLostContext, it will set the
> lost state on the ShareGroup. Now glGetGraphicsResetStatusKHR() will
> return an error if the gles2::ShareGroup has been lost.
>
> We remove the GpuControl::IsChannelLost() method as this is now unused.
> The CmdBufferHelper::IsContextLost() is still needed for the
> QueryTracker, so insert a comment and a TODO explaining why and how
> it could be removed.
>
> R=piman, sievers
> BUG= 584497 
> CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_optional_gpu_tests_rel;tryserver.chromium.mac:mac_optional_gpu_tests_rel;tryserver.chromium.win:win_optional_gpu_tests_rel
>
> Committed: https://crrev.com/b1f17d6f3f988d7b190cab1147b1e20da0647a44
> Cr-Commit-Position: refs/heads/master@{#389947}

TBR=piman@chromium.org,sievers@chromium.org,fsamuel@google.com,sky@chromium.org,fsamuel@chromium.org,danakj@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG= 584497 

Review URL: https://codereview.chromium.org/1919203003

Cr-Commit-Position: refs/heads/master@{#389980}

[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/cc/output/gl_renderer_unittest.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/components/mus/gles2/command_buffer_local.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/components/mus/gles2/command_buffer_local.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/client_test_helper.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/gles2_implementation.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/gles2_implementation.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/gles2_implementation_unittest.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/gpu_control.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/query_tracker.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/share_group.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/client/share_group.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/service/in_process_command_buffer.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/service/in_process_command_buffer.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/tests/gl_manager.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/command_buffer/tests/gl_manager.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/gles2_conform_support/egl/display.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/gles2_conform_support/egl/display.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/ipc/client/command_buffer_proxy_impl.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/gpu/ipc/client/command_buffer_proxy_impl.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/mojo/gles2/command_buffer_client_impl.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/mojo/gles2/command_buffer_client_impl.h
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/ppapi/proxy/ppapi_command_buffer_proxy.cc
[modify] https://crrev.com/2cc4cad984106eeb0309f2af08a35cad6c53a59f/ppapi/proxy/ppapi_command_buffer_proxy.h

Comment 3 by mgiuca@chromium.org, Apr 27 2016

Cc: mgiuca@chromium.org
 Issue 607049  has been merged into this issue.

Comment 4 by mgiuca@chromium.org, Apr 27 2016

Status: Started (was: Available)
Well now I'm completely confused:

The Cast Linux bot started failing reliably *after* I reverted r389947 (commit r389980).
https://build.chromium.org/p/chromium.linux/builders/Cast%20Linux

How did my revert start to cause a reliable test failure that wasn't failing before r389947?

Comment 5 by mgiuca@chromium.org, Apr 27 2016

Oh damn, danakj landed a new test (GLES2ImplementationTest.SignalSyncTokenAfterContextLoss) in r389958 which relies upon r389947 (which I reverted). So that new test is now failing.

I'll temporarily disable the test so we don't have to revert two CLs, and you can re-enable it when you re-land r389947.
Project Member

Comment 6 by bugdroid1@chromium.org, Apr 27 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a4ddadedbaab3e60e3aa856a6b5447cd06ea08ae

commit a4ddadedbaab3e60e3aa856a6b5447cd06ea08ae
Author: mgiuca <mgiuca@chromium.org>
Date: Wed Apr 27 04:35:52 2016

Disable GLES2ImplementationTest.SignalSyncTokenAfterContextLoss.

This test depends on r389947 but that CL was reverted in r389980.
Disabling the test to fix gpu_unittests. It should be re-enabled as part
of the re-land of r389947.

NOTRY=true
TBR=danakj@chromium.org
BUG= 607048 

Review URL: https://codereview.chromium.org/1923733003

Cr-Commit-Position: refs/heads/master@{#389990}

[modify] https://crrev.com/a4ddadedbaab3e60e3aa856a6b5447cd06ea08ae/gpu/command_buffer/client/gles2_implementation_unittest.cc

Comment 7 by mgiuca@chromium.org, Apr 27 2016

Summary: GPU-context-loss-related crashes and failures on builders (was: BrowserGpuChannelHostFactoryTest crashes on Linux)

Comment 8 by danakj@chromium.org, Apr 28 2016

Status: Fixed (was: Started)

Sign in to add a comment