New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 810642 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
OOO until 2019-01-24
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug

Blocked on:
issue 581526



Sign in to add a comment

"gpu_tests.context_lost_integration_test.ContextLostIntegrationTest.ContextLost_WebGLContextLostFromGPUProcessExit" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Feb 9 2018

Issue description

"gpu_tests.context_lost_integration_test.ContextLostIntegrationTest.ContextLost_WebGLContextLostFromGPUProcessExit" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 14 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfAsSBUZsYWtlInFncHVfdGVzdHMuY29udGV4dF9sb3N0X2ludGVncmF0aW9uX3Rlc3QuQ29udGV4dExvc3RJbnRlZ3JhdGlvblRlc3QuQ29udGV4dExvc3RfV2ViR0xDb250ZXh0TG9zdEZyb21HUFVQcm9jZXNzRXhpdAw.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 

Comment 1 by olka@chromium.org, Feb 9 2018

Components: Internals>GPU>Testing Internals>GPU
Labels: OS-Mac
Flakes on mac_chromium_rel_ng

Example
https://ci.chromium.org/buildbot/tryserver.chromium.mac/mac_chromium_rel_ng/647077

10264:775:0208/152850.038042:INFO:CONSOLE(217)] "Harness injected.", source:  (217)
Received signal 11 SEGV_MAPERR 000000000000
0   Chromium Framework                  0x0000000107cb6adc base::debug::StackTrace::StackTrace(unsigned long) + 28
1   Chromium Framework                  0x0000000107cb6911 base::debug::(anonymous namespace)::StackDumpSignalHandler(int, __siginfo*, void*) + 2401
2   libsystem_platform.dylib            0x00007fffcdc32b3a _sigtramp + 26
3   ???                                 0x0000700012fb16a0 0x0 + 123145620756128
4   Chromium Framework                  0x00000001055f2363 viz::mojom::GpuServiceStubDispatch::Accept(viz::mojom::GpuService*, mojo::Message*) + 1587
5   Chromium Framework                  0x000000010884901b mojo::InterfaceEndpointClient::HandleValidatedMessage(mojo::Message*) + 955
6   Chromium Framework                  0x0000000108848916 mojo::FilterChain::Accept(mojo::Message*) + 150
7   Chromium Framework                  0x000000010884a465 mojo::InterfaceEndpointClient::HandleIncomingMessage(mojo::Message*) + 117
8   Chromium Framework                  0x0000000108850a5d mojo::internal::MultiplexRouter::ProcessIncomingMessage(mojo::internal::MultiplexRouter::MessageWrapper*, mojo::internal::MultiplexRouter::ClientCallBehavior, base::SequencedTaskRunner*) + 1293
9   Chromium Framework                  0x000000010884ff66 mojo::internal::MultiplexRouter::Accept(mojo::Message*) + 374
10  Chromium Framework                  0x0000000108848916 mojo::FilterChain::Accept(mojo::Message*) + 150
11  Chromium Framework                  0x0000000108843b05 mojo::Connector::ReadSingleMessage(unsigned int*) + 421
12  Chromium Framework                  0x0000000108844611 mojo::Connector::ReadAllAvailableMessages() + 97
13  Chromium Framework                  0x00000001088444c9 mojo::Connector::OnHandleReadyInternal(unsigned int) + 137
14  Chromium Framework                  0x00000001055e1097 mojo::SimpleWatcher::DiscardReadyState(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&) + 103
15  Chromium Framework                  0x000000010876948c mojo::SimpleWatcher::OnHandleReady(int, unsigned int, mojo::HandleSignalsState const&) + 332
16  Chromium Framework                  0x00000001087698d7 mojo::SimpleWatcher::Context::Notify(unsigned int, MojoHandleSignalsState, unsigned int) + 855
17  Chromium Framework                  0x0000000108768494 mojo::SimpleWatcher::Context::CallNotify(unsigned long, unsigned int, MojoHandleSignalsState, unsigned int) + 20
18  Chromium Framework                  0x0000000109aded12 mojo::edk::Watch::InvokeCallback(unsigned int, mojo::HandleSignalsState const&, unsigned int) + 194
19  Chromium Framework                  0x0000000109ad9d55 mojo::edk::RequestContext::~RequestContext() + 1045
20  Chromium Framework                  0x0000000109ac9b29 mojo::edk::NodeChannel::OnChannelMessage(void const*, unsigned long, std::__1::vector<mojo::edk::ScopedPlatformHandle, std::__1::allocator<mojo::edk::ScopedPlatformHandle> >) + 345
21  Chromium Framework                  0x0000000109ab853a mojo::edk::Channel::OnReadComplete(unsigned long, unsigned long*) + 602
22  Chromium Framework                  0x0000000109ae5427 mojo::edk::(anonymous namespace)::ChannelPosix::OnFileCanReadWithoutBlocking(int) + 535
23  Chromium Framework                  0x0000000107cf9881 base::MessagePumpLibevent::OnLibeventNotification(int, short, void*) + 369
24  Chromium Framework                  0x0000000107dd70fc event_base_loop + 1228
25  Chromium Framework                  0x0000000107cf9c1c base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) + 428
26  Chromium Framework                  0x0000000107cf6709 base::MessageLoop::Run(bool) + 169
27  Chromium Framework                  0x0000000107d35419 base::RunLoop::Run() + 249
28  Chromium Framework                  0x0000000107d83d0e base::Thread::Run(base::RunLoop*) + 206
29  Chromium Framework                  0x0000000107d842cc base::Thread::ThreadMain() + 908
30  Chromium Framework                  0x0000000107d791bf base::(anonymous namespace)::ThreadFunc(void*) + 95
31  libsystem_pthread.dylib             0x00007fffcdc3c93b _pthread_body + 180
32  libsystem_pthread.dylib             0x00007fffcdc3c887 _pthread_body + 0
33  libsystem_pthread.dylib             0x00007fffcdc3c08d thread_start + 13
[end of stack trace]
[10264:775:0208/152850.647085:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://127.0.0.1:51639/gpu_process_crash.html (0)
[10277:775:0208/152851.124932:ERROR:gles2_cmd_decoder.cc(18285)] [.DisplayCompositor-0x7fc25a923400]GL ERROR :GL_INVALID_OPERATION : glCreateAndConsumeTextureCHROMIUM: invalid mailbox name
[10277:775:0208/152851.125008:ERROR:gles2_cmd_decoder.cc(12449)] [.DisplayCompositor-0x7fc25a923400]GL ERROR :GL_INVALID_VALUE : glScheduleCALayerCHROMIUM: unsupported texture format
Received signal 11 SEGV_MAPERR 000000000000
0   Chromium Framework                  0x00000001140a2adc base::debug::StackTrace::StackTrace(unsigned long) + 28
1   Chromium Framework                  0x00000001140a2911 base::debug::(anonymous namespace)::StackDumpSignalHandler(int, __siginfo*, void*) + 2401
2   libsystem_platform.dylib            0x00007fffcdc32b3a _sigtramp + 26
3   ???                                 0x000070000d77b6a0 0x0 + 123145528260256
4   Chromium Framework                  0x00000001119de363 viz::mojom::GpuServiceStubDispatch::Accept(viz::mojom::GpuService*, mojo::Message*) + 1587
5   Chromium Framework                  0x0000000114c3501b mojo::InterfaceEndpointClient::HandleValidatedMessage(mojo::Message*) + 955
6   Chromium Framework                  0x0000000114c34916 mojo::FilterChain::Accept(mojo::Message*) + 150
7   Chromium Framework                  0x0000000114c36465 mojo::InterfaceEndpointClient::HandleIncomingMessage(mojo::Message*) + 117
8   Chromium Framework                  0x0000000114c3ca5d mojo::internal::MultiplexRouter::ProcessIncomingMessage(mojo::internal::MultiplexRouter::MessageWrapper*, mojo::internal::MultiplexRouter::ClientCallBehavior, base::SequencedTaskRunner*) + 1293
9   Chromium Framework                  0x0000000114c3bf66 mojo::internal::MultiplexRouter::Accept(mojo::Message*) + 374
10  Chromium Framework                  0x0000000114c34916 mojo::FilterChain::Accept(mojo::Message*) + 150
11  Chromium Framework                  0x0000000114c2fb05 mojo::Connector::ReadSingleMessage(unsigned int*) + 421
12  Chromium Framework                  0x0000000114c30611 mojo::Connector::ReadAllAvailableMessages() + 97
13  Chromium Framework                  0x0000000114c304c9 mojo::Connector::OnHandleReadyInternal(unsigned int) + 137
14  Chromium Framework                  0x00000001119cd097 mojo::SimpleWatcher::DiscardReadyState(base::RepeatingCallback<void (unsigned int)> const&, unsigned int, mojo::HandleSignalsState const&) + 103
15  Chromium Framework                  0x0000000114b5548c mojo::SimpleWatcher::OnHandleReady(int, unsigned int, mojo::HandleSignalsState const&) + 332
16  Chromium Framework                  0x0000000114b558d7 mojo::SimpleWatcher::Context::Notify(unsigned int, MojoHandleSignalsState, unsigned int) + 855
17  Chromium Framework                  0x0000000114b54494 mojo::SimpleWatcher::Context::CallNotify(unsigned long, unsigned int, MojoHandleSignalsState, unsigned int) + 20
18  Chromium Framework                  0x0000000115ecad12 mojo::edk::Watch::InvokeCallback(unsigned int, mojo::HandleSignalsState const&, unsigned int) + 194
19  Chromium Framework                  0x0000000115ec5d55 mojo::edk::RequestContext::~RequestContext() + 1045
20  Chromium Framework                  0x0000000115eb5b29 mojo::edk::NodeChannel::OnChannelMessage(void const*, unsigned long, std::__1::vector<mojo::edk::ScopedPlatformHandle, std::__1::allocator<mojo::edk::ScopedPlatformHandle> >) + 345
21  Chromium Framework                  0x0000000115ea453a mojo::edk::Channel::OnReadComplete(unsigned long, unsigned long*) + 602
22  Chromium Framework                  0x0000000115ed1427 mojo::edk::(anonymous namespace)::ChannelPosix::OnFileCanReadWithoutBlocking(int) + 535
23  Chromium Framework                  0x00000001140e5881 base::MessagePumpLibevent::OnLibeventNotification(int, short, void*) + 369
24  Chromium Framework                  0x00000001141c30fc event_base_loop + 1228
25  Chromium Framework                  0x00000001140e5c1c base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) + 428
26  Chromium Framework                  0x00000001140e2709 base::MessageLoop::Run(bool) + 169
27  Chromium Framework                  0x0000000114121419 base::RunLoop::Run() + 249
28  Chromium Framework                  0x000000011416fd0e base::Thread::Run(base::RunLoop*) + 206
29  Chromium Framework                  0x00000001141702cc base::Thread::ThreadMain() + 908
30  Chromium Framework                  0x00000001141651bf base::(anonymous namespace)::ThreadFunc(void*) + 95
31  libsystem_pthread.dylib             0x00007fffcdc3c93b _pthread_body + 180
32  libsystem_pthread.dylib             0x00007fffcdc3c887 _pthread_body + 0
33  libsystem_pthread.dylib             0x00007fffcdc3c08d thread_start + 13
[end of stack trace]
[10264:775:0208/152856.851302:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://127.0.0.1:51639/gpu_process_crash.html (0)
[10280:775:0208/152856.869844:ERROR:gles2_cmd_decoder.cc(18285)] [.DisplayCompositor-0x7fc25c8ca400]GL ERROR :GL_INVALID_OPERATION : glCreateAndConsumeTextureCHROMIUM: invalid mailbox name
[10280:775:0208/152856.869901:ERROR:gles2_cmd_decoder.cc(12449)] [.DisplayCompositor-0x7fc25c8ca400]GL ERROR :GL_INVALID_VALUE : glScheduleCALayerCHROMIUM: unsupported texture format
Restarting browser due to must restart after tests that kill the GPU process

Comment 2 by olka@chromium.org, Feb 9 2018

Cc: kbr@chromium.org

Comment 3 by kbr@chromium.org, Feb 9 2018

Cc: sadrul@chromium.org roc...@chromium.org piman@chromium.org zmo@chromium.org fsam...@chromium.org
Components: Internals>Services>Viz
There's been a clear regression in GPU process message dispatching.

Unfortunately this is only mostly visible on the trybots and not on the waterfall bots.

https://ci.chromium.org/buildbot/chromium.gpu/Mac%20Retina%20Release%20%28AMD%29/?limit=200

Who knows about recent changes in this area?

Comment 4 by kbr@chromium.org, Feb 9 2018

Cc: danakj@chromium.org

Comment 5 by kbr@chromium.org, Feb 9 2018

Cc: samans@chromium.org
I'm looking through git logs in src/services/viz/ and other directories. From this list:

https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyfAsSBUZsYWtlInFncHVfdGVzdHMuY29udGV4dF9sb3N0X2ludGVncmF0aW9uX3Rlc3QuQ29udGV4dExvc3RJbnRlZ3JhdGlvblRlc3QuQ29udGV4dExvc3RfV2ViR0xDb250ZXh0TG9zdEZyb21HUFVQcm9jZXNzRXhpdAw

The crashes started on February 7.

We're looking for a CL around that timeframe that could have caused the above crash. Keep in mind that this test deliberately crashes the GPU process and then brings it back up.

Comment 6 by kbr@chromium.org, Feb 9 2018

Cc: dcheng@chromium.org
Components: Internals>Mojo
Maybe a semantic change to Mojo recently? Not sure. Need help triaging.

Comment 7 by kbr@chromium.org, Feb 9 2018

Cc: kylec...@chromium.org
Cc: m...@chromium.org
/cc+ miu@

The browser crash seems to be in viz::GLRenderer::CopyRenderPassDrawQuadToOverlayResource:

  Thread 0 (crashed)
   0  Chromium Framework!base::debug::BreakDebugger() + 0x11
      rax = 0x00007ffaa00e843a   rdx = 0x00007ffaa00e843a
      rcx = 0x0000000000000056   rbx = 0x00007ffaa00e843a
      rsi = 0x0000000000000db5   rdi = 0x0000000123ba1270
      rbp = 0x00007fff545b8c50   rsp = 0x00007fff545b8c50
       r8 = 0x00007ffaa00e8490    r9 = 0x0000000000001f9f
      r10 = 0x00007ffaa00e8490   r11 = 0x000000011be5a380
      r12 = 0x0000000000000056   r13 = 0x00007ffaa00e8490
      r14 = 0x00007fff545b9698   r15 = 0x00007fff545b9690
      rip = 0x000000011bd1a721
      Found by: given as instruction pointer in context
   1  Chromium Framework!logging::LogMessage::~LogMessage() + 0x8b0
      rbp = 0x00007fff545b9210   rsp = 0x00007fff545b8c60
      rip = 0x000000011bd412e0
      Found by: previous frame's frame pointer
   2  Chromium Framework!viz::GLRenderer::CopyRenderPassDrawQuadToOverlayResource(viz::CALayerOverlay const*, std::__1::unique_ptr<viz::GLRenderer::OverlayTexture, std::__1::default_delete<viz::GLRenderer::OverlayTexture> >*, gfx::RectF*) + 0xc15
      rbp = 0x00007fff545b9810   rsp = 0x00007fff545b9220
      rip = 0x000000011d9f7775
      Found by: previous frame's frame pointer
   3  Chromium Framework!viz::GLRenderer::ScheduleRenderPassDrawQuad(viz::CALayerOverlay const*) + 0xa0
      rbp = 0x00007fff545b9a40   rsp = 0x00007fff545b9820
      rip = 0x000000011d9f64d0
      Found by: previous frame's frame pointer
   4  Chromium Framework!viz::GLRenderer::ScheduleCALayers() + 0xe6
      rbp = 0x00007fff545b9c60   rsp = 0x00007fff545b9a50
      rip = 0x000000011d9f3126
      Found by: previous frame's frame pointer
   5  Chromium Framework!viz::GLRenderer::FinishDrawingFrame() + 0x21e
      rbp = 0x00007fff545b9de0   rsp = 0x00007fff545b9c70
      rip = 0x000000011d9f26ee
      Found by: previous frame's frame pointer
   6  Chromium Framework!viz::DirectRenderer::DrawFrame(std::__1::vector<std::__1::unique_ptr<viz::RenderPass, std::__1::default_delete<viz::RenderPass> >, std::__1::allocator<std::__1::unique_ptr<viz::RenderPass, std::__1::default_delete<viz::RenderPass> > > >*, float, gfx::Size const&) + 0xe79
      rbp = 0x00007fff545ba010   rsp = 0x00007fff545b9df0
      rip = 0x000000011d9d6a29
      Found by: previous frame's frame pointer
   7  Chromium Framework!viz::Display::DrawAndSwap() + 0x6c0
      rbp = 0x00007fff545ba310   rsp = 0x00007fff545ba020
      rip = 0x000000011d9dc940
      Found by: previous frame's frame pointer
   8  Chromium Framework!viz::DisplayScheduler::DrawAndSwap() + 0xec
      rbp = 0x00007fff545ba490   rsp = 0x00007fff545ba320
      rip = 0x000000011d9e06dc
      Found by: previous frame's frame pointer
   9  Chromium Framework!viz::DisplayScheduler::AttemptDrawAndSwap() + 0xdf
      rbp = 0x00007fff545ba5e0   rsp = 0x00007fff545ba4a0
      rip = 0x000000011d9dfd3f
      Found by: previous frame's frame pointer
  10  Chromium Framework!viz::DisplayScheduler::OnBeginFrameDeadline() + 0x88
      rbp = 0x00007fff545ba760   rsp = 0x00007fff545ba5f0
      rip = 0x000000011d9df608
      Found by: previous frame's frame pointer
  11  Chromium Framework!base::internal::Invoker<base::internal::BindState<void (viz::DisplayScheduler::*)(), base::WeakPtr<viz::DisplayScheduler> >, void ()>::Run(base::internal::BindStateBase*) + 0xb7
      rbp = 0x00007fff545ba8c0   rsp = 0x00007fff545ba770
      rip = 0x000000011d9e21a7
      Found by: previous frame's frame pointer
  12  Chromium Framework!void base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> >::ForwardRepeating<>() + 0x5f
      rbp = 0x00007fff545baa10   rsp = 0x00007fff545ba8d0
      rip = 0x000000011913e69f
      Found by: previous frame's frame pointer
  13  Chromium Framework!base::internal::Invoker<base::internal::BindState<void (base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> >::*)(), base::WeakPtr<base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> > > >, void ()>::Run(base::internal::BindStateBase*) + 0xb7
      rbp = 0x00007fff545bab70   rsp = 0x00007fff545baa20
      rip = 0x000000011913e807
      Found by: previous frame's frame pointer
  14  Chromium Framework!ui::WindowResizeHelperMac::WaitForSingleTaskToRun(base::TimeDelta const&) + 0x248
      rbp = 0x00007fff545bad10   rsp = 0x00007fff545bab80
      rip = 0x000000011d7242e8
      Found by: previous frame's frame pointer
  15  Chromium Framework!content::RenderWidgetHostImpl::PauseForPendingResizeOrRepaints() + 0x34a
      rbp = 0x00007fff545baef0   rsp = 0x00007fff545bad20
      rip = 0x0000000119e4880a
      Found by: previous frame's frame pointer
  16  Chromium Framework!content::RenderWidgetHostViewMac::Show() + 0x135
      rbp = 0x00007fff545bafd0   rsp = 0x00007fff545baf00
      rip = 0x0000000119e61645
      Found by: previous frame's frame pointer
  17  Chromium Framework!content::WebContentsImpl::WasShown() + 0x45
      rbp = 0x00007fff545bb170   rsp = 0x00007fff545bafe0
      rip = 0x0000000119fa3595
      Found by: previous frame's frame pointer
  18  AppKit + 0x28b7e
      rbp = 0x00007fff545bb390   rsp = 0x00007fff545bb180
      rip = 0x00007fffb5d4fb7e
      Found by: previous frame's frame pointer
  19  CoreFoundation + 0x8da52
      rbp = 0x00007fff545bb3c0   rsp = 0x00007fff545bb3a0
      rip = 0x00007fffb827aa52
      Found by: previous frame's frame pointer
  20  CoreFoundation + 0x8d8e6
      rbp = 0x00007fff545bb440   rsp = 0x00007fff545bb3d0
      rip = 0x00007fffb827a8e6
      Found by: previous frame's frame pointer
  21  AppKit + 0x8c32a7
      rbp = 0x00007fff545bb4a0   rsp = 0x00007fff545bb450
      rip = 0x00007fffb65ea2a7
      Found by: previous frame's frame pointer
  22  AppKit + 0x28ae6
      rbp = 0x00007fff545bb6c0   rsp = 0x00007fff545bb4b0
      rip = 0x00007fffb5d4fae6
      Found by: previous frame's frame pointer
  23  AppKit + 0x25161
      rbp = 0x00007fff545bb7a0   rsp = 0x00007fff545bb6d0
      rip = 0x00007fffb5d4c161
      Found by: previous frame's frame pointer
  24  Chromium Framework!-[TabContentsController ensureContentsVisibleInSuperview:] + 0x244
      rbp = 0x00007fff545bb840   rsp = 0x00007fff545bb7b0
      rip = 0x000000011f5b88d4
      Found by: previous frame's frame pointer
  25  Chromium Framework!-[TabStripController swapInTabAtIndex:] + 0x1af
      rbp = 0x00007fff545bb9a0   rsp = 0x00007fff545bb850
      rip = 0x000000011f5c079f
      Found by: previous frame's frame pointer
  26  Chromium Framework!-[TabStripController activateTabWithContents:previousContents:atIndex:reason:] + 0x157
      rbp = 0x00007fff545bba20   rsp = 0x00007fff545bb9b0
      rip = 0x000000011f5c4147
      Found by: previous frame's frame pointer
  27  Chromium Framework!TabStripModel::NotifyIfActiveTabChanged(content::WebContents*, TabStripModel::Notify) + 0x15f
      rbp = 0x00007fff545bbbd0   rsp = 0x00007fff545bba30
      rip = 0x000000011f3b46cf
      Found by: previous frame's frame pointer
  28  Chromium Framework!TabStripModel::NotifyIfActiveOrSelectionChanged(content::WebContents*, TabStripModel::Notify, ui::ListSelectionModel const&) + 0x1f
      rbp = 0x00007fff545bbd80   rsp = 0x00007fff545bbbe0
      rip = 0x000000011f3b892f
      Found by: previous frame's frame pointer
  29  Chromium Framework!TabStripModel::SetSelection(ui::ListSelectionModel, TabStripModel::Notify) + 0x96
      rbp = 0x00007fff545bbde0   rsp = 0x00007fff545bbd90
      rip = 0x000000011f3b3e76
      Found by: previous frame's frame pointer
  30  Chromium Framework!TabStripModel::ActivateTabAt(int, bool) + 0xaf
      rbp = 0x00007fff545bbf60   rsp = 0x00007fff545bbdf0
      rip = 0x000000011f3b47df
      Found by: previous frame's frame pointer
  31  Chromium Framework!non-virtual thunk to Browser::ActivateContents(content::WebContents*) + 0x25
      rbp = 0x00007fff545bbf80   rsp = 0x00007fff545bbf70
      rip = 0x000000011f364375
      Found by: previous frame's frame pointer
  32  Chromium Framework!content::RenderFrameDevToolsAgentHost::Activate() + 0x21
      rbp = 0x00007fff545bbf90   rsp = 0x00007fff545bbf90
      rip = 0x0000000119af9561
      Found by: previous frame's frame pointer
  33  Chromium Framework!content::DevToolsHttpHandler::OnJsonRequest(int, net::HttpServerRequestInfo const&) + 0x10ab
      rbp = 0x00007fff545bc280   rsp = 0x00007fff545bbfa0
      rip = 0x0000000119a8de8b
      Found by: previous frame's frame pointer
  34  Chromium Framework!base::internal::Invoker<base::internal::BindState<void (content::DevToolsHttpHandler::*)(int, net::HttpServerRequestInfo const&), base::WeakPtr<content::DevToolsHttpHandler>, int, net::HttpServerRequestInfo>, void ()>::RunOnce(base::internal::BindStateBase*) + 0xc4
      rbp = 0x00007fff545bc3e0   rsp = 0x00007fff545bc290
      rip = 0x0000000119a919d4
      Found by: previous frame's frame pointer
  35  Chromium Framework!base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) + 0x105
      rbp = 0x00007fff545bc5c0   rsp = 0x00007fff545bc3f0
      rip = 0x000000011bd1c455
      Found by: previous frame's frame pointer
  36  Chromium Framework!base::internal::IncomingTaskQueue::RunTask(base::PendingTask*) + 0x79
      rbp = 0x00007fff545bc710   rsp = 0x00007fff545bc5d0
      rip = 0x000000011bd574b9
      Found by: previous frame's frame pointer
  37  Chromium Framework!base::MessageLoop::RunTask(base::PendingTask*) + 0x26a
      rbp = 0x00007fff545bc900   rsp = 0x00007fff545bc720
      rip = 0x000000011bd5be1a
      Found by: previous frame's frame pointer
  38  Chromium Framework!base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) + 0xc3
      rbp = 0x00007fff545bcac0   rsp = 0x00007fff545bc910
      rip = 0x000000011bd5c203
      Found by: previous frame's frame pointer
  39  Chromium Framework!base::MessageLoop::DoWork() + 0x266
      rbp = 0x00007fff545bcd50   rsp = 0x00007fff545bcad0
      rip = 0x000000011bd5c4a6
      Found by: previous frame's frame pointer
  40  Chromium Framework!base::MessagePumpCFRunLoopBase::RunWork() + 0x2a
      rbp = 0x00007fff545bcd80   rsp = 0x00007fff545bcd60
      rip = 0x000000011bd5fc9a
      Found by: previous frame's frame pointer
  41  Chromium Framework!base::mac::CallWithEHFrame(void () block_pointer) + 0xa
      rbp = 0x00007fff545bcd90   rsp = 0x00007fff545bcd90
      rip = 0x000000011bd4350a
      Found by: previous frame's frame pointer
  42  Chromium Framework!base::MessagePumpCFRunLoopBase::RunWorkSource(void*) + 0x3f
      rbp = 0x00007fff545bcdd0   rsp = 0x00007fff545bcda0
      rip = 0x000000011bd5f57f
      Found by: previous frame's frame pointer
  43  CoreFoundation + 0xa7321
      rbp = 0x00007fff545bcde0   rsp = 0x00007fff545bcde0
      rip = 0x00007fffb8294321
      Found by: previous frame's frame pointer
  44  CoreFoundation + 0x8821d
      rbp = 0x00007fff545bce40   rsp = 0x00007fff545bcdf0
      rip = 0x00007fffb827521d
      Found by: previous frame's frame pointer
  45  CoreFoundation + 0x87716
      rbp = 0x00007fff545bdb30   rsp = 0x00007fff545bce50
      rip = 0x00007fffb8274716
      Found by: previous frame's frame pointer
  46  CoreFoundation + 0x87114
      rbp = 0x00007fff545bdbc0   rsp = 0x00007fff545bdb40
      rip = 0x00007fffb8274114
      Found by: previous frame's frame pointer
  47  HIToolbox + 0x30ebc
      rbp = 0x00007fff545bdc00   rsp = 0x00007fff545bdbd0
      rip = 0x00007fffb77d4ebc
      Found by: previous frame's frame pointer
  48  HIToolbox + 0x30cf1
      rbp = 0x00007fff545bdc80   rsp = 0x00007fff545bdc10
      rip = 0x00007fffb77d4cf1
      Found by: previous frame's frame pointer
  49  HIToolbox + 0x30b26
      rbp = 0x00007fff545bdca0   rsp = 0x00007fff545bdc90
      rip = 0x00007fffb77d4b26
      Found by: previous frame's frame pointer
  50  AppKit + 0x46a54
      rbp = 0x00007fff545be0c0   rsp = 0x00007fff545bdcb0
      rip = 0x00007fffb5d6da54
      Found by: previous frame's frame pointer
  51  AppKit + 0x7c27ee
      rbp = 0x00007fff545be370   rsp = 0x00007fff545be0d0
      rip = 0x00007fffb64e97ee
      Found by: previous frame's frame pointer
  52  Chromium Framework!__71-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:]_block_invoke + 0x40
      rbp = 0x00007fff545be3a0   rsp = 0x00007fff545be380
      rip = 0x000000011b7f6a40
      Found by: previous frame's frame pointer
  53  Chromium Framework!base::mac::CallWithEHFrame(void () block_pointer) + 0xa
      rbp = 0x00007fff545be3b0   rsp = 0x00007fff545be3b0
      rip = 0x000000011bd4350a
      Found by: previous frame's frame pointer
  54  Chromium Framework!-[BrowserCrApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 0xa4
      rbp = 0x00007fff545be450   rsp = 0x00007fff545be3c0
      rip = 0x000000011b7f6984
      Found by: previous frame's frame pointer
  55  AppKit + 0x3b3db
      rbp = 0x00007fff545be540   rsp = 0x00007fff545be460
      rip = 0x00007fffb5d623db
      Found by: previous frame's frame pointer
  56  Chromium Framework!base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) + 0x16c
      rbp = 0x00007fff545be6c0   rsp = 0x00007fff545be550
      rip = 0x000000011bd60d6c
      Found by: previous frame's frame pointer
  57  Chromium Framework!base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) + 0x6e
      rbp = 0x00007fff545be6f0   rsp = 0x00007fff545be6d0
      rip = 0x000000011bd5efce
      Found by: previous frame's frame pointer
  58  Chromium Framework!base::MessageLoop::Run(bool) + 0xa9
      rbp = 0x00007fff545be850   rsp = 0x00007fff545be700
      rip = 0x000000011bd5b709
      Found by: previous frame's frame pointer
  59  Chromium Framework!base::RunLoop::Run() + 0xf9
      rbp = 0x00007fff545be9a0   rsp = 0x00007fff545be860
      rip = 0x000000011bd9a419
      Found by: previous frame's frame pointer
  60  Chromium Framework!ChromeBrowserMainParts::MainMessageLoopRun(int*) + 0x138
      rbp = 0x00007fff545beb10   rsp = 0x00007fff545be9b0
      rip = 0x000000011b7fd908
      Found by: previous frame's frame pointer
  61  Chromium Framework!content::BrowserMainLoop::RunMainMessageLoopParts() + 0x34
      rbp = 0x00007fff545beb40   rsp = 0x00007fff545beb20
      rip = 0x0000000119a21044
      Found by: previous frame's frame pointer
  62  Chromium Framework!content::BrowserMainRunnerImpl::Run() + 0x66
      rbp = 0x00007fff545bec90   rsp = 0x00007fff545beb50
      rip = 0x0000000119a240a6
      Found by: previous frame's frame pointer
  63  Chromium Framework!content::BrowserMain(content::MainFunctionParams const&) + 0xea
      rbp = 0x00007fff545bece0   rsp = 0x00007fff545beca0
      rip = 0x0000000119a1d41a
      Found by: previous frame's frame pointer
  64  Chromium Framework!content::ContentMainRunnerImpl::Run() + 0x235
      rbp = 0x00007fff545bee60   rsp = 0x00007fff545becf0
      rip = 0x000000011b78ce95
      Found by: previous frame's frame pointer
  65  Chromium Framework!service_manager::Main(service_manager::MainParams const&) + 0x9b2
      rbp = 0x00007fff545bf3d0   rsp = 0x00007fff545bee70
      rip = 0x000000011db90612
      Found by: previous frame's frame pointer
  66  Chromium Framework!content::ContentMain(content::ContentMainParams const&) + 0x44
      rbp = 0x00007fff545bf460   rsp = 0x00007fff545bf3e0
      rip = 0x000000011b78c224
      Found by: previous frame's frame pointer
  67  Chromium Framework!ChromeMain + 0xba
      rbp = 0x00007fff545bf550   rsp = 0x00007fff545bf470
      rip = 0x0000000118c1912a
      Found by: previous frame's frame pointer
  68  Chromium!main + 0x194
      rbp = 0x00007fff545bf590   rsp = 0x00007fff545bf560
      rip = 0x000000010b640de4
      Found by: previous frame's frame pointer
  69  libdyld.dylib + 0x5235
      rbp = 0x00007fff545bf5a0   rsp = 0x00007fff545bf5a0
      rip = 0x00007fffcda23235
      Found by: previous frame's frame pointer
  

Comment 9 by kbr@chromium.org, Feb 9 2018

Owner: kbr@chromium.org
Status: Assigned (was: Untriaged)
Ah, yes, thank you sadrul@. Look at this run:

https://ci.chromium.org/buildbot/tryserver.chromium.mac/mac_chromium_rel_ng/647077

and shard:

https://chromium-swarm.appspot.com/task?id=3b9090eec2ea0610&refresh=10&show_raw=1

Here is the actual problem:

[10194:775:0208/152820.637349:FATAL:gl_renderer.cc(3509)] Check failed: gl_->CheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE. 
0   Chromium Framework                  0x000000011bd1badc base::debug::StackTrace::StackTrace(unsigned long) + 28
1   Chromium Framework                  0x000000011bd40b10 logging::LogMessage::~LogMessage() + 224
2   Chromium Framework                  0x000000011d9f7775 viz::GLRenderer::CopyRenderPassDrawQuadToOverlayResource(viz::CALayerOverlay const*, std::__1::unique_ptr<viz::GLRenderer::OverlayTexture, std::__1::default_delete<viz::GLRenderer::OverlayTexture> >*, gfx::RectF*) + 3093
3   Chromium Framework                  0x000000011d9f64d0 viz::GLRenderer::ScheduleRenderPassDrawQuad(viz::CALayerOverlay const*) + 160
4   Chromium Framework                  0x000000011d9f3126 viz::GLRenderer::ScheduleCALayers() + 230
5   Chromium Framework                  0x000000011d9f26ee viz::GLRenderer::FinishDrawingFrame() + 542
6   Chromium Framework                  0x000000011d9d6a29 viz::DirectRenderer::DrawFrame(std::__1::vector<std::__1::unique_ptr<viz::RenderPass, std::__1::default_delete<viz::RenderPass> >, std::__1::allocator<std::__1::unique_ptr<viz::RenderPass, std::__1::default_delete<viz::RenderPass> > > >*, float, gfx::Size const&) + 3705
7   Chromium Framework                  0x000000011d9dc940 viz::Display::DrawAndSwap() + 1728
8   Chromium Framework                  0x000000011d9e06dc viz::DisplayScheduler::DrawAndSwap() + 236
9   Chromium Framework                  0x000000011d9dfd3f viz::DisplayScheduler::AttemptDrawAndSwap() + 223
10  Chromium Framework                  0x000000011d9df608 viz::DisplayScheduler::OnBeginFrameDeadline() + 136
11  Chromium Framework                  0x000000011d9e21a7 base::internal::Invoker<base::internal::BindState<void (viz::DisplayScheduler::*)(), base::WeakPtr<viz::DisplayScheduler> >, void ()>::Run(base::internal::BindStateBase*) + 183
12  Chromium Framework                  0x000000011913e69f void base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> >::ForwardRepeating<>() + 95
13  Chromium Framework                  0x000000011913e807 base::internal::Invoker<base::internal::BindState<void (base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> >::*)(), base::WeakPtr<base::internal::CancelableCallbackImpl<base::RepeatingCallback<void ()> > > >, void ()>::Run(base::internal::BindStateBase*) + 183
14  Chromium Framework                  0x000000011d7242e8 ui::WindowResizeHelperMac::WaitForSingleTaskToRun(base::TimeDelta const&) + 584
15  Chromium Framework                  0x0000000119e4880a content::RenderWidgetHostImpl::PauseForPendingResizeOrRepaints() + 842
16  Chromium Framework                  0x0000000119e61645 content::RenderWidgetHostViewMac::Show() + 309
17  Chromium Framework                  0x0000000119fa3595 content::WebContentsImpl::WasShown() + 69
18  AppKit                              0x00007fffb5d4fb7e -[NSView _setWindow:] + 3117

Looking for a recent CL to revert.

Comment 10 by kbr@chromium.org, Feb 9 2018

Cc: yiyix@chromium.org
Possible CLs:

commit e544a1ed8f62f4416060033ee0ff2ca0050af896
Author: yiyix <yiyix@chromium.org>
Date:   Tue Feb 6 23:45:34 2018 +0000

    Using CompositorFrameSinkSupport to handle the copy request
    
    In the current implementation, the copy request is handled by
    DelegatedFrameHost::RequestCopyOfOutput, which add copy request to
    the compositor frame of the browser. After this patch, the copy
    request will be handled by the
    CompositorFrameSinkSupport:RequestCopyOfSurface which adds the
    copy request to the surface where the copy request needs to be make.
    
    TEST: exiting test cases on copy requests should cover all the usage
    
    Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.android:android_optional_gpu_tests_rel
    Change-Id: I5c0b4572a7b8ed5fce0f5ee10f5d4dd982912e0a
    Reviewed-on: https://chromium-review.googlesource.com/881548
    Reviewed-by: Yuri Wiitala <miu@chromium.org>
    Reviewed-by: ccameron <ccameron@chromium.org>
    Commit-Queue: Yi Xu <yiyix@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#534838}

content/browser/renderer_host/browser_compositor_view_mac.h
content/browser/renderer_host/browser_compositor_view_mac.mm
content/browser/renderer_host/delegated_frame_host.cc
content/browser/renderer_host/delegated_frame_host.h
content/browser/renderer_host/render_widget_host_view_mac.mm


commit 483530c1094f0ad6fe36663262b484be300f472e
Author: Yuri Wiitala <miu@chromium.org>
Date:   Thu Feb 8 07:17:34 2018 +0000

    Layer CopyOutputRequests execute in VIZ service.
    
    Implements the final missing pieces allowing CopyOutputRequests made on
    Layers to propagate and execute in the VIZ display compositor:
    
    1. Added missing mojo fields and struct traits for viz::CopyOutputRequest
    and viz::RenderPass.
    
    2. Added extra transform code to cc::EffectTree::TakeCRsAndTransform()
    to account for the scaling ratio, not just the source area.
    
    3. Updated DelegatedFrameHost::CopyFromCompositingSurface() to use the
    new scaling APIs, enabling testing and otherwise proving that all "the
    plumbing" is in-place.
    
    4. Updated RenderWidgetHostViewBrowserTests so that they now work with
    --enabled-features=VizDisplayCompositor. (And, disabled dead tests I did
    not feel we should bother updating; to be removed in a later change.)
    
    5. Added privilege checks to make sure CopyOutputRequests cannot be made
    from unprivileged clients. Added new RWHostImpl browser test to confirm.
    
    Testing (all with and without --enable-features=VizDisplayCompositor):
    
    1. All RenderWidgetHostViewBrowserTest content_browsertests pass.
    
    2. On CrOS Ash desktop, tested CTRL-F5 (full screenshot) and
    SHIFT-CTRL-F5 (partial screenshot).
    
    3. On Linux, confirmed chrome.captureVisibleTab() screenshot API works
    via the common/extensions/docs/examples/api/tabs/screenshot demo
    extension.
    
    Bug:  806375 ,  806366 ,  759310 ,  771354 
    Cq-Include-Trybots: master.tryserver.blink:linux_trusty_blink_rel;master.tryserver.chromium.android:android_optional_gpu_tests_rel
    Change-Id: I98ad6cd90f332318dbdea9449e5d9c3a31679114
    Reviewed-on: https://chromium-review.googlesource.com/890339
    Commit-Queue: Yuri Wiitala <miu@chromium.org>
    Reviewed-by: Charlie Reis <creis@chromium.org>
    Reviewed-by: Daniel Cheng <dcheng@chromium.org>
    Reviewed-by: Fady Samuel <fsamuel@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#535325}

cc/trees/property_tree.cc
components/viz/common/quads/compositor_frame.cc
components/viz/common/quads/compositor_frame.h
components/viz/common/quads/render_pass.h
components/viz/service/display/surface_aggregator_unittest.cc
components/viz/service/frame_sinks/compositor_frame_sink_impl.cc
components/viz/service/frame_sinks/compositor_frame_sink_support.cc
components/viz/service/frame_sinks/compositor_frame_sink_support.h
components/viz/service/frame_sinks/compositor_frame_sink_support_unittest.cc
components/viz/service/frame_sinks/root_compositor_frame_sink_impl.cc
components/viz/test/test_layer_tree_frame_sink.cc
content/browser/bad_message.h
content/browser/renderer_host/delegated_frame_host.cc
content/browser/renderer_host/render_widget_host_browsertest.cc
content/browser/renderer_host/render_widget_host_impl.cc
content/browser/renderer_host/render_widget_host_view_browsertest.cc
content/test/BUILD.gn
services/viz/public/cpp/compositing/copy_output_request_struct_traits.cc
services/viz/public/cpp/compositing/copy_output_request_struct_traits.h
services/viz/public/cpp/compositing/render_pass_struct_traits.cc
services/viz/public/cpp/compositing/render_pass_struct_traits.h
services/viz/public/cpp/compositing/struct_traits_unittest.cc
services/viz/public/interfaces/compositing/copy_output_request.mojom
services/viz/public/interfaces/compositing/render_pass.mojom
testing/buildbot/filters/viz.content_browsertests.filter
tools/metrics/histograms/enums.xml

Comment 11 by kbr@chromium.org, Feb 9 2018

Note: here's the full output of:

git log --since 2018-02-06 --name-only > log-since-2018-02-06.txt 

log-since-2018-02-06.txt
1.4 MB View Download
Cc: erikc...@chromium.org
Normally we

DCHECK(gl_->CheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE || IsContextLost());

This DCHECK is

DCHECK(gl_->CheckFramebufferStatus(GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE);

For some reason. It was added by https://chromium.googlesource.com/chromium/src/+/d9a09cd1eb5cb9bab507ec2841726c02b316260d

Maybe it's just unluckily losing context before that DCHECK now?

Comment 13 by kbr@chromium.org, Feb 9 2018

Hmm. Interesting, yes, maybe that's what's happening.

Comment 14 by kbr@chromium.org, Feb 9 2018

Blockedon: 581526
Project Member

Comment 15 by bugdroid1@chromium.org, Feb 9 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7bd31af1a43cadbb406495d1d9a30d36e8837972

commit 7bd31af1a43cadbb406495d1d9a30d36e8837972
Author: Kenneth Russell <kbr@chromium.org>
Date: Fri Feb 09 21:22:33 2018

Relax DCHECK in GLRenderer::CopyRenderPassDrawQuadToOverlayResource.

It should also be allowing the context to be lost.

Unclear why the timing of this changed recently to cause this DCHECK
to be hit.

NOTRY=true

Bug:  810642 
Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel
Change-Id: Ida4804f7a2eba4ee71971ccecf14276731548b7b
Reviewed-on: https://chromium-review.googlesource.com/911819
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: danakj <danakj@chromium.org>
Cr-Commit-Position: refs/heads/master@{#535832}
[modify] https://crrev.com/7bd31af1a43cadbb406495d1d9a30d36e8837972/components/viz/service/display/gl_renderer.cc

Comment 16 by treib@chromium.org, Feb 12 2018

Labels: -Sheriff-Chromium

Comment 17 by kbr@chromium.org, Feb 16 2018

Status: Verified (was: Assigned)
chromium-try-flakes hasn't reported another flake since the above CL landed.

Sign in to add a comment