New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 613229 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocking:
issue 608923



Sign in to add a comment

context_lost flaky on initialization

Reported by dyen@chromium.org, May 19 2016

Issue description

We are getting some crashes upon initialization recently (last 2 days or so). Here is the backtrace (Although it looks wrong):

https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/224347
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/223414
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/223446

        (No symbol) [0x0F000000]
	(No symbol) [0x603D9717]
	gpu::gles2::FeatureInfo::InitializeFeatures [0x6401DEE0+5360]
	RtlInitUnicodeString [0x77BBE38C+356]
	RtlAllocateHeap [0x77BBE0F2+172]
	free [0x61526174+20]
	base::TimeTicks::Now [0x614DBF5F+255]

piman@, it looks like you are the only one that has changed anything related to initialization recently, could you take a look?
 

Comment 1 by dyen@chromium.org, May 19 2016

I ran ToT locally and was able to get a full stack:

Backtrace:
	(No symbol) [0x00007454]
	(No symbol) [0x67959697]
	gpu::gles2::FeatureInfo::InitializeFeatures [0x56CF3370+5360] (d:\base\chromium\src\gpu\command_buffer\service\feature_info.cc:736)
	gpu::gles2::FeatureInfo::Initialize [0x56CF1D80+48] (d:\base\chromium\src\gpu\command_buffer\service\feature_info.cc:199)
	gpu::gles2::ContextGroup::Initialize [0x56CF5500+256] (d:\base\chromium\src\gpu\command_buffer\service\context_group.cc:128)
	gpu::gles2::GLES2DecoderImpl::Initialize [0x56CE78C7+951] (d:\base\chromium\src\gpu\command_buffer\service\gles2_cmd_decoder.cc:2796)
	gpu::GpuCommandBufferStub::Initialize [0x56285FB1+2481] (d:\base\chromium\src\gpu\ipc\service\gpu_command_buffer_stub.cc:596)
	gpu::GpuCommandBufferStub::Create [0x56284F47+71] (d:\base\chromium\src\gpu\ipc\service\gpu_command_buffer_stub.cc:177)
	gpu::GpuChannel::CreateCommandBuffer [0x5627E6C5+693] (d:\base\chromium\src\gpu\ipc\service\gpu_channel.cc:972)
	gpu::GpuChannel::OnCreateCommandBuffer [0x5627FDF7+279] (d:\base\chromium\src\gpu\ipc\service\gpu_channel.cc:906)
	IPC::MessageT<GpuChannelMsg_CreateCommandBuffer_Meta,std::tuple<GPUCreateCommandBufferConfig,int,base::SharedMemoryHandle>,std::tuple<bool,gpu::Capabilities> >::Dispatch<gpu::GpuChannel,gpu::GpuChannel,void,void (__thiscall gpu::GpuChannel::*)(GPUCreateCo [0x5627CCF9+377] (d:\base\chromium\src\ipc\ipc_message_templates.h:174)
	gpu::GpuChannel::OnControlMessageReceived [0x5627FCC1+305] (d:\base\chromium\src\gpu\ipc\service\gpu_channel.cc:744)
	gpu::GpuChannel::HandleMessageHelper [0x5627F74D+109] (d:\base\chromium\src\gpu\ipc\service\gpu_channel.cc:811)
	gpu::GpuChannel::HandleMessage [0x5627F617+359] (d:\base\chromium\src\gpu\ipc\service\gpu_channel.cc:795)
	base::internal::Invoker<base::IndexSequence<0,1>,base::internal::BindState<base::internal::RunnableAdapter<void (__thiscall gpu::GpuChannel::*)(scoped_refptr<gpu::GpuChannelMessageQueue> const &)>,void __cdecl(gpu::GpuChannel *,scoped_refptr<gpu::GpuChann [0x56281C7E+78] (d:\base\chromium\src\base\bind_internal.h:362)
	base::debug::TaskAnnotator::RunTask [0x5430C8D7+247] (d:\base\chromium\src\base\debug\task_annotator.cc:51)
	base::MessageLoop::RunTask [0x542D94FB+1211] (d:\base\chromium\src\base\message_loop\message_loop.cc:479)
	base::MessageLoop::DoWork [0x542D8855+549] (d:\base\chromium\src\base\message_loop\message_loop.cc:605)
	base::MessagePumpForUI::DoRunLoop [0x5430D99A+90] (d:\base\chromium\src\base\message_loop\message_pump_win.cc:174)
	base::MessagePumpWin::Run [0x5430E1AA+74] (d:\base\chromium\src\base\message_loop\message_pump_win.cc:56)
	base::MessageLoop::RunHandler [0x542D9037+103] (d:\base\chromium\src\base\message_loop\message_loop.cc:442)
	base::RunLoop::Run [0x5430E969+41] (d:\base\chromium\src\base\run_loop.cc:36)
	base::MessageLoop::Run [0x542D8FB2+98] (d:\base\chromium\src\base\message_loop\message_loop.cc:295)
	content::GpuMain [0x55FA9ABE+2686] (d:\base\chromium\src\content\gpu\gpu_main.cc:414)
	content::RunNamedProcessTypeMain [0x542B0604+260] (d:\base\chromium\src\content\app\content_main_runner.cc:420)
	content::ContentMainRunnerImpl::Run [0x542B04B1+321] (d:\base\chromium\src\content\app\content_main_runner.cc:787)
	content::ContentMain [0x542AD5E3+35] (d:\base\chromium\src\content\app\content_main.cc:20)
	ChromeMain [0x5420CFE6+118] (d:\base\chromium\src\chrome\app\chrome_main.cc:87)
	MainDllLoader::Launch [0x00E982FC+812] (d:\base\chromium\src\chrome\app\main_dll_loader_win.cc:186)
	wWinMain [0x00E97667+567] (d:\base\chromium\src\chrome\app\chrome_exe_main_win.cc:264)
	__scrt_common_main_seh [0x00F0A266+253] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:255)
	BaseThreadInitThunk [0x771E338A+18]
	RtlInitializeExceptionChain [0x77CB9902+99]
	RtlInitializeExceptionChain [0x77CB98D5+54]

It looks like the feature info part was correct so it seems it is related to your last initialization refactor.

Comment 2 by piman@chromium.org, May 19 2016

Thanks for the stack trace, I'll take a look.

Comment 3 by piman@chromium.org, May 19 2016

It would mean it crashes here: https://code.google.com/p/chromium/codesearch#chromium/src/gpu/command_buffer/service/feature_info.cc&q=feature_info.cc:736&sq=package:chromium&l=736 ??

   glGenTextures(1, &tex_id);
   glGenFramebuffersEXT(1, &fb_id);
   glBindTexture(GL_TEXTURE_2D, tex_id);
   // Nearest filter needed for framebuffer completeness on some drivers.
-> glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F, width, width, 0, GL_RGBA,
                GL_FLOAT, NULL);

Comment 4 by dyen@chromium.org, May 19 2016

Sounds like it is related to this then:

https://bugs.chromium.org/p/chromium/issues/detail?id=612866

Comment 5 by dyen@chromium.org, May 19 2016

Cc: dyen@chromium.org kbr@chromium.org vmi...@chromium.org reve...@chromium.org
 Issue 612866  has been merged into this issue.

Comment 6 by dyen@chromium.org, May 19 2016

Labels: -Pri-3 Pri-1

Comment 7 by kbr@chromium.org, May 24 2016

Cc: capn@chromium.org
Components: Internals>GPU>Testing
Antoine, will you be able to investigate this? It's still showing up flaky on the commit queue per https://bugs.chromium.org/p/chromium/issues/detail?id=608923#c26 .

Comment 8 by piman@chromium.org, May 24 2016

I have made very little progress so far. The bug is still impossible to explain, most likely some memory corruption, either corrupting our driver function table (or the GLApi vtable, but that is less likely).

There are some likely related crashes on crash/. They only show up on Chrome OS, Android and Windows - but not Linux and Mac, which could indicate this is specific to our EGL use. This shows across different drivers/GPUs, so this is most likely an issue in our code.

I found and fixed several potential memory corruption issues over the last couple of days, but nothing that seems to have an impact on this yet.

Comment 9 Deleted

Comment 10 by kbr@chromium.org, May 26 2016

Another symptom (not sure whether this was captured elsewhere) seems to be a crash in DoBindTexture:

https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/228520/steps/context_lost_tests%20on%20NVIDIA%20GPU%20on%20Windows%20%28with%20patch%29%20on%20Windows-2008ServerR2-SP1/logs/stdio
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/228327/steps/context_lost_tests%20on%20ATI%20GPU%20on%20Windows%20%28with%20patch%29%20on%20Windows-2008ServerR2-SP1/logs/stdio


[3392:3764:0525/193653:ERROR:command_buffer_proxy_impl.cc(236)] Failed to send GpuChannelMsg_CreateCommandBuffer.
[3392:3764:0525/193653:ERROR:context_provider_command_buffer.cc(159)] GpuChannelHost failed to create command buffer.
[3228:4032:0525/193653:INFO:CONSOLE(0)] "WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost", source: http://127.0.0.1:53128/gpu_process_crash.html (0)
[3228:4032:0525/193653:ERROR:gpu_process_transport_factory.cc(754)] Lost UI shared context.
Backtrace:
	(No symbol) [0x54415453]
	(No symbol) [0x630533CB]
	gpu::gles2::GLES2DecoderImpl::DoBindTexture [0x64B25FFF+339]
	RtlFreeHeap [0x775EE023+126]
	HeapFree [0x759614AD+20]
	free [0x64172772+20]
	gpu::gles2::GLES2DecoderImpl::GenTexturesHelper [0x64B2F166+152]
	gpu::gles2::GLES2DecoderImpl::HandleBindTexture [0x64B31F91+89]
	gpu::gles2::GLES2DecoderImpl::DoCommandsImpl<0> [0x64B1E594+197]
	gpu::CommandParser::ProcessCommands [0x64B0D282+49]
	gpu::CommandExecutor::PutChanged [0x64B0DEFE+484]
	gpu::CommandBufferService::Flush [0x64B0D63A+30]
	gpu::GpuCommandBufferStub::OnAsyncFlush [0x64C15C9F+345]
	IPC::MessageT<GpuCommandBufferMsg_AsyncFlush_Meta,std::tuple<int,unsigned int,std::vector<ui::LatencyInfo,std::allocator<ui::LatencyInfo> > >,void>::Dispatch<gpu::GpuCommandBufferStub,gpu::GpuCommandBufferStub,void,void (__thiscall gpu::GpuCommandBufferSt [0x64C14007+142]
	gpu::GpuCommandBufferStub::OnMessageReceived [0x64C164DE+485]
	gpu::GpuChannel::HandleMessageHelper [0x64C0FFF2+44]
	gpu::GpuChannel::HandleMessage [0x64C0FF33+346]
	base::internal::InvokeHelper<1,void,base::internal::RunnableAdapter<void (__thiscall content::WebMediaPlayerMS::*)(scoped_refptr<media::VideoFrame> const &)> >::MakeItSo<base::WeakPtr<content::WebMediaPlayerMS>,scoped_refptr<media::VideoFrame> const &> [0x65806A5C+48]
	base::internal::Invoker<base::IndexSequence<0,1>,base::internal::BindState<base::internal::RunnableAdapter<void (__thiscall content::WebBluetoothServiceImpl::*)(mojo::Callback<void __cdecl(enum blink::mojom::WebBluetoothError)> const &)>,void __cdecl(cont [0x64C11D1C+45]
	base::debug::TaskAnnotator::RunTask [0x66DEB0F7+247]
	base::MessageLoop::RunTask [0x66D77CFB+1211]
	base::MessageLoop::DoWork [0x66D76DA5+549]
	base::MessagePumpForUI::DoRunLoop [0x66DBDFCA+90]
	base::MessagePumpWin::Run [0x66DBE8AA+74]
	base::MessageLoop::RunHandler [0x66D77837+103]
	base::RunLoop::Run [0x66DE3159+41]
	base::MessageLoop::Run [0x66D777C2+98]
	content::GpuMain [0x66A12131+1759]
	content::RunNamedProcessTypeMain [0x66D5019D+176]
	content::ContentMainRunnerImpl::Run [0x66D500BC+274]
	content::ContentMain [0x66D4F4D2+35]
	ChromeMain [0x6416A71C+108]
	MainDllLoader::Launch [0x011BEA73+488]
	wWinMain [0x011BD94D+450]
	__scrt_common_main_seh [0x01A3BF3B+253] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:255)
	BaseThreadInitThunk [0x7596336A+18]
	RtlInitializeExceptionChain [0x775F92B2+99]
	RtlInitializeExceptionChain [0x775F9285+54]

(full stdout attached)

Looks like this may be happening immediately after the GPU process relaunches after it was terminated via about:gpucrash.

stdout.txt
141 KB View Download

Comment 11 by piman@chromium.org, May 26 2016

Yes, we noticed that in  bug 612866 

That's why I suspect corruption of our driver function table, but I haven't been able to repro this yet when adding instrumentation to catch the condition.
Ping. Should this be Pri-1? If so please assign a milestone.

Comment 13 by kbr@chromium.org, Jun 21 2016

I'm not seeing this flake any more on https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng?numbuilds=200 and would suggest closing it as WontFix (not reproducible).

Comment 14 by piman@chromium.org, Jun 22 2016

Status: WontFix (was: Untriaged)
Thanks Ken. I think some of the unrelated fixes may have magically removed this.  ¯\_(ツ)_/¯

Sign in to add a comment