desktopui_MashLogin crashes in gbm_bo_destroy from gbm_wrapper::Buffer::~Buffer() |
|||||
Issue descriptionStarted earlier today, has happened several times on the ToT chromium waterfall. I'm suspicious of https://chromium-review.googlesource.com/c/chromium/src/+/1198451 "ozone/drm: Use minigbm mmap for modeset buffers" dcastagna, could that CL be the cause? +achuith as gardener Sample failure: Test: desktopui_MashLogin. Suite: chrome-informational. Chrome Version: 71.0.3546.0. Build: tricky-tot-chrome-pfq-informational/R71-11045.0.0-b2921341. Reason: Unhandled BrowserGoneException: WebsocketException of type <class 'websocket._exceptions.WebSocketConnectionClosedException'>. Error message: Connection is already closed.. build artifacts: https://storage.cloud.google.com/?arg=chromeos-image-archive/tricky-tot-chrome-pfq-informational/R71-11045.0.0-b2921341. results log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/235298813-chromeos-test/chromeos4-row2-rack3-host13/debug/. status log: http://ubercautotest.corp.google.com/tko/retrieve_logs.cgi?job=/results/235298813-chromeos-test/chromeos4-row2-rack3-host13/status.log. job link: http://cautotest-prod/afe/#tab_id=view_job&object_id=235298813. SIGSEGV Crash address: 0x0 Process uptime: not available Thread 0 (crashed) 0 libminigbm.so.1.0.0!gbm_bo_destroy [gbm.c : 153 + 0x0] rax = 0x000064e4444c1ac0 rdx = 0x0000000000000003 rcx = 0x0000000000000000 rbx = 0x0000000000000000 rsi = 0x0000228266f1ea90 rdi = 0x0000000000000000 rbp = 0x000071d6fbaade20 rsp = 0x000071d6fbaade10 r8 = 0x0000000000000000 r9 = 0x0000228268fdc790 r10 = 0x000022826911f301 r11 = 0x0000000000000000 r12 = 0x000064e4446245b8 r13 = 0x000022826814f300 r14 = 0x0000228269139300 r15 = 0x000022826911d200 rip = 0x000071d702318689 Found by: given as instruction pointer in context 1 chrome!gbm_wrapper::Buffer::~Buffer() [gbm_wrapper.cc : 121 + 0x5] rbx = 0x00002282690a9120 rbp = 0x000071d6fbaade50 rsp = 0x000071d6fbaade30 r12 = 0x000064e4446245b8 r13 = 0x000022826814f300 r14 = 0x0000228269139300 r15 = 0x000022826911d200 rip = 0x000064e43b58b341 Found by: call frame info 2 chrome!ui::GbmPixmap::~GbmPixmap() [memory : 2321 + 0x5] rbx = 0x00002282690a9120 rbp = 0x000071d6fbaade70 rsp = 0x000071d6fbaade60 r12 = 0x000064e4446245b8 r13 = 0x000022826814f300 r14 = 0x0000000000000000 r15 = 0x000022826911d200 rip = 0x000064e43b561269 Found by: call frame info 3 chrome!gl::GLImageNativePixmap::~GLImageNativePixmap() [ref_counted.h : 414 + 0x5] rbx = 0x0000228269026ac0 rbp = 0x000071d6fbaade90 rsp = 0x000071d6fbaade80 r12 = 0x000064e4446245b8 r13 = 0x000022826814f300 r14 = 0x0000228267f1b4a8 r15 = 0x000022826911d200 rip = 0x000064e43ea5d35c Found by: call frame info 4 chrome!std::__1::__vector_base<gpu::gles2::Texture::LevelInfo, std::__1::allocator<gpu::gles2::Texture::LevelInfo> >::~__vector_base() [ref_counted.h : 352 + 0x3] rbx = 0x000022826911d258 rbp = 0x000071d6fbaadec0 rsp = 0x000071d6fbaadea0 r12 = 0x000064e4446245b8 r13 = 0x000022826814f300 r14 = 0x0000228267f1b4a8 r15 = 0x000022826911d200 rip = 0x000064e43ed3e12a Found by: call frame info 5 chrome!gpu::gles2::Texture::~Texture() [texture_manager.cc : 652 + 0x9] rbx = 0x0000228267f1b4a0 rbp = 0x000071d6fbaadef0 rsp = 0x000071d6fbaaded0 r12 = 0x0000000000000000 r13 = 0x000022826814f300 r14 = 0x000022826912c8c0 r15 = 0x0000228267f1b4a0 rip = 0x000064e43ed3d6fd Found by: call frame info 6 chrome!gpu::gles2::Texture::RemoveTextureRef(gpu::gles2::TextureRef*, bool) [texture_manager.cc : 602 + 0x8] rbx = 0x0000228266f4a760 rbp = 0x000071d6fbaadf40 rsp = 0x000071d6fbaadf00 r12 = 0x0000000000000000 r13 = 0x000022826814f300 r14 = 0x0000000000000001 r15 = 0x000022826912c8c0 rip = 0x000064e43ed3dd6d Found by: call frame info 7 chrome!std::__1::__hash_table<std::__1::__hash_value_type<unsigned int, scoped_refptr<gpu::gles2::TextureRef> >, std::__1::__unordered_map_hasher<unsigned int, std::__1::__hash_value_type<unsigned int, scoped_refptr<gpu::gles2::TextureRef> >, base_hash::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, std::__1::__hash_value_type<unsigned int, scoped_refptr<gpu::gles2::TextureRef> >, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned int, scoped_refptr<gpu::gles2::TextureRef> > > >::erase(std::__1::__hash_const_iterator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned int, scoped_refptr<gpu::gles2::TextureRef> >, void*>*>) [texture_manager.cc : 1943 + 0xb] rbx = 0x000022826817fb00 rbp = 0x000071d6fbaadf60 rsp = 0x000071d6fbaadf50 r12 = 0x0000228268f05600 r13 = 0x000022826814f300 r14 = 0x0000228268f05600 r15 = 0x0000000000000001 rip = 0x000064e43ed4a5c6 Found by: call frame info 8 chrome!gpu::gles2::GLES2DecoderImpl::HandleDeleteTexturesImmediate(unsigned int, void const volatile*) [gles2_cmd_decoder.cc : 892 + 0x7] rbx = 0x000000000000000c rbp = 0x000071d6fbaadfb0 rsp = 0x000071d6fbaadf70 r12 = 0x0000228268f05600 r13 = 0x000022826814f300 r14 = 0x0000000000000000 r15 = 0x0000000000000001 rip = 0x000064e441da342b Found by: call frame info 9 chrome!gpu::gles2::GLES2DecoderImpl::DoCommands(unsigned int, void const volatile*, int, int*) [gles2_cmd_decoder.cc : 5654 + 0x3] rbx = 0x000071d6f30fb200 rbp = 0x000071d6fbaae1b0 rsp = 0x000071d6fbaadfc0 r12 = 0x000022826814f300 r13 = 0x0000000000000003 r14 = 0x0000000000000134 r15 = 0x0000000000000003 rip = 0x000064e441dde966 Found by: call frame info 10 chrome!gpu::CommandBufferService::Flush(int, gpu::AsyncAPIInterface*) [command_buffer_service.cc : 69 + 0xe] rbx = 0x000064e444677130 rbp = 0x000071d6fbaae260 rsp = 0x000071d6fbaae1c0 r12 = 0x0000000000000883 r13 = 0x0000228269017e70 r14 = 0x000022826814f300 r15 = 0x000071d6fbaae208 rip = 0x000064e443323bc4 Found by: call frame info 11 chrome!gpu::CommandBufferStub::OnAsyncFlush(int, unsigned int) [command_buffer_stub.cc : 621 + 0x8] rbx = 0x0000000000000000 rbp = 0x000071d6fbaae310 rsp = 0x000071d6fbaae270 r12 = 0x000064e444c708f0 r13 = 0x0000228268ceee00 r14 = 0x0000000000000883 r15 = 0x0000000000000001 rip = 0x000064e441e57319 Found by: call frame info 12 chrome!gpu::CommandBufferStub::OnMessageReceived(IPC::Message const&) [tuple.h : 52 + 0x8] rbx = 0x000064e444c708c0 rbp = 0x000071d6fbaae500 rsp = 0x000071d6fbaae320 r12 = 0x0000000000000009 r13 = 0x0000000000000001 r14 = 0x000022826819d1c0 r15 = 0x0000228268ceee00 rip = 0x000064e441e5473c Found by: call frame info 13 chrome!gpu::GpuChannel::HandleMessageHelper(IPC::Message const&) [message_router.cc : 56 + 0x8] rbx = 0x0000228268ceee00 rbp = 0x000071d6fbaae8c0 rsp = 0x000071d6fbaae510 r12 = 0x0000000000000009 r13 = 0x000022826819d1c0 r14 = 0x000022826872f280 r15 = 0x000022826819d180 rip = 0x000064e441e5ec2a Found by: call frame info 14 chrome!gpu::GpuChannel::HandleMessage(IPC::Message const&) [gpu_channel.cc : 509 + 0xb] rbx = 0x0000228268ceee00 rbp = 0x000071d6fbaae920 rsp = 0x000071d6fbaae8d0 r12 = 0x0000000000000009 r13 = 0x000022826872f280 r14 = 0x000022826819d1c0 r15 = 0x000022826819d180 rip = 0x000064e441e5c372 Found by: call frame info 15 chrome!gpu::Scheduler::RunNextTask() [callback.h : 99 + 0x7] rbx = 0x0000228267033408 rbp = 0x000071d6fbaaea00 rsp = 0x000071d6fbaae930 r12 = 0x0000000000000009 r13 = 0x0000228269003480 r14 = 0x0000000000000002 r15 = 0x000022826819d180 rip = 0x000064e441d89932 Found by: call frame info 16 chrome!base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) [callback.h : 99 + 0x6] rbx = 0x000022826917ab00 rbp = 0x000071d6fbaaeaf0 rsp = 0x000071d6fbaaea10 r12 = 0x0000228267004a58 r13 = 0x000071d6fbaaebc0 r14 = 0x0000000000000000 r15 = 0x000064e444c70840 rip = 0x000064e43d94d558 Found by: call frame info 17 chrome!base::MessageLoop::RunTask(base::PendingTask*) [message_loop.cc : 434 + 0xf] rbx = 0x0000000000000000 rbp = 0x000071d6fbaaeba0 rsp = 0x000071d6fbaaeb00 r12 = 0x000071d6fbaaebc0 r13 = 0x000071d6fbaaeca8 r14 = 0x0000228266ff9d80 r15 = 0x0000000000000000 rip = 0x000064e43d8ae837 Found by: call frame info
,
Sep 8
Actually, looking at crrev.com/c/1198451 again, I don't see how that can change bo_ to nullptr. Feel free to revert while we investigate further.
,
Sep 10
From 3f074a0649cc1ad4cab8745882a8dc4eb0d3b394 we started assuming we always import a bo when we import buffers from fds. Reassigning to spang@ as owner of that CL.
,
Sep 10
+CC newcomer/raymes chromeos gardeners. I think this is causing most of the desktopui_MashLogin crashes on the ToT chrome waterfall. I put a revert of the CL in #2 in the CQ: https://chromium-review.googlesource.com/c/chromium/src/+/1217022
,
Sep 10
I don't think I see any recent tast.ui.MashLogin failures on the Chrome OS -release builders due to this (but I don't think that there were any recent ones before the revert either). I can't comment on desktopui_MashLogin, though.
,
Sep 11
,
Sep 11
We're also seeing flaky failures in the PFQ in login_VMSanity due to code very similar to desktopui_MashLogin.
,
Sep 11
Getting another ui.MashLogin failure in the chrome PFQ betty-arcnext-chrome-pfq 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:31 Running ui.MashLogin 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:31 Restarting ui job 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:34 Waiting for org.chromium.SessionManager D-Bus service 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:34 Asking session_manager to enable Chrome testing 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:34 Waiting for Chrome to write its debugging port to /home/chronos/DevToolsActivePort 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:35 Removing cryptohome for testuser@gmail.com 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:35 Finding OOBE DevTools target 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:36 Connecting to Chrome at ws://127.0.0.1:35026/devtools/page/0F298426C055CFDADDB15E32127C4727 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:36 Waiting for OOBE 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:44 Logging in as user "testuser@gmail.com" 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:44 Waiting for cryptohome /home/user/d64376d09c0de35a3636283392f8c803f1a5067f 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:15:52 Waiting for OOBE to be dismissed 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:17:31 Error: [mash_login.go:37] Chrome login failed: OOBE not dismissed: browser process 21514 replaced by 21903 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:17:32 Finished ui.MashLogin 04:17:47 INFO | autoserv| AUTOTEST_STATUS:: 2018/09/11 06:17:32 --------------------------------------------------------------------------------
,
Sep 11
Also seen on betty-chrome-pfq
,
Sep 11
The chrome-pfq-informational builders (running ToT chrome) seem happier since the revert in #4 landed: https://cros-goldeneye.corp.google.com/chromeos/legoland/builderSummary?buildBranch=master&builderGroups=chrome_informational&limit=5&email=&buildConfig= However, we have betty-chrome-pfq failure in login_VMSanity: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8935703164647090336 chrome revision is 71.0.3549.0, r590119. Revert was r589959 so that chrome contains the revert. Either the revert didn't fix things, or the login_VMSanity is a different crash. Unfortunately I don't have a symbolized stack.
,
Sep 11
The failures mentioned in #6 to #10 are not related. I filed issue 882976 for those. So my best guess is that the revert fixed things, given that the ToT chrome builders went green.
,
Sep 17
Seems fixed! |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by dcasta...@chromium.org
, Sep 8