"about:crashes" crashes |
||||||||||
Issue descriptionVersion: 51.0.2688.0 (Official Build) canary (64-bit) OS: Mac OS X 10.11.4 What steps will reproduce the problem? (1) Visit about:crashes on Canary. What is the expected output? Expect to see the list of recent crashes. What do you see instead? The entire browser crashes with the following report to the Terminal: Google Chrome Canary(29953,0x7fff79b0b000) malloc: *** error for object 0x7fd8c2d46608: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug Abort trap: 6 This severely hampers crash reporting by customers, and even internal diagnoses.
,
Mar 23 2016
Did you get a crash report? :) Have a look in your Crashpad database directory, ~/Library/Application Support/Google/Chrome Canary/Crashpad. If you have crashpad_database_util, that’s the preferred way to interface with it. See https://crashpad.chromium.org/man/crashpad_database_util.html. You’d want to use --show-completed-reports (and perhaps --show-pending-reports if you don’t see a matching report). You can poke around without crashpad_database_util too. Find a crash in the completed (or pending) directory whose timestamp matches, and run xattr -l on it. If it has a com.googlecode.crashpad.id attribute, the report was uploaded and received that ID on the server.
,
Mar 23 2016
By the way, I gave you those instructions because I can’t reproduce on my own in canary 51.0.2688.0. Otherwise I would have just crashed it myself and posted my own crash ID here.
,
Mar 23 2016
Sounds like it could be issue 582413 .
,
Mar 23 2016
Works for me - 51.0.2688.0, 10.10.5 Will upgrade OSX and try again.
,
Mar 23 2016
kbr-macbookpro4:src kbr$ ls -lt ~/Library/Application\ Support/Google/Chrome\ Canary/Crashpad/completed/ total 55328 -rw-------@ 1 kbr eng 636208 Mar 23 13:46 9ef1d45a-9ec7-4cf6-9b05-95a164bfb999.dmp -rw-------@ 1 kbr eng 667584 Mar 23 13:46 238c88a6-ae76-4bd4-80a8-a8b6eb13763b.dmp -rw-------@ 1 kbr eng 578800 Mar 23 13:45 7598fb5d-c3bc-498a-8a5f-4f1d93562f62.dmp -rw-------@ 1 kbr eng 603472 Mar 23 13:44 53dee2a2-9950-4590-8325-162175e189a7.dmp -rw-------@ 1 kbr eng 666544 Mar 23 13:44 3b71b45c-74a7-49ed-a776-25bf93310c0b.dmp -rw-------@ 1 kbr eng 131440 Mar 23 13:44 ea995af0-e0b6-49d0-a69c-49a776e59835.dmp -rw-------@ 1 kbr eng 141184 Mar 23 13:38 dbbfe2f3-bbb9-48c2-854b-68370d1f07b5.dmp -rw-------@ 1 kbr eng 210992 Mar 23 13:37 77ca0210-30cf-43b9-9f2d-8ed73916a4e8.dmp -rw-------@ 1 kbr eng 161264 Mar 23 13:24 292aa577-34ff-4740-897f-5b2c11086d9a.dmp -rw-------@ 1 kbr eng 211264 Mar 23 13:24 348cd412-9152-4c48-8230-988546929923.dmp kbr-macbookpro4:src kbr$ xattr -l ~/Library/Application\ Support/Google/Chrome\ Canary/Crashpad/completed/9ef1d45a-9ec7-4cf6-9b05-95a164bfb999.dmp com.googlecode.crashpad.creation_time: 1458765997 com.googlecode.crashpad.uuid: 9ef1d45a-9ec7-4cf6-9b05-95a164bfb999 I'm guessing the uuid isn't the same as the id?
,
Mar 23 2016
BTW, nothing in the "pending" or "new" directories.
,
Mar 23 2016
That means that it didn't get uploaded. Probably it was rate-limited. If you send me a copy of that dmp file, I'll get it uploaded. Or I could tell you how to force the upload yourself. Really, I've been meaning to write a utility to make a dump eligible for upload again, or to just do the upload. But until then, there are a couple of commands we can use on OS X.
,
Mar 23 2016
Shouldn't the oldest crash have been uploaded, and not rate limited?
,
Mar 23 2016
Yeah, but I assume the oldest crash was for something else if kbr was trying to visit about:crashes.
,
Mar 23 2016
Here are the four most recent .dmp files from that directory.
,
Mar 23 2016
7598fb5d-c3bc-498a-8a5f-4f1d93562f62 = go/crash/94dbdd3800000000 238c88a6-ae76-4bd4-80a8-a8b6eb13763b = go/crash/d091dd3800000000 9ef1d45a-9ec7-4cf6-9b05-95a164bfb999 = go/crash/7979dd3800000000 1c42239b-fdd6-4403-8476-5b5dfc4f6150 = go/crash/fd9bdd3800000000
,
Mar 23 2016
The most recent report, go/crash/fd9bdd3800000000: Thread 0 CRASHED [EXC_BAD_INSTRUCTION / 0x00000001 @ 0x00007fff9bddd02f ] MAGIC SIGNATURE THREAD 0x00007fff9bddd02f (libobjc.A.dylib + 0x0001302f ) 0x00007fff9bddd17e (libobjc.A.dylib + 0x0001317e ) 0x00007fff9bde86bd (libobjc.A.dylib + 0x0001e6bd ) 0x00007fff9bdd1e62 (libobjc.A.dylib + 0x00007e62 ) 0x00007fff9bdd1de3 (libobjc.A.dylib + 0x00007de3 ) 0x00007fff9bdd1d5b (libobjc.A.dylib + 0x00007d5b ) 0x00007fff9bdd1cce (libobjc.A.dylib + 0x00007cce ) 0x0000000102ac5550 (Google Chrome Framework -objc_zombie.mm:158 ) (anonymous namespace)::ZombieDealloc(objc_object*, objc_selector*) 0x00007fff9649d5ab (AppKit + 0x000825ab ) 0x00007fff96423975 (AppKit + 0x00008975 ) 0x00007fff965ba93b (AppKit + 0x0019f93b ) 0x00007fff965b9591 (AppKit + 0x0019e591 ) 0x00007fff965ba65f (AppKit + 0x0019f65f ) 0x0000000103206418 (Google Chrome Framework -skia_utils_mac.mm:248 ) skia:: 0x000000010322098a (Google Chrome Framework -image_skia_util_mac.mm:110 ) gfx::NSImageFromImageSkiaWithColorSpace(gfx::ImageSkia const&, CGColorSpace*) 0x000000010321b71c (Google Chrome Framework -image.cc:566 ) gfx::Image::ToNSImage() const 0x00000001058e7355 (Google Chrome Framework -browser_action_button.mm:409 ) -[BrowserActionButton updateState] 0x0000000106731192 (Google Chrome Framework -toolbar_actions_model.cc:156 ) ToolbarActionsModel::OnExtensionActionUpdated(ExtensionAction*, content::WebContents*, content::BrowserContext*) 0x000000010660edc7 (Google Chrome Framework -extension_action_api.cc:207 ) extensions::ExtensionActionAPI::NotifyChange(ExtensionAction*, content::WebContents*, content::BrowserContext*) 0x000000010660f2a6 (Google Chrome Framework -extension_action_api.cc:263 ) extensions::ExtensionActionAPI::ClearAllValuesForTab(content::WebContents*) 0x0000000106719cd5 (Google Chrome Framework -tab_helper.cc:263 ) extensions::TabHelper::DidNavigateMainFrame(content::LoadCommittedDetails const&, content::FrameNavigateParams const&) 0x0000000105f1f608 (Google Chrome Framework -web_contents_impl.cc:3175 ) content::WebContentsImpl::DidNavigateMainFramePostCommit(content::RenderFrameHostImpl*, content::LoadCommittedDetails const&, FrameHostMsg_DidCommitProvisionalLoad_Params const&) 0x0000000105cf409e (Google Chrome Framework -navigator_impl.cc:595 ) content::NavigatorImpl::DidNavigate(content::RenderFrameHostImpl*, FrameHostMsg_DidCommitProvisionalLoad_Params const&) 0x0000000105cf99e0 (Google Chrome Framework -render_frame_host_impl.cc:1076 ) content::RenderFrameHostImpl::OnDidCommitProvisionalLoad(IPC::Message const&) 0x0000000105cf8b5a (Google Chrome Framework -render_frame_host_impl.cc:507 ) content::RenderFrameHostImpl::OnMessageReceived(IPC::Message const&) 0x0000000105e3b998 (Google Chrome Framework -render_process_host_impl.cc:1801 ) content::RenderProcessHostImpl::OnMessageReceived(IPC::Message const&) 0x0000000103403237 (Google Chrome Framework -ipc_channel_proxy.cc:293 ) IPC::ChannelProxy::Context::OnDispatchMessage(IPC::Message const&) 0x000000010296e3da (Google Chrome Framework -callback.h:397 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&) 0x0000000102990982 (Google Chrome Framework -message_loop.cc:476 ) base::MessageLoop::RunTask(base::PendingTask const&) 0x0000000102990c5b (Google Chrome Framework -message_loop.cc:485 ) base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&) 0x0000000102990e4a (Google Chrome Framework -message_loop.cc:597 ) base::MessageLoop::DoWork() 0x00000001029638b0 (Google Chrome Framework -message_pump_mac.mm:330 ) base::MessagePumpCFRunLoopBase::RunWork() 0x0000000102986159 (Google Chrome Framework + 0x00565159 ) base::mac::CallWithEHFrame(void () block_pointer) 0x00000001029632b3 (Google Chrome Framework -message_pump_mac.mm:306 ) base::MessagePumpCFRunLoopBase::RunWorkSource(void*) 0x00007fff9321b880 (CoreFoundation + 0x000aa880 ) 0x00007fff931fafbb (CoreFoundation + 0x00089fbb ) 0x00007fff931fa4de (CoreFoundation + 0x000894de ) 0x00007fff931f9ed7 (CoreFoundation + 0x00088ed7 ) 0x00007fff8bbff934 (HIToolbox + 0x00030934 ) 0x00007fff8bbff76e (HIToolbox + 0x0003076e ) 0x00007fff8bbff5ae (HIToolbox + 0x000305ae ) 0x00007fff96463ef9 (AppKit + 0x00048ef9 ) 0x00007fff96463329 (AppKit + 0x00048329 ) 0x00007fff96457e83 (AppKit + 0x0003ce83 ) 0x00000001029640c5 (Google Chrome Framework -message_pump_mac.mm:665 ) base::MessagePumpNSApplication::DoRun(base::MessagePump::Delegate*) 0x0000000102963703 (Google Chrome Framework -message_pump_mac.mm:238 ) base::MessagePumpCFRunLoopBase::Run(base::MessagePump::Delegate*) 0x00000001029a6b22 (Google Chrome Framework -run_loop.cc:35 ) base::RunLoop::Run() 0x00000001024b8537 (Google Chrome Framework -chrome_browser_main.cc:1845 ) ChromeBrowserMainParts::MainMessageLoopRun(int*) 0x0000000105c39a46 (Google Chrome Framework -browser_main_loop.cc:944 ) content::BrowserMainLoop::RunMainMessageLoopParts() 0x0000000105c3c011 (Google Chrome Framework -browser_main_runner.cc:150 ) content::BrowserMainRunnerImpl::Run() 0x0000000105c357bc (Google Chrome Framework -browser_main.cc:44 ) content::BrowserMain(content::MainFunctionParams const&) 0x0000000102925b33 (Google Chrome Framework -content_main_runner.cc:754 ) content::ContentMainRunnerImpl::Run() 0x00000001029250a5 (Google Chrome Framework -content_main.cc:19 ) content::ContentMain(content::ContentMainParams const&) 0x0000000102424191 (Google Chrome Framework -chrome_main.cc:84 ) ChromeMain 0x00000001021bad41 (Google Chrome Canary -chrome_exe_main_mac.c:87 ) main 0x00000001021bab23 (Google Chrome Canary + 0x00000b23 ) start Looks to be the same as bug 582413 . And it looks like we need to upload 10.11.4 symbols.
,
Mar 23 2016
We also recorded this annotation: List Annotations objc[30080]: Hash table corrupted. This is probably a memory error somewhere. (table at 0x7fff7a362b90, buckets at 0x7f9fed4e2000 (16384 bytes), 1024 buckets, 88 entries, 689 tombstones, data 0xa430a130d530ed30 0xbd03b5039303eb30 0x2000cc03ba03b903 0xc603bf03c103c003) There’s extension stuff on the stack. What extensions do you have installed? Were you doing anything extensiony at around the time of the crash?
,
Mar 23 2016
Thanks. Most look related to extensions: ----- Thread 0 CRASHED [0x00000000 / 0x00000000 @ 0x00007fff87657f06 ] MAGIC SIGNATURE THREAD 0x00007fff87657f06 (libsystem_kernel.dylib + 0x00016f06 ) 0x00007fff8f9a06e6 (libsystem_c.dylib + 0x0005e6e6 ) 0x00007fff9bafa395 (libsystem_malloc.dylib + 0x00010395 ) 0x00007fff9baefda5 (libsystem_malloc.dylib + 0x00005da5 ) 0x00007fff9baecb63 (libsystem_malloc.dylib + 0x00002b63 ) 0x00000001063cd97f (Google Chrome Framework -memory_mac.mm:134 ) base::(anonymous namespace)::oom_killer_malloc(_malloc_zone_t*, unsigned long) 0x00007fff9baec5a0 (libsystem_malloc.dylib + 0x000025a0 ) 0x00007fff9baeb0cb (libsystem_malloc.dylib + 0x000010cb ) 0x000000010ac7ca59 (Google Chrome Framework + 0x04e30a59 ) operator new(unsigned long) 0x00000001063fef92 (Google Chrome Framework -new:168 ) void std::__1::vector<base::Value*, std::__1::allocator<base::Value*> >::__push_back_slow_path<base::Value* const&>(base::Value* const&&&) 0x00000001063fe7da (Google Chrome Framework -vector:1594 ) base::ListValue::DeepCopy() const 0x00000001063fd2ef (Google Chrome Framework -values.cc:850 ) base::DictionaryValue::DeepCopy() const 0x00000001063fd2ef (Google Chrome Framework -values.cc:850 ) base::DictionaryValue::DeepCopy() const 0x00000001063fd2ef (Google Chrome Framework -values.cc:850 ) base::DictionaryValue::DeepCopy() const 0x000000010912244e (Google Chrome Framework -extension_messages.cc:59 ) ExtensionMsg_Loaded_Params::ExtensionMsg_Loaded_Params(extensions::Extension const*, bool) 0x0000000109f1f4ed (Google Chrome Framework -renderer_startup_helper.cc:76 ) extensions::RendererStartupHelper::Observe(int, content::NotificationSource const&, content::NotificationDetails const&) 0x0000000109800cf7 (Google Chrome Framework -notification_service_impl.cc:123 ) content::NotificationServiceImpl::Notify(int, content::NotificationSource const&, content::NotificationDetails const&) 0x000000010986957b (Google Chrome Framework -render_process_host_impl.cc:2601 ) content::RenderProcessHostImpl::OnProcessLaunched() 0x0000000109691e73 (Google Chrome Framework -child_process_launcher.cc:543 ) content::ChildProcessLauncher::Notify(std::nullptr_t, base::Process) 0x00000001096916f9 (Google Chrome Framework -child_process_launcher.cc:508 ) content::ChildProcessLauncher::DidLaunch(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process) 0x0000000109692157 (Google Chrome Framework -bind_internal.h:159 ) base::internal::Invoker<base::IndexSequence<0ul, 1ul>, base::internal::BindState<base::internal::RunnableAdapter<void (*)(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process)>, void (base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process), base::WeakPtr<content::ChildProcessLauncher>, bool const&>, base::internal::InvokeHelper<false, void, base::internal::RunnableAdapter<void (*)(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process)> >, void (std::nullptr_t, base::Process)>::Run(base::internal::BindStateBase*, std::nullptr_t&&, base::Process&&) 0x0000000109691ff7 (Google Chrome Framework -callback.h:397 ) base::internal::Invoker<base::IndexSequence<0ul, 1ul>, base::internal::BindState<base::Callback<void (std::nullptr_t, base::Process), (base::internal::CopyMode)1>, void (std::nullptr_t, base::Process), std::nullptr_t&, base::internal::PassedWrapper<base::Process> >, base::internal::InvokeHelper<false, void, base::Callback<void (std::nullptr_t, base::Process), (base::internal::CopyMode)1> >, void ()>::Run(base::internal::BindStateBase*) 0x00000001063993da (Google Chrome Framework -callback.h:397 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&) 0x00000001063bb982 (Google Chrome Framework -message_loop.cc:476 ) base::MessageLoop::RunTask(base::PendingTask const&) 0x00000001063bbc5b (Google Chrome Framework -message_loop.cc:485 ) base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&) 0x00000001063bbe4a (Google Chrome Framework -message_loop.cc:597 ) base::MessageLoop::DoWork() 0x000000010638e8b0 (Google Chrome Framework -message_pump_mac.mm:330 ) base::MessagePumpCFRunLoopBase::RunWork() 0x00000001063b1159 (Google Chrome Framework + 0x00565159 ) base::mac::CallWithEHFrame(void () block_pointer) 0x000000010638e2b3 (Google Chrome Framework -message_pump_mac.mm:306 ) base::MessagePumpCFRunLoopBase::RunWorkSource(void*) ----- Thread 0 CRASHED [0x00000000 / 0x00000000 @ 0x00007fff87657f06 ] MAGIC SIGNATURE THREAD 0x00007fff87657f06 (libsystem_kernel.dylib + 0x00016f06 ) 0x00007fff8f9a06e6 (libsystem_c.dylib + 0x0005e6e6 ) 0x00007fff9bafa395 (libsystem_malloc.dylib + 0x00010395 ) 0x00007fff9baefda5 (libsystem_malloc.dylib + 0x00005da5 ) 0x00007fff9baecb63 (libsystem_malloc.dylib + 0x00002b63 ) 0x00007fff9baf0bb9 (libsystem_malloc.dylib + 0x00006bb9 ) 0x0000000102954a3f (Google Chrome Framework -memory_mac.mm:165 ) base::(anonymous namespace)::oom_killer_realloc(_malloc_zone_t*, void*, unsigned long) 0x00007fff9baf07ea (libsystem_malloc.dylib + 0x000067ea ) 0x00007fff9baf0709 (libsystem_malloc.dylib + 0x00006709 ) 0x0000000102950ccd (Google Chrome Framework -pickle.cc:377 ) base::Pickle::WriteBytes(void const*, int) 0x0000000102950c2d (Google Chrome Framework -pickle.cc:329 ) base::Pickle::WriteString(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > const&) 0x00000001033ba54a (Google Chrome Framework -ipc_message_utils.h:302 ) IPC::(anonymous namespace)::WriteValue(base::Pickle*, base::Value const*, int) 0x00000001033ba67d (Google Chrome Framework -ipc_message_utils.cc:198 ) IPC::(anonymous namespace)::WriteValue(base::Pickle*, base::Value const*, int) 0x00000001033ba5f1 (Google Chrome Framework -ipc_message_utils.cc:189 ) IPC::(anonymous namespace)::WriteValue(base::Pickle*, base::Value const*, int) 0x00000001033ba5f1 (Google Chrome Framework -ipc_message_utils.cc:189 ) IPC::(anonymous namespace)::WriteValue(base::Pickle*, base::Value const*, int) 0x00000001033ba5f1 (Google Chrome Framework -ipc_message_utils.cc:189 ) IPC::(anonymous namespace)::WriteValue(base::Pickle*, base::Value const*, int) 0x00000001056aa246 (Google Chrome Framework -ipc_message_utils.h:88 ) IPC::ParamTraits<ExtensionMsg_Loaded_Params>::Write(base::Pickle*, ExtensionMsg_Loaded_Params const&) 0x00000001056a1bca (Google Chrome Framework -ipc_message_utils.h:88 ) IPC::MessageT<ExtensionMsg_Loaded_Meta, std::__1::tuple<std::__1::vector<ExtensionMsg_Loaded_Params, std::__1::allocator<ExtensionMsg_Loaded_Params> > >, void>::MessageT(IPC::Routing, std::__1::vector<ExtensionMsg_Loaded_Params, std::__1::allocator<ExtensionMsg_Loaded_Params> > const&) 0x00000001064a658f (Google Chrome Framework -ipc_message_templates.h:98 ) extensions::RendererStartupHelper::Observe(int, content::NotificationSource const&, content::NotificationDetails const&) 0x0000000105d87cf7 (Google Chrome Framework -notification_service_impl.cc:123 ) content::NotificationServiceImpl::Notify(int, content::NotificationSource const&, content::NotificationDetails const&) 0x0000000105df057b (Google Chrome Framework -render_process_host_impl.cc:2601 ) content::RenderProcessHostImpl::OnProcessLaunched() 0x0000000105c18e73 (Google Chrome Framework -child_process_launcher.cc:543 ) content::ChildProcessLauncher::Notify(std::nullptr_t, base::Process) 0x0000000105c186f9 (Google Chrome Framework -child_process_launcher.cc:508 ) content::ChildProcessLauncher::DidLaunch(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process) 0x0000000105c19157 (Google Chrome Framework -bind_internal.h:159 ) base::internal::Invoker<base::IndexSequence<0ul, 1ul>, base::internal::BindState<base::internal::RunnableAdapter<void (*)(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process)>, void (base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process), base::WeakPtr<content::ChildProcessLauncher>, bool const&>, base::internal::InvokeHelper<false, void, base::internal::RunnableAdapter<void (*)(base::WeakPtr<content::ChildProcessLauncher>, bool, std::nullptr_t, base::Process)> >, void (std::nullptr_t, base::Process)>::Run(base::internal::BindStateBase*, std::nullptr_t&&, base::Process&&) 0x0000000105c18ff7 (Google Chrome Framework -callback.h:397 ) base::internal::Invoker<base::IndexSequence<0ul, 1ul>, base::internal::BindState<base::Callback<void (std::nullptr_t, base::Process), (base::internal::CopyMode)1>, void (std::nullptr_t, base::Process), std::nullptr_t&, base::internal::PassedWrapper<base::Process> >, base::internal::InvokeHelper<false, void, base::Callback<void (std::nullptr_t, base::Process), (base::internal::CopyMode)1> >, void ()>::Run(base::internal::BindStateBase*) 0x00000001029203da (Google Chrome Framework -callback.h:397 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&) 0x0000000102942982 (Google Chrome Framework -message_loop.cc:476 ) base::MessageLoop::RunTask(base::PendingTask const&) 0x0000000102942c5b (Google Chrome Framework -message_loop.cc:485 ) base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&) 0x0000000102942e4a (Google Chrome Framework -message_loop.cc:597 ) base::MessageLoop::DoWork() 0x00000001029158b0 (Google Chrome Framework -message_pump_mac.mm:330 ) base::MessagePumpCFRunLoopBase::RunWork() 0x0000000102938159 (Google Chrome Framework + 0x00565159 ) base::mac::CallWithEHFrame(void () block_pointer) 0x00000001029152b3 (Google Chrome Framework -message_pump_mac.mm:306 ) base::MessagePumpCFRunLoopBase::RunWorkSource(void*) ----- Thread 0 CRASHED [EXC_BAD_INSTRUCTION / 0x00000001 @ 0x00007fff9bddd02f ] MAGIC SIGNATURE THREAD 0x00007fff9bddd02f (libobjc.A.dylib + 0x0001302f ) 0x00007fff9bddd17e (libobjc.A.dylib + 0x0001317e ) 0x00007fff9bde86bd (libobjc.A.dylib + 0x0001e6bd ) 0x00007fff9bdd1e62 (libobjc.A.dylib + 0x00007e62 ) 0x00007fff9bdd1de3 (libobjc.A.dylib + 0x00007de3 ) 0x00007fff9bdd1d5b (libobjc.A.dylib + 0x00007d5b ) 0x00007fff9bdd1cce (libobjc.A.dylib + 0x00007cce ) 0x0000000102ac5550 (Google Chrome Framework -objc_zombie.mm:158 ) (anonymous namespace)::ZombieDealloc(objc_object*, objc_selector*) 0x00007fff9649d5ab (AppKit + 0x000825ab ) 0x00007fff96423975 (AppKit + 0x00008975 ) 0x00007fff965ba93b (AppKit + 0x0019f93b ) 0x00007fff965b9591 (AppKit + 0x0019e591 ) 0x00007fff965ba65f (AppKit + 0x0019f65f ) 0x0000000103206418 (Google Chrome Framework -skia_utils_mac.mm:248 ) skia:: 0x000000010322098a (Google Chrome Framework -image_skia_util_mac.mm:110 ) gfx::NSImageFromImageSkiaWithColorSpace(gfx::ImageSkia const&, CGColorSpace*) 0x000000010321b71c (Google Chrome Framework -image.cc:566 ) gfx::Image::ToNSImage() const 0x00000001058e7355 (Google Chrome Framework -browser_action_button.mm:409 ) -[BrowserActionButton updateState] 0x0000000106731192 (Google Chrome Framework -toolbar_actions_model.cc:156 ) ToolbarActionsModel::OnExtensionActionUpdated(ExtensionAction*, content::WebContents*, content::BrowserContext*) 0x000000010660edc7 (Google Chrome Framework -extension_action_api.cc:207 ) extensions::ExtensionActionAPI::NotifyChange(ExtensionAction*, content::WebContents*, content::BrowserContext*) 0x000000010660f2a6 (Google Chrome Framework -extension_action_api.cc:263 ) extensions::ExtensionActionAPI::ClearAllValuesForTab(content::WebContents*) 0x0000000106719cd5 (Google Chrome Framework -tab_helper.cc:263 ) extensions::TabHelper::DidNavigateMainFrame(content::LoadCommittedDetails const&, content::FrameNavigateParams const&) 0x0000000105f1f608 (Google Chrome Framework -web_contents_impl.cc:3175 ) content::WebContentsImpl::DidNavigateMainFramePostCommit(content::RenderFrameHostImpl*, content::LoadCommittedDetails const&, FrameHostMsg_DidCommitProvisionalLoad_Params const&) 0x0000000105cf409e (Google Chrome Framework -navigator_impl.cc:595 ) content::NavigatorImpl::DidNavigate(content::RenderFrameHostImpl*, FrameHostMsg_DidCommitProvisionalLoad_Params const&) 0x0000000105cf99e0 (Google Chrome Framework -render_frame_host_impl.cc:1076 ) content::RenderFrameHostImpl::OnDidCommitProvisionalLoad(IPC::Message const&) 0x0000000105cf8b5a (Google Chrome Framework -render_frame_host_impl.cc:507 ) content::RenderFrameHostImpl::OnMessageReceived(IPC::Message const&) 0x0000000105e3b998 (Google Chrome Framework -render_process_host_impl.cc:1801 ) content::RenderProcessHostImpl::OnMessageReceived(IPC::Message const&) 0x0000000103403237 (Google Chrome Framework -ipc_channel_proxy.cc:293 ) IPC::ChannelProxy::Context::OnDispatchMessage(IPC::Message const&) 0x000000010296e3da (Google Chrome Framework -callback.h:397 ) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&) 0x0000000102990982 (Google Chrome Framework -message_loop.cc:476 ) base::MessageLoop::RunTask(base::PendingTask const&) 0x0000000102990c5b (Google Chrome Framework -message_loop.cc:485 ) base::MessageLoop::DeferOrRunPendingTask(base::PendingTask const&) 0x0000000102990e4a (Google Chrome Framework -message_loop.cc:597 ) base::MessageLoop::DoWork() 0x00000001029638b0 (Google Chrome Framework -message_pump_mac.mm:330 ) base::MessagePumpCFRunLoopBase::RunWork() 0x0000000102986159 (Google Chrome Framework + 0x00565159 ) base::mac::CallWithEHFrame(void () block_pointer) 0x00000001029632b3 (Google Chrome Framework -message_pump_mac.mm:306 ) base::MessagePumpCFRunLoopBase::RunWorkSource(void*) ----- Does this indicate a bug in Chrome's extension subsystem or a bug in an extension I have installed?
,
Mar 23 2016
Issue 582413 has been merged into this issue.
,
Mar 23 2016
Also: In bug 583227 , something was related to --user-data-dir. Looks based on these dumps that you’re still using --user-data-dir. Can you confirm? Do you see this with --user-data-dir? Does anything change if you don’t use --user-data-dir?
,
Mar 23 2016
Right, I'd specified --user-data-dir and some other flags to test a customer's WebGL 2.0 prototype product. I was then attempting to view those crashes in about:crashes. Don't remember at the time whether that was in the Canary instance using --user-data-dir, or my usual one. Anyway, the most recent extension I installed was "WebGL Insight" and it looks like disabling that extension has allowed Chrome to once again view about:crashes. I'll try re-enabling it to confirm.
,
Mar 23 2016
Re-enabling WebGL Insight had no effect; still able to view about:crashes. Here's copy/paste of about:extensions. I removed details about Google-specific extensions. ADB 0.9.8 ADB Plugin for remote debugging Chrome on Android (Now deprecated) Permissions Details ID: dpngiggdglpdnjdoaefidgiigpemgage Inspect views: background.html Allow in incognito Collect errors Enabled BuildBot Error 8 Skip through errors in buildbot output. Permissions Details ID: iehocdgbbocmkdidlbnnfbmbinnahbae Allow in incognito Collect errors Warning: Google Chrome cannot prevent extensions from recording your browsing history. To disable this extension in incognito mode, unselect this option. Enabled Chrome Remote Desktop 50.0.2661.22 Access other computers or allow another user to access your computer securely over the Internet. Permissions Details ID: ajoainacpilcemgiakehflpbkbfipojk Enable Chrome Remote Desktop 49.0.2623.40 Access other computers or allow another user to access your computer securely over the Internet. Permissions Details ID: gbchcmhmhahfdphkhkmpfmihenigjmpp Inspect views: background.html (Inactive) Allow in incognito Collect errors Allow access to file URLs Enabled Chromebook Recovery Utility 3.1.6 Create recovery media for your Chromebook. Permissions Details ID: jndclpdbaamdhonoechobihbbiimdgai Inspect views: background page (Inactive) Allow in incognito Collect errors Enabled Earth View from Google Earth 2.11.1 Experience a beautiful image from Google Earth every time you open a new tab. Permissions Details ID: bhloflhklmhfpedakmangadcdofhnnoh Inspect views: background page (Inactive) Allow in incognito Collect errors Enabled Google Docs 0.9 Create and edit documents Permissions Details ID: aohghmighlieiainnegkcijnfilokake Allow in incognito Collect errors Enabled Google Docs Offline 1.4 Get things done offline with the Google Docs family of products. Permissions Details ID: ghbmnnjooekpmoecnnnilnnbdlolhkhi Inspect views: background page Allow in incognito Collect errors Enabled Google Sheets 1.1 Create and edit spreadsheets Permissions Details ID: felcaaldnbdncclmgdcncolpebgiejap Allow in incognito Collect errors Enabled Google Slides 0.9 Create and edit presentations Permissions Details ID: aapocclcgogkmnckokdopfmhonfmgoek Allow in incognito Collect errors Enabled Polycraft 1.0.0.25 You’ve been stranded on a mysterious island — what bad luck! Build a base and survive the island. Take the stand, hero! Permissions Details ID: eopfmbpfhhfnklgmjpoehcjaajhpbhbl Enable Rietveld Usability Toolkit 0.3.8 Making Rietveld usable Permissions Options Details ID: nmljjlfbnbekmadhbpfpkcminoejelga Inspect views: background page (Inactive) Allow in incognito Collect errors Enabled Shader Editor 1.0.16.1 Live editing shaders in the browser. Permissions Details ID: ggeaidddejpbakgafapihjbgdlbbbpob Inspect views: background page Allow in incognito Collect errors Allow access to file URLs Enabled SshInAWin 2.1.3.4 HTML5/CSS/JS SSH application Permissions Details ID: npcpnahjfihkilahpohiieimoffneflm Inspect views: background page (Inactive) Allow in incognito Collect errors Enabled They Need To Be Fed 1.0.5 Feed the monsters in this 360-degree gravity-based platformer! Permissions Details ID: madbiikglegjjjgpokagkobjaioeekpd Inspect views: background page (Inactive) Allow in incognito Collect errors Enabled Web Tracing Framework 2015.7.15.1 Rich tools for instrumenting, analyzing, and visualizing web apps. Make your app jank-free at 60fps! Permissions Developer website ID: gmdhhnlkjmknaopofnadmoamhmnlicme Inspect views: background page Allow in incognito Collect errors Enabled WebGL Insight 1.0.2 A helpful Chrome extension for WebGL development Permissions Details ID: djdcbmfacaaocoomokenoalbomllhnko Inspect views: background page Allow in incognito Collect errors Allow access to file URLs Enabled
,
Mar 23 2016
I do see --user-data-dir in the stack memory contents of go/crash/fd9bdd3800000000, Are these tests with enabling/disabling the WebGL Insight extension with or without --user-data-dir? Would you mind testing the inverse too?
,
Mar 23 2016
Enabling/disabling the WebGL Insight extension was done from within my Chrome instance that didn't have --user-data-dir set -- in other words, my normal set of extensions. The only extensions installed in the browser launched with --user-data-dir were the managed (Google only) extensions.
,
Mar 23 2016
I meant that I don’t see it in go/crash/fd9bdd3800000000. I only see it in go/crash/94dbdd3800000000.
,
Mar 23 2016
So now we’re back to “can’t reproduce?” That’s too bad.
,
Mar 23 2016
Unfortunately yes, back to not reproducible. Still 51.0.2688.0 (Official Build) canary (64-bit) -- so Canary hasn't updated. Tried launching Chrome with a fresh --user-data-dir, in case it happened the first time the browser installed the managed extensions -- no luck, no crash going to about:crashes.
,
Mar 23 2016
FYI, the crash report I was interested in getting the stack trace for was f6623d3800000000 . Got it, and supplied it to the customer.
,
Mar 24 2016
Glad we at least got something right. Too bad we couldn’t nail down this elusive crasher. :/
,
Mar 24 2016
Is there anything else about your attempted visit to about:crashes that might be relevant? Were you trying to hit it directly from a sad tab page? Is there something about the original crash that might be relevant here?
,
Mar 24 2016
fd9bdd3800000000 and 94dbdd3800000000 both have the crashed stack trace at thread 0, but both also have interesting things happening on a background thread in CrashUploadListCrashpad::LoadUploadList().
,
Mar 24 2016
I tried about:crash followed by about:crashes and managed to crash the browser with go/crash/f330656400000000.
,
Mar 24 2016
Cool. It might have had something to do with crash uploads being in progress. I can't seem to provoke it any more. I just loaded the customer's sample app which crashed the GPU process, then opened a new tab and visited about:crashes; it'd produced go/crash/12d3656400000000 already. When about:crashes was crashing yesterday, visiting that page in a different browser session (different user-data-dir) was also crashing the browser.
,
Mar 24 2016
Robert and I think we figured out what’s going on. CrashUploadListCrashpad::LoadUploadList() runs on a worker thread, dispatched by CrashUploadList::LoadUploadListAsynchronously(). The about:crashes UI calls this when it loads, from CrashesDOMHandler::RegisterMessages(): https://chromium.googlesource.com/chromium/src/+/cbf24e69bd47446397a0283b20c81671469f33ce/chrome/browser/ui/webui/crashes_ui.cc#113 but it also sets up to call it on receipt of a webui event, crash::kCrashesUIRequestCrashList ("requestCrashList"), in CrashesDOMHandler::HandleRequestCrashes(). That’s triggered by the requestCrashes() JS function here: https://chromium.googlesource.com/chromium/src/+/cbf24e69bd47446397a0283b20c81671469f33ce/components/crash/core/browser/resources/crashes.js#11 which is called in two places: five seconds after requesting background upload (seems to be a CrOS-specific thing), and also hanging off of DOMContentLoaded. And there’s the problem. The webui handler (in C++) will have kicked off a LoadUploadList() on a background thread, which will be working on upload_list_. Then, the JS will kick off its own LoadUploadList() on another background thread, which will start working on the same upload_list_. This is so thread-unsafe. This is OS-All but we’ve definitely seen reports on Mac and Windows. A little code archaeology tells me that this started to go off the rails in https://chromiumcodereview.appspot.com/23020015.
,
Mar 24 2016
mark: Could we make a change to the rate limiting logic to avoid this type of problem in the future? [Maybe, allow the first 2 crashes per hour to be uploaded?] Otherwise, we are pretty much guaranteed to never receive any crash reports from crashes that occur during the crash upload process.
,
Mar 24 2016
,
Mar 24 2016
is there a doc somewhere that discusses various thread interaction ? i do not have a good mental model of C++ pages, and i've asked a few other Chromium devs to no avail.
,
Mar 24 2016
Erik (#32), we’re discussing disabling client-side rate-limiting altogether. Windows did this with the Crashpad transition, and we were talking to crash-team to decide whether it was OK to maintain permanently (in which case we’d do it on Mac too).
,
Mar 24 2016
vapier (#34), http://dev.chromium.org/developers/design-documents/threading is an overview of Chrome’s threading model, although it doesn’t cover the worker pool which is used here.
,
Mar 24 2016
#31
The simplest fix would probably be to change the PostTask to PostSequencedWorkerTask like this:
UploadList::UploadList(
Delegate* delegate,
const base::FilePath& upload_log_path,
const scoped_refptr<base::SequencedWorkerPool>& worker_pool)
: delegate_(delegate),
upload_log_path_(upload_log_path),
worker_pool_(worker_pool),
sequence_token_(worker_pool->GetSequenceToken()) {}
UploadList::~UploadList() {}
void UploadList::LoadUploadListAsynchronously() {
DCHECK(thread_checker_.CalledOnValidThread());
worker_pool_->PostSequencedWorkerTask(
sequence_token_,
FROM_HERE,
base::Bind(&UploadList::LoadUploadListAndInformDelegateOfCompletion,
this, base::ThreadTaskRunnerHandle::Get()));
}
I don't have a chrome checkout anymore, but I can probably post this after the long weekend on tuesday unless someone else wants to
,
Mar 24 2016
I'd rather fix the thread safety issues by ensuring that uploads_ isn't accessed concurrently. https://codereview.chromium.org/1830383002
,
Mar 25 2016
Also verified that this race can be detected by TSan: https://codereview.chromium.org/1837503003/. (The unittest to do so will be rolled up into the patch above).
,
Mar 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6221751958a5f073b9557cfb89ba7a4197a5369d commit 6221751958a5f073b9557cfb89ba7a4197a5369d Author: rsesek <rsesek@chromium.org> Date: Mon Mar 28 15:41:38 2016 Fix thread safety issues with //components/upload_list. The uploads_ vector can be accessed simultaneously from multiple threads, which can lead to heap corruption. This change ensures access to uploads_ only on the creator task runner. Updates occur by currying the new data vector in a Closure across threads. BUG= 597384 Review URL: https://codereview.chromium.org/1830383002 Cr-Commit-Position: refs/heads/master@{#383495} [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/chrome/browser/crash_upload_list_crashpad.cc [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/chrome/browser/crash_upload_list_crashpad.h [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/components/upload_list/BUILD.gn [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/components/upload_list/upload_list.cc [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/components/upload_list/upload_list.h [modify] https://crrev.com/6221751958a5f073b9557cfb89ba7a4197a5369d/components/upload_list/upload_list_unittest.cc
,
Mar 28 2016
,
Mar 28 2016
Thanks for tracking that down and fixing it! |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by kbr@chromium.org
, Mar 23 2016