chrome crash reports still busted for certain types of crahses |
|||||
Issue descriptionAs you can see from: https://chromium-review.googlesource.com/c/442284/ ...and the associated bug, crash reports were busted for a while in a pretty major way. That was fixed. You can see the M57 pick landed Feb 16 at: https://chromium-review.googlesource.com/c/443308 ...but we are still seeing a certain class of crashes that has no good crash reports. Specifically see: a767e034a0000000 When I look at that report I still see a stack quality of 18% and I see "Missing symbols: chrome". The crash is from 9202.51.0 on 2017-03-08 --- ...but if I download this type of crash and symbolize manually and go into GDB then I get a wonderfully nice stack crawl. (gdb) bt #0 0xece0fb36 in ?? () from r/lib/libc.so.6 #1 0xece1ecd4 in raise () from r/lib/libc.so.6 #2 0xece1fc36 in abort () from r/lib/libc.so.6 #3 0xb771808e in base::debug::BreakDebugger() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/debug/debugger_posix.cc:251 #4 0xb77248b2 in logging::LogMessage::~LogMessage() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/logging.cc:759 #5 0xb7739630 in base::(anonymous namespace)::OnNoMemorySize(unsigned int) [clone .constprop.4] () at ../../../../../../../home/chrome-bot/chrome_root/src/base/process/memory_linux.cc:35 #6 0xb6296c82 in operator new(unsigned int, std::nothrow_t const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/base/allocator/allocator_shim.cc:68 #7 0xb82b2476 in leveldb::ReadBlock(leveldb::RandomAccessFile*, leveldb::ReadOptions const&, leveldb::BlockHandle const&, leveldb::BlockContents*) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/format.cc:124 #8 0xb82ad92a in leveldb::Table::BlockReader(void*, leveldb::ReadOptions const&, leveldb::Slice const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/table.cc:189 #9 0xb82aecee in leveldb::(anonymous namespace)::TwoLevelIterator::InitDataBlock() () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/two_level_iterator.cc:165 #10 0xb82aeef4 in leveldb::(anonymous namespace)::TwoLevelIterator::Seek(leveldb::Slice const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/two_level_iterator.cc:93 #11 0xb82ad422 in leveldb::IteratorWrapper::Seek(leveldb::Slice const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/iterator_wrapper.h:47 #12 0xb82ad460 in leveldb::(anonymous namespace)::MergingIterator::Seek(leveldb::Slice const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/table/merger.cc:53 #13 0xb82a1ea0 in leveldb::(anonymous namespace)::DBIter::Seek(leveldb::Slice const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/leveldatabase/src/db/db_iter.cc:280 #14 0xb6e2f934 in content::LevelDBIteratorImpl::Seek(base::BasicStringPiece<std::string> const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/indexed_db/leveldb/leveldb_iterator_impl.cc:45 #15 0xb6e303ca in content::LevelDBTransaction::TransactionIterator::Seek(base::BasicStringPiece<std::string> const&) () at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/indexed_db/leveldb/leveldb_transaction.cc:230 #16 0xb6e056c0 in content::IndexedDBBackingStore::Cursor::FirstSeek(leveldb::Status*) () at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/indexed_db/indexed_db_backing_store.cc:3182 #17 0xb6e09778 in content::IndexedDBBackingStore::OpenObjectStoreCursor(content::IndexedDBBackingStore::Transaction*, long long, long long, content::IndexedDBKeyRange const&, blink::WebIDBCursorDirection, leveldb::Status*) () at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/indexed_db/indexed_db_backing_store.cc:3969 #18 0xb6e1a138 in content::IndexedDBDatabase::OpenCursorOperation(std::unique_ptr<content::IndexedDBDatabase::OpenCursorOperationParams, std::default_delete<content::IndexedDBDatabase::OpenCursorOperationParams> >, content::IndexedDBTransaction*) () at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/indexed_db/indexed_db_database.cc:1536 #19 0xb6e1c5f8 in base::internal::Invoker<base::internal::BindState<leveldb::Status (content::IndexedDBDatabase::*)(std::unique_ptr<content::IndexedDBDatabase::OpenCursorOperationParams, std::default_delete<content::IndexedDBDatabase::OpenCursorOperationParams> >, content::IndexedDBTransaction*), scoped_refptr<content::IndexedDBDatabase>, base::internal::PassedWrapper<std::unique_ptr<content::IndexedDBDatabase::OpenCursorOperationParams, std::default_delete<content::IndexedDBDatabase::OpenCursorOperationParams> > > >, leveldb::Status (content::IndexedDBTransaction*)>::Run(base::internal::BindStateBase*, content::IndexedDBTransaction*&&) () at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:214 #20 0xb6e2d200 in content::IndexedDBTransaction::ProcessTaskQueue() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:85 #21 0xb63fd84c in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) () at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:68 #22 0xb63f2212 in base::MessageLoop::DoWork() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_loop.cc:421 #23 0xb63f2608 in base::MessagePumpDefault::Run(base::MessagePump::Delegate*) () at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_pump_default.cc:33 #24 0xb773e02a in base::RunLoop::Run() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/run_loop.cc:37 #25 0xb7750c30 in base::Thread::ThreadMain() () at ../../../../../../../home/chrome-bot/chrome_root/src/base/threading/thread.cc:328 #26 0xb774ed5e in base::(anonymous namespace)::ThreadFunc(void*) () at ../../../../../../../home/chrome-bot/chrome_root/src/base/threading/platform_thread_posix.cc:71 #27 0xed4385fa in ?? () from r/lib/libpthread.so.0 --- What's wrong?
,
Mar 14 2017
This is likely an issue in the Breakpad client for Linux. It only affects about 3% of the crash reports for this particular version of Chrome: https://dashboards.corp.google.com/edit/google::_a0806b44_bb6a_4325_8e34_459b59c89c4f Screenshot: https://screenshot.googleplex.com/tuRZ4XYG6ZN.png Given the small percentage of affected reports and the fact that Breakpad will be replaced with a Crashpad client by the end of this year, I think we should not spend too much time on this issue. Resolving as won't fix.
,
Mar 14 2017
,
Mar 14 2017
> Breakpad will be replaced with a Crashpad client by the end of this year for Linux ? have a link ?
,
Mar 14 2017
Here is a link to Crashpad 2016-1017 roadmap: https://docs.google.com/document/d/1ZjPuKe-2TPyfcy5S9WtXIJAMTb_koMY6IV9BOqkyF4A/edit#heading=h.mlfzhjfiwd62 Joshua Peraza and Mark Mentovai are currently working on the Crashpad client for Android. The Linux client should follow soon after that.
,
Mar 16 2017
re #2: I don't get this. Where do you get 3% from? I am seeing 90% of all top crashes for ARM completely busted here. We can't make any call about ARM build stability based on this: https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_ChromeOS%27%20AND%20product.version%3D%2757.0.2987.100%27%20AND%20custom_data.ChromeCrashProto.os_linux.lsb_release%20CONTAINS%20LOWER(%27MINNIE%27)%20AND%20cpu.architecture%3D%27arm%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27renderer%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#-samplereports:5,+magicsignature
,
Mar 16 2017
@6: I think that's actually a separate issue and is the topic of: - bug #665083: Chrome_ChromeOS: Crash Report - __divdi3, __clock_gettime, ... - bug #702388: minidumps don't seem to handle stack overflows well --- Actually, my analysis in bug #665083 (comment 106) indicates that this may account for some of the things claimed as "TCMalloc corruption", too.
,
Mar 16 2017
Hi Zel, the 3% I was referring to are about the issue described in #1. I can see that many of the top clusters in the above URL (#6) have broken stack traces. I haven't looked at these in much detail yet, however according to bug #665083 the stack of the main thread seems busted which explains why Breakpad cannot walk the stack and symbolize its contents. The stack traces for the rest of the treads looks good though.
,
Mar 16 2017
Let's close this and discuss the other issues in their corresponding bugs.
,
May 3 2017
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by vapier@chromium.org
, Mar 14 2017