[USS] ModelTypeStore fails to init for DeviceInfo |
|||||||
Issue descriptionMy recent changes have revealed a number of errors in USS DeviceInfo: https://uma.googleplex.com/p/chrome/timeline_v2/?sid=0f070dccf97e6ea8f09ad2609c9191db It appears these are due to an unexpectedly high rate of failure creating the ModelTypeStore. We should fix this in M57 beta, or we shouldn't roll out USS to M57 stable. Sample stack dumps: https://crash.corp.google.com/browse?q=ReportID%3D%271f4f384680000000 https://crash.corp.google.com/browse?q=ReportID%3D%27c135ad8680000000 0x000007feea6ad5d3 (chrome_elf.dll -crashpad_win.cc:155 ) DumpProcessWithoutCrash 0x000007fee97328cc (chrome.dll -report_unrecoverable_error.cc:27 ) syncer::ReportUnrecoverableError(version_info::Channel) 0x000007fee9ba255e (chrome.dll -shared_model_type_processor.cc:154 ) syncer::SharedModelTypeProcessor::ReportError(syncer::ModelError const &) 0x000007fee9ba25e5 (chrome.dll -shared_model_type_processor.cc:170 ) syncer::SharedModelTypeProcessor::ReportError(tracked_objects::Location const &,std::basic_string<char,std::char_traits<char>,std::allocator<char> > const &) 0x000007fee9b94e19 (chrome.dll -device_info_sync_bridge.cc:346 ) syncer::DeviceInfoSyncBridge::OnStoreCreated(syncer::ModelTypeStore::Result,std::unique_ptr<syncer::ModelTypeStore,std::default_delete<syncer::ModelTypeStore> >) 0x000007fee7df10ff (chrome.dll -bind_internal.h:339 ) base::internal::Invoker<base::internal::BindState<void ( content::ServiceWorkerControlleeRequestHandler::*)(content::ServiceWorkerStatusCode,scoped_refptr<content::ServiceWorkerRegistration>),base::WeakPtr<content::ServiceWorkerControlleeRequestHandler> >,void >::Run(base::internal::BindStateBase *,content::ServiceWorkerStatusCode &&,scoped_refptr<content::ServiceWorkerRegistration> &&) 0x000007fee9ba6abf (chrome.dll -model_type_store_impl.cc:148 ) syncer::ModelTypeStoreImpl::BackendInitDone(syncer::ModelType,std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend>) 0x000007fee9ba8d3f (chrome.dll -bind_internal.h:339 ) base::internal::Invoker<base::internal::BindState<void (*)(syncer::ModelType,std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend>),syncer::ModelType,base::internal::PassedWrapper<std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> > >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> >,void >::Run(base::internal::BindStateBase *,scoped_refptr<syncer::ModelTypeStoreBackend> &&) 0x000007fee9ba88cb (chrome.dll -post_task_and_reply_with_result_internal.h:28 ) base::internal::ReplyAdapter<scoped_refptr<syncer::ModelTypeStoreBackend>,scoped_refptr<syncer::ModelTypeStoreBackend> >(base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend> *) 0x000007fee820acb0 (chrome.dll -post_task_and_reply_impl.cc:61 ) base::`anonymous namespace'::PostTaskAndReplyRelay::RunReplyAndSelfDestruct 0x000007fee7c1c523 (chrome.dll -callback.h:68 ) base::internal::RunMixin<base::Callback<void ,0,0> >::Run( ?? ) 0x000007fee821b711 (chrome.dll -task_annotator.cc:52 ) base::debug::TaskAnnotator::RunTask(char const *,base::PendingTask *) 0x000007fee81c9473 (chrome.dll -message_loop.cc:421 ) base::MessageLoop::RunTask(base::PendingTask *) 0x000007fee81ca0b4 (chrome.dll -message_loop.cc:523 ) base::MessageLoop::DoWork() 0x000007fee821bd10 (chrome.dll -message_pump_win.cc:173 ) base::MessagePumpForUI::DoRunLoop() 0x000007fee821b943 (chrome.dll -message_pump_win.cc:56 ) base::MessagePumpWin::Run(base::MessagePump::Delegate *) 0x000007fee81f2753 (chrome.dll -run_loop.cc:37 ) base::RunLoop::Run()
,
Jan 30 2017
Users experienced this crash on the following builds: Android Dev 57.0.2987.9 - 1.86 CPM, 52 reports, 37 clients (signature [Dump without crash] syncer::SharedModelTypeProcessor::ReportError) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Jan 31 2017
,
Feb 1 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/8db816e5533f4447c4c4fabc6ad9be956f354c71 commit 8db816e5533f4447c4c4fabc6ad9be956f354c71 Author: pavely <pavely@chromium.org> Date: Wed Feb 01 21:17:21 2017 [Sync] Record histogram of ModelTypeStoreBackend initialization result This change records reasons of ModelTypeStoreBackend failures in UMA histogram. BUG= 684774 R=maxbogue@chromium.org,asvitkine@chromium.org Review-Url: https://codereview.chromium.org/2666793002 Cr-Commit-Position: refs/heads/master@{#447610} [modify] https://crrev.com/8db816e5533f4447c4c4fabc6ad9be956f354c71/components/sync/model_impl/model_type_store_backend.cc [modify] https://crrev.com/8db816e5533f4447c4c4fabc6ad9be956f354c71/tools/metrics/histograms/histograms.xml
,
Feb 4 2017
Users experienced this crash on the following builds: Android Dev 58.0.3000.3 - 3.36 CPM, 15 reports, 13 clients (signature [Dump without crash] syncer::SharedModelTypeProcessor::ReportError) If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates. - Go/Fracas
,
Feb 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b commit 323b23d6a1ebe7f3d6ed581b6ef7334c6207460b Author: pavely <pavely@chromium.org> Date: Tue Feb 14 18:50:01 2017 [Sync] Reset ModelTypeStore's leveldb database after corruption The issue is that once leveldb database is corrupt it will keep failing initialization. I decided to reset database instead of attempting to recover it because recover doesn't seem to guarantee consistency of data. R=pnoland@chromium.org BUG= 684774 Review-Url: https://codereview.chromium.org/2682063004 Cr-Commit-Position: refs/heads/master@{#450423} [modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend.cc [modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend.h [modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend_unittest.cc [modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/tools/metrics/histograms/histograms.xml
,
Feb 15 2017
I'm requesting a merge to M57. This issue causes clients that us USS store and experienced database corruption to stay in corrupt state. It will affect signed in users of ReadingList which is launching in M57 on iOS. The change is localized, I've covered functionality with unittest and did manual verification.
,
Feb 15 2017
Your change meets the bar and is auto-approved for M57. Please go ahead and merge the CL to branch 2987 manually. Please contact milestone owner if you have questions. Owners: amineer@(clank), cmasso@(bling), ketakid@(cros), govind@(desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Feb 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/748442278ae4b6020cf91b8f21b7d7b505ddb4fc commit 748442278ae4b6020cf91b8f21b7d7b505ddb4fc Author: Pavel Yatsuk <pavely@chromium.org> Date: Wed Feb 15 20:40:33 2017 [Sync] Record histogram of ModelTypeStoreBackend initialization result This change records reasons of ModelTypeStoreBackend failures in UMA histogram. BUG= 684774 R=maxbogue@chromium.org,asvitkine@chromium.org Review-Url: https://codereview.chromium.org/2666793002 Cr-Commit-Position: refs/heads/master@{#447610} (cherry picked from commit 8db816e5533f4447c4c4fabc6ad9be956f354c71) Review-Url: https://codereview.chromium.org/2696703007 . Cr-Commit-Position: refs/branch-heads/2987@{#530} Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943} [modify] https://crrev.com/748442278ae4b6020cf91b8f21b7d7b505ddb4fc/components/sync/model_impl/model_type_store_backend.cc [modify] https://crrev.com/748442278ae4b6020cf91b8f21b7d7b505ddb4fc/tools/metrics/histograms/histograms.xml
,
Feb 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2e69e04e27c0e5d763bdf30b9cb696d0269dd302 commit 2e69e04e27c0e5d763bdf30b9cb696d0269dd302 Author: Pavel Yatsuk <pavely@chromium.org> Date: Wed Feb 15 20:44:58 2017 [Sync] Reset ModelTypeStore's leveldb database after corruption The issue is that once leveldb database is corrupt it will keep failing initialization. I decided to reset database instead of attempting to recover it because recover doesn't seem to guarantee consistency of data. R=pnoland@chromium.org BUG= 684774 Review-Url: https://codereview.chromium.org/2682063004 Cr-Commit-Position: refs/heads/master@{#450423} (cherry picked from commit 323b23d6a1ebe7f3d6ed581b6ef7334c6207460b) Review-Url: https://codereview.chromium.org/2692993008 . Cr-Commit-Position: refs/branch-heads/2987@{#531} Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943} [modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend.cc [modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend.h [modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend_unittest.cc [modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/tools/metrics/histograms/histograms.xml |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by maxbogue@chromium.org
, Jan 30 2017