New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 684774 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

[USS] ModelTypeStore fails to init for DeviceInfo

Project Member Reported by maxbogue@chromium.org, Jan 24 2017

Issue description

My recent changes have revealed a number of errors in USS DeviceInfo:

https://uma.googleplex.com/p/chrome/timeline_v2/?sid=0f070dccf97e6ea8f09ad2609c9191db

It appears these are due to an unexpectedly high rate of failure creating the ModelTypeStore. We should fix this in M57 beta, or we shouldn't roll out USS to M57 stable. Sample stack dumps:

https://crash.corp.google.com/browse?q=ReportID%3D%271f4f384680000000
https://crash.corp.google.com/browse?q=ReportID%3D%27c135ad8680000000

0x000007feea6ad5d3	(chrome_elf.dll -crashpad_win.cc:155 )	DumpProcessWithoutCrash
0x000007fee97328cc	(chrome.dll -report_unrecoverable_error.cc:27 )	syncer::ReportUnrecoverableError(version_info::Channel)
0x000007fee9ba255e	(chrome.dll -shared_model_type_processor.cc:154 )	syncer::SharedModelTypeProcessor::ReportError(syncer::ModelError const &)
0x000007fee9ba25e5	(chrome.dll -shared_model_type_processor.cc:170 )	syncer::SharedModelTypeProcessor::ReportError(tracked_objects::Location const &,std::basic_string<char,std::char_traits<char>,std::allocator<char> > const &)
0x000007fee9b94e19	(chrome.dll -device_info_sync_bridge.cc:346 )	syncer::DeviceInfoSyncBridge::OnStoreCreated(syncer::ModelTypeStore::Result,std::unique_ptr<syncer::ModelTypeStore,std::default_delete<syncer::ModelTypeStore> >)
0x000007fee7df10ff	(chrome.dll -bind_internal.h:339 )	base::internal::Invoker<base::internal::BindState<void ( content::ServiceWorkerControlleeRequestHandler::*)(content::ServiceWorkerStatusCode,scoped_refptr<content::ServiceWorkerRegistration>),base::WeakPtr<content::ServiceWorkerControlleeRequestHandler> >,void >::Run(base::internal::BindStateBase *,content::ServiceWorkerStatusCode &&,scoped_refptr<content::ServiceWorkerRegistration> &&)
0x000007fee9ba6abf	(chrome.dll -model_type_store_impl.cc:148 )	syncer::ModelTypeStoreImpl::BackendInitDone(syncer::ModelType,std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend>)
0x000007fee9ba8d3f	(chrome.dll -bind_internal.h:339 )	base::internal::Invoker<base::internal::BindState<void (*)(syncer::ModelType,std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend>),syncer::ModelType,base::internal::PassedWrapper<std::unique_ptr<syncer::ModelTypeStore::Result,std::default_delete<syncer::ModelTypeStore::Result> > >,scoped_refptr<base::SequencedTaskRunner>,base::Callback<void ,1,1> >,void >::Run(base::internal::BindStateBase *,scoped_refptr<syncer::ModelTypeStoreBackend> &&)
0x000007fee9ba88cb	(chrome.dll -post_task_and_reply_with_result_internal.h:28 )	base::internal::ReplyAdapter<scoped_refptr<syncer::ModelTypeStoreBackend>,scoped_refptr<syncer::ModelTypeStoreBackend> >(base::Callback<void ,1,1> const &,scoped_refptr<syncer::ModelTypeStoreBackend> *)
0x000007fee820acb0	(chrome.dll -post_task_and_reply_impl.cc:61 )	base::`anonymous namespace'::PostTaskAndReplyRelay::RunReplyAndSelfDestruct
0x000007fee7c1c523	(chrome.dll -callback.h:68 )	base::internal::RunMixin<base::Callback<void ,0,0> >::Run( ?? )
0x000007fee821b711	(chrome.dll -task_annotator.cc:52 )	base::debug::TaskAnnotator::RunTask(char const *,base::PendingTask *)
0x000007fee81c9473	(chrome.dll -message_loop.cc:421 )	base::MessageLoop::RunTask(base::PendingTask *)
0x000007fee81ca0b4	(chrome.dll -message_loop.cc:523 )	base::MessageLoop::DoWork()
0x000007fee821bd10	(chrome.dll -message_pump_win.cc:173 )	base::MessagePumpForUI::DoRunLoop()
0x000007fee821b943	(chrome.dll -message_pump_win.cc:56 )	base::MessagePumpWin::Run(base::MessagePump::Delegate *)
0x000007fee81f2753	(chrome.dll -run_loop.cc:37 )	base::RunLoop::Run()
 
Issue 686896 has been merged into this issue.
Project Member

Comment 2 by sheriffbot@chromium.org, Jan 30 2017

Labels: FoundIn-M-57 Fracas OS-Android
Users experienced this crash on the following builds:

Android Dev 57.0.2987.9 -  1.86 CPM, 52 reports, 37 clients (signature [Dump without crash] syncer::SharedModelTypeProcessor::ReportError)

If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates.

- Go/Fracas

Comment 3 by pav...@chromium.org, Jan 31 2017

Status: Started (was: Assigned)
Project Member

Comment 4 by bugdroid1@chromium.org, Feb 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8db816e5533f4447c4c4fabc6ad9be956f354c71

commit 8db816e5533f4447c4c4fabc6ad9be956f354c71
Author: pavely <pavely@chromium.org>
Date: Wed Feb 01 21:17:21 2017

[Sync] Record histogram of ModelTypeStoreBackend initialization result

This change records reasons of ModelTypeStoreBackend failures in UMA histogram.

BUG= 684774 
R=maxbogue@chromium.org,asvitkine@chromium.org

Review-Url: https://codereview.chromium.org/2666793002
Cr-Commit-Position: refs/heads/master@{#447610}

[modify] https://crrev.com/8db816e5533f4447c4c4fabc6ad9be956f354c71/components/sync/model_impl/model_type_store_backend.cc
[modify] https://crrev.com/8db816e5533f4447c4c4fabc6ad9be956f354c71/tools/metrics/histograms/histograms.xml

Project Member

Comment 5 by sheriffbot@chromium.org, Feb 4 2017

Labels: FoundIn-M-58
Users experienced this crash on the following builds:

Android Dev 58.0.3000.3 -  3.36 CPM, 15 reports, 13 clients (signature [Dump without crash] syncer::SharedModelTypeProcessor::ReportError)

If this update was incorrect, please add "Fracas-Wrong" label to prevent future updates.

- Go/Fracas
Project Member

Comment 6 by bugdroid1@chromium.org, Feb 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b

commit 323b23d6a1ebe7f3d6ed581b6ef7334c6207460b
Author: pavely <pavely@chromium.org>
Date: Tue Feb 14 18:50:01 2017

[Sync] Reset ModelTypeStore's leveldb database after corruption

The issue is that once leveldb database is corrupt it will keep failing
initialization.

I decided to reset database instead of attempting to recover it because recover
doesn't seem to guarantee consistency of data.

R=pnoland@chromium.org
BUG= 684774 

Review-Url: https://codereview.chromium.org/2682063004
Cr-Commit-Position: refs/heads/master@{#450423}

[modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend.cc
[modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend.h
[modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/components/sync/model_impl/model_type_store_backend_unittest.cc
[modify] https://crrev.com/323b23d6a1ebe7f3d6ed581b6ef7334c6207460b/tools/metrics/histograms/histograms.xml

Comment 7 by pav...@chromium.org, Feb 15 2017

Labels: M-57 Merge-Request-57
Status: Fixed (was: Started)
I'm requesting a merge to M57. This issue causes clients that us USS store and experienced database corruption to stay in corrupt state. It will affect signed in users of ReadingList which is launching in M57 on iOS.

The change is localized, I've covered functionality with unittest and did manual verification.
Project Member

Comment 8 by sheriffbot@chromium.org, Feb 15 2017

Labels: -Merge-Request-57 Hotlist-Merge-Approved Merge-Approved-57
Your change meets the bar and is auto-approved for M57. Please go ahead and merge the CL to branch 2987 manually. Please contact milestone owner if you have questions.
Owners: amineer@(clank), cmasso@(bling), ketakid@(cros), govind@(desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Project Member

Comment 9 by bugdroid1@chromium.org, Feb 15 2017

Labels: -merge-approved-57 merge-merged-2987
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/748442278ae4b6020cf91b8f21b7d7b505ddb4fc

commit 748442278ae4b6020cf91b8f21b7d7b505ddb4fc
Author: Pavel Yatsuk <pavely@chromium.org>
Date: Wed Feb 15 20:40:33 2017

[Sync] Record histogram of ModelTypeStoreBackend initialization result

This change records reasons of ModelTypeStoreBackend failures in UMA histogram.

BUG= 684774 
R=maxbogue@chromium.org,asvitkine@chromium.org

Review-Url: https://codereview.chromium.org/2666793002
Cr-Commit-Position: refs/heads/master@{#447610}
(cherry picked from commit 8db816e5533f4447c4c4fabc6ad9be956f354c71)

Review-Url: https://codereview.chromium.org/2696703007 .
Cr-Commit-Position: refs/branch-heads/2987@{#530}
Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943}

[modify] https://crrev.com/748442278ae4b6020cf91b8f21b7d7b505ddb4fc/components/sync/model_impl/model_type_store_backend.cc
[modify] https://crrev.com/748442278ae4b6020cf91b8f21b7d7b505ddb4fc/tools/metrics/histograms/histograms.xml

Project Member

Comment 10 by bugdroid1@chromium.org, Feb 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2e69e04e27c0e5d763bdf30b9cb696d0269dd302

commit 2e69e04e27c0e5d763bdf30b9cb696d0269dd302
Author: Pavel Yatsuk <pavely@chromium.org>
Date: Wed Feb 15 20:44:58 2017

[Sync] Reset ModelTypeStore's leveldb database after corruption

The issue is that once leveldb database is corrupt it will keep failing
initialization.

I decided to reset database instead of attempting to recover it because recover
doesn't seem to guarantee consistency of data.

R=pnoland@chromium.org
BUG= 684774 

Review-Url: https://codereview.chromium.org/2682063004
Cr-Commit-Position: refs/heads/master@{#450423}
(cherry picked from commit 323b23d6a1ebe7f3d6ed581b6ef7334c6207460b)

Review-Url: https://codereview.chromium.org/2692993008 .
Cr-Commit-Position: refs/branch-heads/2987@{#531}
Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943}

[modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend.cc
[modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend.h
[modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/components/sync/model_impl/model_type_store_backend_unittest.cc
[modify] https://crrev.com/2e69e04e27c0e5d763bdf30b9cb696d0269dd302/tools/metrics/histograms/histograms.xml

Sign in to add a comment