New issue
Advanced search Search tips

Issue 716728 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug


Participants' hotlists:
dmurph-shortlist-bugs


Sign in to add a comment

"IndexedDBBrowserTest.CursorTestIncognito" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, Apr 29 2017

Issue description

"IndexedDBBrowserTest.CursorTestIncognito" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 6 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyMwsSBUZsYWtlIihJbmRleGVkREJCcm93c2VyVGVzdC5DdXJzb3JUZXN0SW5jb2duaXRvDA.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
 

Comment 1 by suzyh@chromium.org, May 1 2017

Cc: jsb...@chromium.org dmu...@chromium.org
Components: Blink>Storage>IndexedDB
Labels: -Sheriff-Chromium
The chromium-try-flakes link above lists this as flaky on linux_android_rel_ng. The flakiness dashboard suggests it's flakily crashing.

Selectively disabling on Android: https://codereview.chromium.org/2853753002
Project Member

Comment 2 by bugdroid1@chromium.org, May 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/093418e5cfed99faf4b70372ef605d1e06c3098f

commit 093418e5cfed99faf4b70372ef605d1e06c3098f
Author: suzyh <suzyh@chromium.org>
Date: Mon May 01 04:24:18 2017

Disable flaky CursorTestIncognito on Android

IndexedDBBrowserTest.CursorTestIncognito is flaky on
linux_android_rel_ng.

TBR=jsbell@chromium.org
BUG= 716728 

Review-Url: https://codereview.chromium.org/2853753002
Cr-Commit-Position: refs/heads/master@{#468282}

[modify] https://crrev.com/093418e5cfed99faf4b70372ef605d1e06c3098f/content/browser/indexed_db/indexed_db_browsertest.cc

Looks like:

[FATAL:memory_dump_manager.cc(384)] Check failed: (*mdp_iter)->task_runner && (*mdp_iter)->task_runner->RunsTasksOnCurrentThread(). MemoryDumpProvider "IndexedDBBackingStore" attempted to unregister itself in a racy way. Please file a crbug.

Cc: pwnall@chromium.org
Owner: ssid@chromium.org
Cc: -dmu...@chromium.org ssid@chromium.org
Owner: dmu...@chromium.org
Probably nothing specific about that test, although Incognito means it's going through the IndexedDBBackingStore::OpenInMemory path. Shouldn't matter though.

Still... the IndexedDBBackingStore should be used only from the IDB task runner, it owns the LevelDBDatabase instance.

Comment 6 by ssid@chromium.org, May 1 2017

check is fired when unregistration happens on different task runner. I can see 2 fixes: Fix the task runner on which LevelDBDatabase::~LevelDBDatabase is destructed. If that is not possible for the test, disable the dump provider registration using set_dumper_registrations_ignored_for_testing().

Comment 7 by ssid@chromium.org, May 1 2017

Yeah, we should probably start flaking on all the similar tests and all the platforms.
Weird... ~LevelDBDatabase is indeed run only on the task runner passed to the dump provider (ContentBrowserTest doesn't have any relevant threading changes I'm aware of). I'm not seeing what might be flaky.

Comment 9 by ssid@google.com, May 1 2017

There are no changes in the indexed DB, but I made the dcheck to fire always in this cl https://codereview.chromium.org/2844373002/ this caused the flakes to increase.
where did you all get the symbolized stack trace?

Comment 11 by ssid@chromium.org, May 2 2017

I cannot find symbolized stack traces either.
I didn't get a stack, just the CHECK output:

OP > "list of all flakes can be found at" > pick one "try run" > buildbot output > content_browsertests > stdout > "Check failed"

ah - got it.

Comment 14 by mek@chromium.org, May 8 2017

Status: Assigned (was: Untriaged)
Ah! I found this issue while creating tests for the mojo interface. It was happening when the threads were joining, and later destruction of the test quota client caused this (after threads were joined / task runners destructed / different).
So the problem that I encountered that had this checkfail was this stacktrace:

[7035:7035:0512/171253.201149:7968656769096:FATAL:memory_dump_manager.cc(385)] Check failed: (*mdp_iter)->task_runner && (*mdp_iter)->task_runner->RunsTasksOnCurrentThread(). MemoryDumpProvider "IndexedDBBackingStore" attempted to unregister itself in a racy way. Please file a crbug.
#0 0x7fa18562f47b base::debug::StackTrace::StackTrace()
#1 0x7fa18562e17c base::debug::StackTrace::StackTrace()
#2 0x7fa1856a19b3 logging::LogMessage::~LogMessage()
#3 0x7fa18584fda9 base::trace_event::MemoryDumpManager::UnregisterDumpProviderInternal()
#4 0x7fa18584f9bf base::trace_event::MemoryDumpManager::UnregisterDumpProvider()
#5 0x7fa181f91590 content::LevelDBDatabase::~LevelDBDatabase()
#6 0x7fa181f91789 content::LevelDBDatabase::~LevelDBDatabase()
#7 0x7fa180c0ec2f std::default_delete<>::operator()()
#8 0x7fa181eba84c std::unique_ptr<>::reset()
#9 0x7fa181e9965c content::IndexedDBBackingStore::~IndexedDBBackingStore()
#10 0x7fa181e99789 content::IndexedDBBackingStore::~IndexedDBBackingStore()
#11 0x7fa181ec8f72 base::RefCounted<>::Release()
#12 0x7fa181ec8f19 scoped_refptr<>::Release()
#13 0x7fa181ebb25a scoped_refptr<>::~scoped_refptr()
#14 0x7fa181f57793 std::pair<>::~pair()
#15 0x7fa181f57769 std::_Rb_tree_node<>::~_Rb_tree_node()
#16 0x7fa181f57749 __gnu_cxx::new_allocator<>::destroy<>()
#17 0x7fa181f5771c std::_Rb_tree<>::_M_destroy_node()
#18 0x7fa181f576d8 std::_Rb_tree<>::_M_erase()
#19 0x7fa181f57665 std::_Rb_tree<>::~_Rb_tree()
#20 0x7fa181f57635 std::__cxx1998::map<>::~map()
#21 0x7fa181f54d3f std::__debug::map<>::~map()
#22 0x7fa181f50fd9 content::IndexedDBFactoryImpl::~IndexedDBFactoryImpl()
#23 0x7fa181f51029 content::IndexedDBFactoryImpl::~IndexedDBFactoryImpl()
#24 0x7fa181eef76b base::RefCountedThreadSafe<>::DeleteInternal()
#25 0x7fa181eef735 base::DefaultRefCountedThreadSafeTraits<>::Destruct()
#26 0x7fa181eef70c base::RefCountedThreadSafe<>::Release()
#27 0x7fa181eef6c9 scoped_refptr<>::Release()
#28 0x7fa181efc8c2 scoped_refptr<>::operator=()
#29 0x7fa181efb835 content::IndexedDBContextImpl::~IndexedDBContextImpl() <---- HERE
#30 0x7fa181efbe09 content::IndexedDBContextImpl::~IndexedDBContextImpl()
#31 0x7fa180c3bd2b base::RefCountedThreadSafe<>::DeleteInternal()
#32 0x7fa180c3bcf5 base::DefaultRefCountedThreadSafeTraits<>::Destruct()
#33 0x7fa1819df52c base::RefCountedThreadSafe<>::Release()
#34 0x7fa1819df4e9 scoped_refptr<>::Release()
#35 0x7fa1819d70ea scoped_refptr<>::~scoped_refptr()
#36 0x7fa181f76bc1 content::IndexedDBQuotaClient::~IndexedDBQuotaClient()
#37 0x7fa181f76bf9 content::IndexedDBQuotaClient::~IndexedDBQuotaClient()
#38 0x7fa181f76c4b content::IndexedDBQuotaClient::OnQuotaManagerDestroyed()
#39 0x7fa186215eec storage::QuotaManager::~QuotaManager()
#40 0x000002335a4e content::MockQuotaManager::~MockQuotaManager()
#41 0x000002335a99 content::MockQuotaManager::~MockQuotaManager()
#42 0x7fa18621ae0a storage::QuotaManager::DeleteOnCorrectThread()


In the destructor of IndexedDBContextImpl, if the taskrunner is for a destroyed/joined thread, then that will cause it to be destructed on the joined thread where the task runners won't be correct.

I'm going to make a patch that fixes that line so we std::move the pointer - maybe that will help.
Note: That isn't from this test. it was from a different tests I Was making for another patch.
Project Member

Comment 18 by bugdroid1@chromium.org, May 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/957b96d1ad97e774141de94caec6644174f5d651

commit 957b96d1ad97e774141de94caec6644174f5d651
Author: dmurph <dmurph@chromium.org>
Date: Fri May 19 01:58:12 2017

[IndexedDB] Fixed possible refcount race

BUG= 716728 

Review-Url: https://codereview.chromium.org/2892703003
Cr-Commit-Position: refs/heads/master@{#473029}

[modify] https://crrev.com/957b96d1ad97e774141de94caec6644174f5d651/content/browser/indexed_db/indexed_db_context_impl.cc

reverting patch to see if it is fixed.
Project Member

Comment 20 by bugdroid1@chromium.org, May 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/082f0e6aa47050a15c1d2adef8966cad11298f76

commit 082f0e6aa47050a15c1d2adef8966cad11298f76
Author: dmurph <dmurph@chromium.org>
Date: Fri May 19 20:05:23 2017

Revert of Disable flaky CursorTestIncognito on Android (patchset #1 id:1 of https://codereview.chromium.org/2853753002/ )

Reason for revert:
May have fixed flakiness

Original issue's description:
> Disable flaky CursorTestIncognito on Android
>
> IndexedDBBrowserTest.CursorTestIncognito is flaky on
> linux_android_rel_ng.
>
> TBR=jsbell@chromium.org
> BUG= 716728 
>
> Review-Url: https://codereview.chromium.org/2853753002
> Cr-Commit-Position: refs/heads/master@{#468282}
> Committed: https://chromium.googlesource.com/chromium/src/+/093418e5cfed99faf4b70372ef605d1e06c3098f

TBR=jsbell@chromium.org,suzyh@chromium.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG= 716728 

Review-Url: https://codereview.chromium.org/2895493003
Cr-Commit-Position: refs/heads/master@{#473293}

[modify] https://crrev.com/082f0e6aa47050a15c1d2adef8966cad11298f76/content/browser/indexed_db/indexed_db_browsertest.cc

Status: Fixed (was: Assigned)

Sign in to add a comment