New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 639812 link

Starred by 2 users
Status: Verified
Owner:
gab@chromium.org
Closed: Sep 2016
Cc:
█ dvyukov@chromium.org
danakj@chromium.org
robliao@chromium.org
thestig@chromium.org
gab@chromium.org
fdoray@chromium.org
█ brettw@chromium.org
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 3
Type: Bug



Sign in to add a comment

TSan reports data races on g_all_pools_state

Project Member Reported by glider@chromium.org, Aug 22 2016 
See https://build.chromium.org/p/chromium.memory.full/builders/Linux%20TSan%20Tests/builds/788/steps/content_browsertests%20on%20Ubuntu-12.04/logs/stdio:

[ RUN      ] SitePerProcessBrowserTest.ScrollEventToOOPIF
[16412:16412:0822/041708:35681770591:WARNING:audio_manager.cc(317)] Multiple instances of AudioManager detected
[16412:16412:0822/041708:35681770810:WARNING:audio_manager.cc(278)] Multiple instances of AudioManager detected
Xlib:  extension "RANDR" missing on display ":9".
==================
WARNING: ThreadSanitizer: data race (pid=16412)
  Read of size 4 at 0x0000089fb6f8 by main thread (mutexes: write M2027):
    #0 base::SequencedWorkerPool::Inner::IsRunningSequenceOnCurrentThread(base::SequencedWorkerPool::SequenceToken) const base/threading/sequenced_worker_pool.cc:826:7 (content_browsertests+0x00000288b444)
    #1 IsRunningSequenceOnCurrentThread base/threading/sequenced_worker_pool.cc:1527:18 (content_browsertests+0x00000288cec8)
    #2 base::(anonymous namespace)::SequencedWorkerPoolSequencedTaskRunner::RunsTasksOnCurrentThread() const base/threading/sequenced_worker_pool.cc:220 (content_browsertests+0x00000288cec8)
    #3 storage::SandboxFileSystemBackendDelegate::SandboxFileSystemBackendDelegate(storage::QuotaManagerProxy*, base::SequencedTaskRunner*, base::FilePath const&, storage::SpecialStoragePolicy*, storage::FileSystemOptions const&) storage/browser/fileapi/sandbox_file_system_backend_delegate.cc:209:27 (content_browsertests+0x0000025992c6)
    #4 storage::FileSystemContext::FileSystemContext(base::SingleThreadTaskRunner*, base::SequencedTaskRunner*, storage::ExternalMountPoints*, storage::SpecialStoragePolicy*, storage::QuotaManagerProxy*, ScopedVector<storage::FileSystemBackend>, std::__1::vector<base::Callback<bool (net::URLRequest const*, storage::FileSystemURL const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::Callback<void (base::File::Error), (base::internal::CopyMode)1> const&), (base::internal::CopyMode)1>, std::__1::allocator<base::Callback<bool (net::URLRequest const*, storage::FileSystemURL const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::Callback<void (base::File::Error), (base::internal::CopyMode)1> const&), (base::internal::CopyMode)1> > > const&, base::FilePath const&, storage::FileSystemOptions const&) storage/browser/fileapi/file_system_context.cc:154:15 (content_browsertests+0x00000256b969)
    #5 content::CreateFileSystemContext(content::BrowserContext*, base::FilePath const&, bool, storage::QuotaManagerProxy*) content/browser/fileapi/browser_file_system_helper.cc:78:11 (content_browsertests+0x000001ecd99d)
    #6 content::StoragePartitionImpl::Create(content::BrowserContext*, bool, base::FilePath const&) content/browser/storage_partition_impl.cc:451:7 (content_browsertests+0x00000222ffa3)
    #7 content::StoragePartitionImplMap::Get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) content/browser/storage_partition_impl_map.cc:397:7 (content_browsertests+0x000002234b6a)
    #8 content::(anonymous namespace)::GetStoragePartitionFromConfig(content::BrowserContext*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) content/browser/browser_context.cc:116:25 (content_browsertests+0x000001dd6ec8)
    #9 content::BrowserContext::GetStoragePartition(content::BrowserContext*, content::SiteInstance*) content/browser/browser_context.cc:244:10 (content_browsertests+0x000001dd6e14)
    #10 content::SiteInstanceImpl::GetProcess() content/browser/site_instance_impl.cc:121:17 (content_browsertests+0x0000022145b5)
    #11 content::WebContentsImpl::Init(content::WebContents::CreateParams const&) content/browser/web_contents/web_contents_impl.cc:1534:24 (content_browsertests+0x00000226c5e3)
    #12 content::WebContentsImpl::CreateWithOpener(content::WebContents::CreateParams const&, content::FrameTreeNode*) content/browser/web_contents/web_contents_impl.cc:596:17 (content_browsertests+0x00000225c0dc)
...
  Previous write of size 4 at 0x0000089fb6f8 by thread T12:
    #0 base::SequencedWorkerPool::Inner::FinishStartingAdditionalThread(int) base/threading/sequenced_worker_pool.cc:1325:23 (content_browsertests+0x00000288b169)
    #1 base::SequencedWorkerPool::Inner::PostTask(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const*, base::SequencedWorkerPool::SequenceToken, base::SequencedWorkerPool::WorkerShutdown, tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::TimeDelta) base/threading/sequenced_worker_pool.cc:725:7 (content_browsertests+0x00000288ad90)
    #2 PostDelayedSequencedWorkerTask base/threading/sequenced_worker_pool.cc:1492:18 (content_browsertests+0x00000288ce67)
    #3 base::(anonymous namespace)::SequencedWorkerPoolSequencedTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::TimeDelta) base/threading/sequenced_worker_pool.cc:216 (content_browsertests+0x00000288ce67)
    #4 base::TaskRunner::PostTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&) base/task_runner.cc:45:10 (content_browsertests+0x00000287e053)
    #5 catalog::Reader::Read(base::FilePath const&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::unique_ptr<catalog::Entry, std::__1::default_delete<catalog::Entry> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::unique_ptr<catalog::Entry, std::__1::default_delete<catalog::Entry> > > > >*, base::Callback<void (), (base::internal::CopyMode)1> const&) services/catalog/reader.cc:168:22 (content_browsertests+0x000000cd8faa)
    #6 catalog::Catalog::ScanSystemPackageDir() services/catalog/catalog.cc:102:19 (content_browsertests+0x000000cd0166)
    #7 catalog::Catalog::Catalog(base::SequencedWorkerPool*, std::__1::unique_ptr<catalog::Store, std::__1::default_delete<catalog::Store> >, catalog::ManifestProvider*) services/catalog/catalog.cc:75:3 (content_browsertests+0x000000ccfd45)
    #8 content::MojoShellContext::InProcessServiceManagerContext::StartOnIOThread(std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service>) content/browser/mojo/mojo_shell_context.cc:188:13 (content_browsertests+0x00000206447c)
    #9 Invoke<const scoped_refptr<content::MojoShellContext::InProcessServiceManagerContext> &, std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service> > base/bind_internal.h:214:12 (content_browsertests+0x000002064827)
    #10 MakeItSo<void (content::MojoShellContext::InProcessServiceManagerContext::*const &)(std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service>), const scoped_refptr<content::MojoShellContext::InProcessServiceManagerContext> &, std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service> > base/bind_internal.h:283 (content_browsertests+0x000002064827)
    #11 RunImpl<void (content::MojoShellContext::InProcessServiceManagerContext::*const &)(std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service>), const std::__1::tuple<scoped_refptr<content::MojoShellContext::InProcessServiceManagerContext>, base::internal::PassedWrapper<std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> > >, base::internal::PassedWrapper<mojo::InterfacePtrInfo<shell::mojom::Service> > > &, 0, 1, 2> base/bind_internal.h:346 (content_browsertests+0x000002064827)
    #12 base::internal::Invoker<base::internal::BindState<void (content::MojoShellContext::InProcessServiceManagerContext::*)(std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> >, mojo::InterfacePtrInfo<shell::mojom::Service>), scoped_refptr<content::MojoShellContext::InProcessServiceManagerContext>, base::internal::PassedWrapper<std::__1::unique_ptr<content::(anonymous namespace)::BuiltinManifestProvider, std::__1::default_delete<content::(anonymous namespace)::BuiltinManifestProvider> > >, base::internal::PassedWrapper<mojo::InterfacePtrInfo<shell::mojom::Service> > >, void ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:324 (content_browsertests+0x000002064827)
    #13 Run base/callback.h:388:12 (content_browsertests+0x0000028d0ab9)
...
  Location is global 'base::(anonymous namespace)::g_all_pools_state' of size 4 at 0x0000089fb6f8 (content_browsertests+0x0000089fb6f8)

  Mutex M2027 (0x7d5000006008) created at:
    #0 pthread_mutex_init <null> (content_browsertests+0x0000004b1cc3)
    #1 base::internal::LockImpl::LockImpl() base/synchronization/lock_impl_posix.cc:45:8 (content_browsertests+0x000002878e3d)
    #2 Lock base/synchronization/lock.h:24:12 (content_browsertests+0x00000288bdde)
    #3 Inner base/threading/sequenced_worker_pool.cc:606 (content_browsertests+0x00000288bdde)
    #4 base::SequencedWorkerPool::SequencedWorkerPool(unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::TaskPriority) base/threading/sequenced_worker_pool.cc:1396 (content_browsertests+0x00000288bdde)
    #5 BrowserThreadGlobals content/browser/browser_thread_impl.cc:105:17 (content_browsertests+0x000001dee0c5)
    #6 New base/lazy_instance.h:69 (content_browsertests+0x000001dee0c5)
    #7 New base/lazy_instance.h:98 (content_browsertests+0x000001dee0c5)
    #8 Pointer base/lazy_instance.h:163 (content_browsertests+0x000001dee0c5)
    #9 base::LazyInstance<content::(anonymous namespace)::BrowserThreadGlobals, base::internal::LeakyLazyInstanceTraits<content::(anonymous namespace)::BrowserThreadGlobals> >::Get() base/lazy_instance.h:137 (content_browsertests+0x000001dee0c5)
    #10 Initialize content/browser/browser_thread_impl.cc:311:45 (content_browsertests+0x000001dee222)
    #11 content::BrowserThreadImpl::BrowserThreadImpl(content::BrowserThread::ID, base::MessageLoop*) content/browser/browser_thread_impl.cc:156 (content_browsertests+0x000001dee222)
...
  Thread T12 'Chrome_IOThread' (tid=16462, running) created by main thread at:
    #0 pthread_create <null> (content_browsertests+0x0000004b0cb5)
    #1 base::(anonymous namespace)::CreateThread(unsigned long, bool, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:110:13 (content_browsertests+0x000002888427)
    #2 base::PlatformThread::CreateWithPriority(unsigned long, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:191:10 (content_browsertests+0x000002888325)
    #3 base::Thread::StartWithOptions(base::Thread::Options const&) base/threading/thread.cc:108:15 (content_browsertests+0x00000288e319)
    #4 content::BrowserThreadImpl::StartWithOptions(base::Thread::Options const&) content/browser/browser_thread_impl.cc:349:25 (content_browsertests+0x000001def144)
    #5 content::BrowserMainLoop::CreateThreads() content/browser/browser_main_loop.cc:924:32 (content_browsertests+0x000001ddd0ca)
    #6 Invoke<content::BrowserMainLoop *> base/bind_internal.h:214:12 (content_browsertests+0x000001de1cb5)
    #7 MakeItSo<int (content::BrowserMainLoop::*const &)(), content::BrowserMainLoop *> base/bind_internal.h:283 (content_browsertests+0x000001de1cb5)
    #8 RunImpl<int (content::BrowserMainLoop::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<content::BrowserMainLoop> > &, 0> base/bind_internal.h:346 (content_browsertests+0x000001de1cb5)
    #9 base::internal::Invoker<base::internal::BindState<int (content::BrowserMainLoop::*)(), base::internal::UnretainedWrapper<content::BrowserMainLoop> >, int ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:324 (content_browsertests+0x000001de1cb5)
    #10 Run base/callback.h:388:12 (content_browsertests+0x00000222ead1)
    #11 content::StartupTaskRunner::RunAllTasksNow() content/browser/startup_task_runner.cc:45 (content_browsertests+0x00000222ead1)
    #12 content::BrowserMainLoop::CreateStartupTasks() content/browser/browser_main_loop.cc:832:25 (content_browsertests+0x000001ddc65a)
...
SUMMARY: ThreadSanitizer: data race base/threading/sequenced_worker_pool.cc:826:7 in base::SequencedWorkerPool::Inner::IsRunningSequenceOnCurrentThread(base::SequencedWorkerPool::SequenceToken) const

Comment 1 by danakj@chromium.org, Aug 22 2016

Cc: gab@chromium.org robliao@chromium.org
Owner: fdoray@chromium.org

Comment 2 by fdoray@chromium.org, Aug 26 2016 

I've been able to reproduce a similar error in DiskCacheBackendTest.DoomEntriesSinceSparse:

WARNING: ThreadSanitizer: data race (pid=15561)
  Write of size 4 at 0x0000052784e8 by thread T4:
    #0 base::SequencedWorkerPool::Inner::FinishStartingAdditionalThread(int) base/threading/sequenced_worker_pool.cc:1327:23 (net_unittests+0x0000033429ef)
    #1 base::SequencedWorkerPool::Inner::PostTask(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const*, base::SequencedWorkerPool::SequenceToken, base::SequencedWorkerPool::WorkerShutdown, tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::TimeDelta) base/threading/sequenced_worker_pool.cc:727:7 (net_unittests+0x000003342394)
    #2 PostDelayedWorkerTask base/threading/sequenced_worker_pool.cc:1467:18 (net_unittests+0x0000033446fc)
    #3 base::SequencedWorkerPool::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::TimeDelta) base/threading/sequenced_worker_pool.cc:1520 (net_unittests+0x0000033446fc)
    #4 PostTask base/task_runner.cc:45:10 (net_unittests+0x000003334830)
    #5 base::(anonymous namespace)::PostTaskAndReplyTaskRunner::PostTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&) base/task_runner.cc:38 (net_unittests+0x000003334830)
    #6 base::internal::PostTaskAndReplyImpl::PostTaskAndReply(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::Callback<void (), (base::internal::CopyMode)1> const&) base/threading/post_task_and_reply_impl.cc:92:8 (net_unittests+0x00000333f468)
    #7 base::TaskRunner::PostTaskAndReply(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)1> const&, base::Callback<void (), (base::internal::CopyMode)1> const&) base/task_runner.cc:52:43 (net_unittests+0x000003334682)
    #8 bool base::PostTaskAndReplyWithResult<int, int>(base::TaskRunner*, tracked_objects::Location const&, base::Callback<int (), (base::internal::CopyMode)1> const&, base::Callback<void (int), (base::internal::CopyMode)1> const&) base/task_runner_util.h:60:23 (net_unittests+0x000002eaf613)
    #9 disk_cache::File::Read(void*, unsigned long, unsigned long, disk_cache::FileIOCallback*, bool*) net/disk_cache/blockfile/file_posix.cc:98:3 (net_unittests+0x000002eaf3ae)
    #10 (anonymous namespace)::ChildrenDeleter::ReadData(disk_cache::Addr, int) net/disk_cache/blockfile/sparse_control.cc:125:14 (net_unittests+0x000002ebc0b4)
    #11 Invoke<const scoped_refptr<(anonymous namespace)::ChildrenDeleter> &, const disk_cache::Addr &, const int &> base/bind_internal.h:214:12 (net_unittests+0x000002ebe7b8)
    #12 MakeItSo<void ((anonymous namespace)::ChildrenDeleter::*const &)(disk_cache::Addr, int), const scoped_refptr<(anonymous namespace)::ChildrenDeleter> &, const disk_cache::Addr &, const int &> base/bind_internal.h:283 (net_unittests+0x000002ebe7b8)
    #13 RunImpl<void ((anonymous namespace)::ChildrenDeleter::*const &)(disk_cache::Addr, int), const std::__1::tuple<scoped_refptr<(anonymous namespace)::ChildrenDeleter>, disk_cache::Addr, int> &, 0, 1, 2> base/bind_internal.h:346 (net_unittests+0x000002ebe7b8)
    #14 base::internal::Invoker<base::internal::BindState<void ((anonymous namespace)::ChildrenDeleter::*)(disk_cache::Addr, int), scoped_refptr<(anonymous namespace)::ChildrenDeleter>, disk_cache::Addr, int>, void ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:324 (net_unittests+0x000002ebe7b8)
    #15 Run base/callback.h:388:12 (net_unittests+0x000003378499)
    #16 base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask const&) base/debug/task_annotator.cc:54 (net_unittests+0x000003378499)
    #17 base::MessageLoop::RunTask(base::PendingTask const&) base/message_loop/message_loop.cc:488:19 (net_unittests+0x0000032f9b2c)
    #18 base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) base/message_loop/message_loop.cc:497:5 (net_unittests+0x0000032fa18d)
    #19 base::MessageLoop::DoWork() base/message_loop/message_loop.cc:621:13 (net_unittests+0x0000032fa944)
    #20 base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) base/message_loop/message_pump_libevent.cc:217:31 (net_unittests+0x000003300790)
    #21 base::MessageLoop::RunHandler() base/message_loop/message_loop.cc:451:10 (net_unittests+0x0000032f9390)
    #22 base::RunLoop::Run() base/run_loop.cc:35:10 (net_unittests+0x00000331f6ee)
    #23 base::Thread::Run(base::RunLoop*) base/threading/thread.cc:228:13 (net_unittests+0x0000033474ab)
    #24 base::Thread::ThreadMain() base/threading/thread.cc:301:3 (net_unittests+0x0000033477b0)
    #25 base::(anonymous namespace)::ThreadFunc(void*) base/threading/platform_thread_posix.cc:71:13 (net_unittests+0x00000333f198)

  Previous read of size 4 at 0x0000052784e8 by main thread (mutexes: write M784890017620847632):
    #0 base::SequencedWorkerPool::Inner::RunsTasksOnCurrentThread() const base/threading/sequenced_worker_pool.cc:813:7 (net_unittests+0x000003342bc0)
    #1 base::SequencedWorkerPool::Inner::CleanupForTesting() base/threading/sequenced_worker_pool.cc:849:3 (net_unittests+0x000003343024)
    #2 base::SequencedWorkerPool::FlushForTesting() base/threading/sequenced_worker_pool.cc:1533:11 (net_unittests+0x0000033447e3)
    #3 FlushForTesting net/disk_cache/simple/simple_backend_impl.cc:72:52 (net_unittests+0x000002ecd5af)
    #4 disk_cache::SimpleBackendImpl::FlushWorkerPoolForTesting() net/disk_cache/simple/simple_backend_impl.cc:724 (net_unittests+0x000002ecd5af)
    #5 DiskCacheTestWithCache::TearDown() net/disk_cache/disk_cache_test_base.cc:273:3 (net_unittests+0x00000324cc8d)
    #6 HandleExceptionsInMethodIfSupported<testing::Test, void> testing/gtest/src/gtest.cc:2458:12 (net_unittests+0x0000033bcc5e)
    #7 testing::Test::Run() testing/gtest/src/gtest.cc:2482 (net_unittests+0x0000033bcc5e)
    #8 testing::TestInfo::Run() testing/gtest/src/gtest.cc:2656:11 (net_unittests+0x0000033bdbcd)
    #9 testing::TestCase::Run() testing/gtest/src/gtest.cc:2774:28 (net_unittests+0x0000033be4a6)
    #10 testing::internal::UnitTestImpl::RunAllTests() testing/gtest/src/gtest.cc:4647:43 (net_unittests+0x0000033c7936)
    #11 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> testing/gtest/src/gtest.cc:2458:12 (net_unittests+0x0000033c72e4)
    #12 testing::UnitTest::Run() testing/gtest/src/gtest.cc:4255 (net_unittests+0x0000033c72e4)
    #13 RUN_ALL_TESTS testing/gtest/include/gtest/gtest.h:2237:46 (net_unittests+0x0000038fe1c4)
    #14 base::TestSuite::Run() base/test/test_suite.cc:246 (net_unittests+0x0000038fe1c4)
    #15 Invoke<NetTestSuite *> base/bind_internal.h:214:12 (net_unittests+0x000001d2da05)
    #16 MakeItSo<int (base::TestSuite::*const &)(), NetTestSuite *> base/bind_internal.h:283 (net_unittests+0x000001d2da05)
    #17 RunImpl<int (base::TestSuite::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<NetTestSuite> > &, 0> base/bind_internal.h:346 (net_unittests+0x000001d2da05)
    #18 base::internal::Invoker<base::internal::BindState<int (base::TestSuite::*)(), base::internal::UnretainedWrapper<NetTestSuite> >, int ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:324 (net_unittests+0x000001d2da05)
    #19 Run base/callback.h:388:12 (net_unittests+0x0000038ffa67)
    #20 base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1> const&) base/test/launcher/unit_test_launcher.cc:206 (net_unittests+0x0000038ffa67)
    #21 base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1> const&) base/test/launcher/unit_test_launcher.cc:445:10 (net_unittests+0x0000038ff8e1)
    #22 main net/test/run_all_unittests.cc:59:10 (net_unittests+0x000001d2d957)

  Location is global 'base::(anonymous namespace)::g_all_pools_state' of size 4 at 0x0000052784e8 (net_unittests+0x0000052784e8)

  Mutex M784890017620847632 is already destroyed.

  Thread T4 'CacheThread' (tid=15603, running) created by main thread at:
    #0 pthread_create <null> (net_unittests+0x0000005be455)
    #1 base::(anonymous namespace)::CreateThread(unsigned long, bool, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:110:13 (net_unittests+0x00000333ed35)
    #2 base::PlatformThread::CreateWithPriority(unsigned long, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) base/threading/platform_thread_posix.cc:191:10 (net_unittests+0x00000333ebe5)
    #3 base::Thread::StartWithOptions(base::Thread::Options const&) base/threading/thread.cc:108:15 (net_unittests+0x000003346d34)
    #4 DiskCacheTestWithCache::InitDiskCache() net/disk_cache/disk_cache_test_base.cc:303:5 (net_unittests+0x00000324aa49)
    #5 DiskCacheTestWithCache::InitCache() net/disk_cache/disk_cache_test_base.cc:99:5 (net_unittests+0x00000324a377)
    #6 DiskCacheBackendTest::InitSparseCache(base::Time*, base::Time*) net/disk_cache/backend_unittest.cc:197:3 (net_unittests+0x000000affc51)
    #7 DiskCacheBackendTest_DoomEntriesSinceSparse_Test::TestBody() net/disk_cache/backend_unittest.cc:1700:3 (net_unittests+0x000000b16e1e)
    #8 HandleExceptionsInMethodIfSupported<testing::Test, void> testing/gtest/src/gtest.cc:2458:12 (net_unittests+0x0000033bcb5c)
    #9 testing::Test::Run() testing/gtest/src/gtest.cc:2474 (net_unittests+0x0000033bcb5c)
    #10 testing::TestInfo::Run() testing/gtest/src/gtest.cc:2656:11 (net_unittests+0x0000033bdbcd)
    #11 testing::TestCase::Run() testing/gtest/src/gtest.cc:2774:28 (net_unittests+0x0000033be4a6)
    #12 testing::internal::UnitTestImpl::RunAllTests() testing/gtest/src/gtest.cc:4647:43 (net_unittests+0x0000033c7936)
    #13 HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> testing/gtest/src/gtest.cc:2458:12 (net_unittests+0x0000033c72e4)
    #14 testing::UnitTest::Run() testing/gtest/src/gtest.cc:4255 (net_unittests+0x0000033c72e4)
    #15 RUN_ALL_TESTS testing/gtest/include/gtest/gtest.h:2237:46 (net_unittests+0x0000038fe1c4)
    #16 base::TestSuite::Run() base/test/test_suite.cc:246 (net_unittests+0x0000038fe1c4)
    #17 Invoke<NetTestSuite *> base/bind_internal.h:214:12 (net_unittests+0x000001d2da05)
    #18 MakeItSo<int (base::TestSuite::*const &)(), NetTestSuite *> base/bind_internal.h:283 (net_unittests+0x000001d2da05)
    #19 RunImpl<int (base::TestSuite::*const &)(), const std::__1::tuple<base::internal::UnretainedWrapper<NetTestSuite> > &, 0> base/bind_internal.h:346 (net_unittests+0x000001d2da05)
    #20 base::internal::Invoker<base::internal::BindState<int (base::TestSuite::*)(), base::internal::UnretainedWrapper<NetTestSuite> >, int ()>::Run(base::internal::BindStateBase*) base/bind_internal.h:324 (net_unittests+0x000001d2da05)
    #21 Run base/callback.h:388:12 (net_unittests+0x0000038ffa67)
    #22 base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1> const&) base/test/launcher/unit_test_launcher.cc:206 (net_unittests+0x0000038ffa67)
    #23 base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1> const&) base/test/launcher/unit_test_launcher.cc:445:10 (net_unittests+0x0000038ff8e1)
    #24 main net/test/run_all_unittests.cc:59:10 (net_unittests+0x000001d2d957)

SUMMARY: ThreadSanitizer: data race base/threading/sequenced_worker_pool.cc:1327:23 in base::SequencedWorkerPool::Inner::FinishStartingAdditionalThread(int)

Comment 3 by fdoray@chromium.org, Aug 26 2016 

|g_all_pools_state| is written when a task is posted to a SequencedWorkerPool (inside SequencedWorkerPool::Inner::FinishStartingAdditionalThread) and read from various SequencedWorkerPool methods. Accesses to this variable aren't synchronized by a lock.

DiskCacheBackendTest.DoomEntriesSinceSparse posts a task to a cache thread which itself posts a task to a SequencedWorkerPool. In the meantime, the test calls FlushForTesting on the pool. This results in a TSan error because the read and write aren't synchronized.

My preferred solution would be to require RedirectSequencedWorkerPoolsToTaskSchedulerForProcess() to be called before any SequencedWorkerPool is instantiated in the process. SequencedWorkerPool's constructor would acquire a global lock, copy |g_all_pools_state| to a member variable and set |g_all_pools_state| to SEQUENCED_WORKER_POOL_CREATED if appropriate. RedirectSequencedWorkerPoolsToTaskSchedulerForProcess() would acquire the same global lock, DCHECK_NE(SEQUENCED_WORKER_POOL_CREATED, g_all_pools_state) and set |g_all_pools_state| to REDIRECTED_TO_TASK_SCHEDULER. Acquiring a lock in SequencedWorkerPool's constructor shouldn't be a problem since instantiating a SequencedWorkerPool isn't frequent.

robliao@ is changing the field trial list initialization to allow that.

Comment 4 by robliao@chromium.org, Aug 30 2016

Cc: fdoray@chromium.org
Owner: robliao@chromium.org
Will be fixed when I split the finch and seed initialization paths.

Comment 5 by robliao@chromium.org, Aug 30 2016

Blockedon: 642473

Comment 6 by gab@chromium.org, Aug 31 2016 

The race is benign. RedirectSequencedWorkerPoolsToTaskSchedulerForProcess() is required to be called before any other threads are created so the race is being caught is read of g_all_pools_state after it was set to WORKER_CREATED, but no logic ever checks that it is set to that value (instead only checks for == and != w.r.t. NONE_ACTIVE).

So in the WORKER mode, it's fine if any thread reads either NONE_ACTIVE or WORKER_CREATED.

@rob don't think your change will fix this, the race is still there after initialization, it's just benign in all cases...

Shall we suppress?

Comment 7 by gab@chromium.org, Aug 31 2016 

 Issue 640152  has been merged into this issue.

Comment 8 by gab@chromium.org, Aug 31 2016 

I guess it's only really a race that matters if the write/read is non-atomic (i.e. half-read/write is possible), but given the int type that seems highly unlikely (impossible?) to me.

I don't want to add a global lock in product code for this.

Comment 9 by glider@chromium.org, Sep 1 2016 

We should generally move towards reduction of the suppressions file, not its extension.
Moreover, there's no such thing as benign data races (see https://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong).
If we want the access to be atomic here, we should use base::subtle to access this variable (namely NoBarrier_Load() and NoBarrier_Store()).

Comment 10 by gab@chromium.org, Sep 1 2016 

Got it, so TSAN is fine with there being a read/write race, it just doesn't like that the operation is not atomic? If so, I'll make it atomic.

Comment 11 by glider@chromium.org, Sep 1 2016 

No, it's just that according to the C++ Standard there is no race if both accesses are atomic.

Comment 12 by gab@chromium.org, Sep 1 2016 

Very good read, thanks for highlighting this, will fix today.

Comment 13 by gab@chromium.org, Sep 1 2016

Owner: gab@chromium.org
Status: Started (was: Assigned)
Project Member

Comment 14 by bugdroid1@chromium.org, Sep 2 2016 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/edb053ea0f3192d89926a1f1fa50747ec00fa078

commit edb053ea0f3192d89926a1f1fa50747ec00fa078
Author: gab <gab@chromium.org>
Date: Fri Sep 02 13:03:51 2016

Fix |g_all_pools_state| data race by using atomics.

And change a few checks from != NONE_ACTIVE to == WORKER_CREATED
where it's guaranteed (i.e. in code that can only run after a
Worker is created).

BUG= 639812 

Review-Url: https://codereview.chromium.org/2295323004
Cr-Commit-Position: refs/heads/master@{#416250}

[modify] https://crrev.com/edb053ea0f3192d89926a1f1fa50747ec00fa078/base/threading/sequenced_worker_pool.cc

Comment 15 by gab@chromium.org, Sep 2 2016

Blockedon: -642473
Status: Fixed (was: Started)
Should be fixed, not sure how to verify (assuming I'll get another bug filed against me if it still triggers TSAN -- forgot to add TSAN bot to CQ..)?

Comment 16 by gab@chromium.org, Sep 2 2016

Status: Verified (was: Fixed)
ClusterFuzz reports as fixed on dupped  bug 640152  (weirdly though it considers the fixed range to be 415934:416233 which is before the above fix in 416250..?!
►

Sign in to add a comment