New issue
Advanced search Search tips

Issue 732018 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 753403



Sign in to add a comment

Use after free of TaskScheduler based task runners from AfterStartupTaskUtils in extension tests

Reported by dyaros...@yandex-team.ru, Jun 10 2017

Issue description

When running all unit_tests sometimes TaskRunner outlives worker_pool_.
Recently there was an attempt to address this (or similar) problem crrev.com/2893823003, but seems like solution wasn't complete.

ASAN log:
[ RUN      ] ExtensionServiceTest.LoadAllExtensionsFromDirectorySuccess
=================================================================
==53478==ERROR: AddressSanitizer: heap-use-after-free on address 0x61500000d000 at pc 0x00010cd63fea bp 0x7fff5faedd90 sp 0x7fff5faedd88
READ of size 8 at 0x61500000d000 thread T0
    #0 0x10cd63fe9 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta) scheduler_worker_pool_impl.cc:64
    #1 0x10d3c7bc5 in (anonymous namespace)::ScheduleTask(std::__1::unique_ptr<(anonymous namespace)::AfterStartupTask, std::__1::default_delete<(anonymous namespace)::AfterStartupTask> >) after_startup_task_utils.cc:73
    #2 0x10d3c31ab in (anonymous namespace)::SetBrowserStartupIsComplete() after_startup_task_utils.cc:116
    #3 0x103214ec3 in testing::internal::TestFactoryImpl<ExtensionServiceTest_LoadAllExtensionsFromDirectorySuccess_Test>::CreateTest() extension_service_unittest.cc:530
    #4 0x1054987fe in testing::TestInfo::Run() gtest.cc:2644
    #5 0x105499c26 in testing::TestCase::Run() gtest.cc:2771
    #6 0x1054ad176 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #7 0x1054ac728 in testing::UnitTest::Run() gtest.cc:4256
    #8 0x10a70926e in base::TestSuite::Run() test_suite.cc:271
    #9 0x10a735cd7 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #10 0x10a735973 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #11 0x10a6e6001 in main run_all_unittests.cc:30
    #12 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

0x61500000d000 is located 0 bytes inside of 512-byte region [0x61500000d000,0x61500000d200)
freed by thread T0 here:
    #0 0x1288e38e2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x618e2)
    #1 0x10cd6c84e in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() memory:2272
    #2 0x10cd6ca4d in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() task_scheduler_impl.cc:43
    #3 0x10cd6bb66 in base::TaskScheduler::SetInstance(std::__1::unique_ptr<base::TaskScheduler, std::__1::default_delete<base::TaskScheduler> >) task_scheduler.cc:76
    #4 0x10a6f6b6e in base::test::ScopedAsyncTaskScheduler::~ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:37
    #5 0x10aa79195 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() memory:2585
    #6 0x10307cb10 in extensions::ExtensionServiceTestBase::~ExtensionServiceTestBase() memory:2585
    #7 0x103065557 in ExtensionServiceTestSupervised_UpdateWithPermissionIncreaseApprovalNewVersion_Test::~ExtensionServiceTestSupervised_UpdateWithPermissionIncreaseApprovalNewVersion_Test() extension_service_sync_unittest.cc:1579
    #8 0x1054989f3 in testing::TestInfo::Run() gtest.h:453
    #9 0x105499c26 in testing::TestCase::Run() gtest.cc:2771
    #10 0x1054ad176 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #11 0x1054ac728 in testing::UnitTest::Run() gtest.cc:4256
    #12 0x10a70926e in base::TestSuite::Run() test_suite.cc:271
    #13 0x10a735cd7 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #14 0x10a735973 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #15 0x10a6e6001 in main run_all_unittests.cc:30
    #16 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

previously allocated by thread T0 here:
    #0 0x1288e32e2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x612e2)
    #1 0x10cd6c115 in base::internal::TaskSchedulerImpl::TaskSchedulerImpl(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::unique_ptr<base::internal::TaskTrackerPosix, std::__1::default_delete<base::internal::TaskTrackerPosix> >) ptr_util.h:56
    #2 0x10cd6ba21 in base::TaskScheduler::Create(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >) ptr_util.h:56
    #3 0x10a6f681d in base::test::ScopedAsyncTaskScheduler::ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:24
    #4 0x10aa79752 in content::TestBrowserThreadBundle::CreateThreads() test_browser_thread_bundle.cc:123
    #5 0x10aa78c62 in content::TestBrowserThreadBundle::Init() test_browser_thread_bundle.cc:109
    #6 0x10307c740 in extensions::ExtensionServiceTestBase::ExtensionServiceTestBase() extension_service_test_base.cc:83
    #7 0x103085e9f in extensions::ExtensionServiceTestWithInstall::ExtensionServiceTestWithInstall() extension_service_test_with_install.cc:42
    #8 0x1030709c5 in ExtensionServiceTestSupervised_UpdateWithPermissionIncreaseApprovalNewVersion_Test::ExtensionServiceTestSupervised_UpdateWithPermissionIncreaseApprovalNewVersion_Test() extension_service_sync_unittest.cc:183
    #9 0x10307090a in testing::internal::TestFactoryImpl<ExtensionServiceTestSupervised_UpdateWithPermissionIncreaseApprovalNewVersion_Test>::CreateTest() gtest-internal.h:484
    #10 0x1054987fe in testing::TestInfo::Run() gtest.cc:2644
    #11 0x105499c26 in testing::TestCase::Run() gtest.cc:2771
    #12 0x1054ad176 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #13 0x1054ac728 in testing::UnitTest::Run() gtest.cc:4256
    #14 0x10a70926e in base::TestSuite::Run() test_suite.cc:271
    #15 0x10a735cd7 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #16 0x10a735973 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #17 0x10a6e6001 in main run_all_unittests.cc:30
    #18 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

SUMMARY: AddressSanitizer: heap-use-after-free scheduler_worker_pool_impl.cc:64 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta)
Shadow bytes around the buggy address:
  0x1c2a000019b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000019c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000019d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000019e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000019f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c2a00001a00:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001a10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001a20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001a30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001a40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2a00001a50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==53478==ABORTING
Received signal 6
 [0x00010cbbafcc]
 [0x00010cbbad25]
 [0x7fffd7494b3a]
 [0x00012fdc2551]
 [0x7fffd7319420]
 [0x0001288fa166]
 [0x0001288f6974]
 [0x0001288dd0c7]
 [0x0001288dcb22]
 [0x0001288dd97b]
 [0x00010cd63fea]
 [0x00010d3c7bc6]
 [0x00010d3c31ac]
 [0x000103214ec4]
 [0x0001054987ff]
 [0x000105499c27]
 [0x0001054ad177]
 [0x0001054ac729]
 [0x00010a70926f]
 [0x00010a735cd8]
 [0x00010a735974]
 [0x00010a6e6002]
 [0x7fffd7285235]
[end of stack trace]
[4307/7616] ExtensionServiceTest.LoadAllExtensionsFromDirectorySuccess (CRASHED)




 

Comment 1 by sdy@chromium.org, Jun 13 2017

Components: Internals>TaskScheduler
Labels: -OS-Mac

Comment 2 by gab@chromium.org, Jun 13 2017

Ah, the problem is with AfterStartupTaskUtils. It globally retains tasks registered with it until it is signaled that startup completed.

ExtensionServiceTest* fire this event with  AfterStartupTaskUtils::SetBrowserStartupIsCompleteForTesting()

but some of the pending tasks come from previous tests in same process (whose ScopedTaskEnvironments have been destroyed and hence whose task runners are invalid) that used the extension system and generated after startup tasks without flushing AfterStartupTaskUtils (understandably since those tests aren't even aware of that happening..).

Hmmmm....

Comment 3 by gab@chromium.org, Jun 13 2017

Status: Untriaged (was: Unconfirmed)
Summary: Use after free of TaskScheduler based task runners from AfterStartupTaskUtils in extension tests (was: ASAN, problem in unit_tests, test_browser_thread_bundle.)

Comment 4 by gab@chromium.org, Jun 14 2017

Owner: gab@chromium.org
Status: Started (was: Untriaged)
Project Member

Comment 5 by bugdroid1@chromium.org, Jun 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/4a7f5acd90e6216394ddbe762656ac35685f21c5

commit 4a7f5acd90e6216394ddbe762656ac35685f21c5
Author: Gabriel Charette <gab@chromium.org>
Date: Thu Jun 15 18:17:42 2017

Make sure all ExtensionServiceTests flush their after-startup-tasks.

Fixes use-after-free in tests that previously did and could run a
previous' test's tasks after the data it refered to was gone.

R=reillyg@chromium.org

Bug:  732018 
Change-Id: Id05de63fd6061bf47682bdb5663dfbaae5ab7a44
Reviewed-on: https://chromium-review.googlesource.com/535953
Reviewed-by: Reilly Grant <reillyg@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#479758}
[modify] https://crrev.com/4a7f5acd90e6216394ddbe762656ac35685f21c5/chrome/browser/extensions/extension_service_test_base.cc
[modify] https://crrev.com/4a7f5acd90e6216394ddbe762656ac35685f21c5/chrome/browser/extensions/extension_service_unittest.cc

Comment 6 by gab@chromium.org, Jun 15 2017

Status: Fixed (was: Started)
Should be fixed.

Comment 7 Deleted

Another one.

[ RUN      ] SupervisedUserServiceExtensionTest.ExtensionManagementPolicyProviderWithoutSUInitiatedInstalls
=================================================================
==41677==ERROR: AddressSanitizer: heap-use-after-free on address 0x61500000c880 at pc 0x00010fdf0b4a bp 0x7fff5c8e8b70 sp 0x7fff5c8e8b68
READ of size 8 at 0x61500000c880 thread T0
    #0 0x10fdf0b49 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta) scheduler_worker_pool_impl.cc:64
    #1 0x11044eed5 in (anonymous namespace)::ScheduleTask(std::__1::unique_ptr<(anonymous namespace)::AfterStartupTask, std::__1::default_delete<(anonymous namespace)::AfterStartupTask> >) after_startup_task_utils.cc:73
    #2 0x11044a4bb in (anonymous namespace)::SetBrowserStartupIsComplete() after_startup_task_utils.cc:116
    #3 0x1062886f0 in extensions::ExtensionServiceTestBase::ExtensionServiceTestBase() extension_service_test_base.cc:98
    #4 0x1083c2162 in SupervisedUserServiceExtensionTestBase::SupervisedUserServiceExtensionTestBase(bool) supervised_user_service_unittest.cc:374
    #5 0x1083c20dd in testing::internal::TestFactoryImpl<SupervisedUserServiceExtensionTest_ExtensionManagementPolicyProviderWithoutSUInitiatedInstalls_Test>::CreateTest() supervised_user_service_unittest.cc:451
    #6 0x1086b1aee in testing::TestInfo::Run() gtest.cc:2644
    #7 0x1086b2f16 in testing::TestCase::Run() gtest.cc:2771
    #8 0x1086c6466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #9 0x1086c5a18 in testing::UnitTest::Run() gtest.cc:4256
    #10 0x10d85104e in base::TestSuite::Run() test_suite.cc:271
    #11 0x10d87df77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #12 0x10d87dc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #13 0x10d82dda1 in main run_all_unittests.cc:30
    #14 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

0x61500000c880 is located 0 bytes inside of 512-byte region [0x61500000c880,0x61500000ca80)
freed by thread T0 here:
    #0 0x12b8469c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x619c2)
    #1 0x10fdf93de in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() memory:2274
    #2 0x10fdf95dd in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() task_scheduler_impl.cc:43
    #3 0x10fdf86f6 in base::TaskScheduler::SetInstance(std::__1::unique_ptr<base::TaskScheduler, std::__1::default_delete<base::TaskScheduler> >) task_scheduler.cc:76
    #4 0x10d83e90e in base::test::ScopedAsyncTaskScheduler::~ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:37
    #5 0x10dbbbd55 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() memory:2587
    #6 0x1083bf5dd in SupervisedUserServiceTest_ChangesIncludedSessionOnChangedSettings_Test::~SupervisedUserServiceTest_ChangesIncludedSessionOnChangedSettings_Test() supervised_user_service_unittest.cc:198
    #7 0x1086b1ce3 in testing::TestInfo::Run() gtest.h:453
    #8 0x1086b2f16 in testing::TestCase::Run() gtest.cc:2771
    #9 0x1086c6466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #10 0x1086c5a18 in testing::UnitTest::Run() gtest.cc:4256
    #11 0x10d85104e in base::TestSuite::Run() test_suite.cc:271
    #12 0x10d87df77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #13 0x10d87dc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #14 0x10d82dda1 in main run_all_unittests.cc:30
    #15 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

previously allocated by thread T0 here:
    #0 0x12b8463c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x613c2)
    #1 0x10fdf8ca5 in base::internal::TaskSchedulerImpl::TaskSchedulerImpl(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::unique_ptr<base::internal::TaskTrackerPosix, std::__1::default_delete<base::internal::TaskTrackerPosix> >) ptr_util.h:56
    #2 0x10fdf85b1 in base::TaskScheduler::Create(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >) ptr_util.h:56
    #3 0x10d83e5bd in base::test::ScopedAsyncTaskScheduler::ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:24
    #4 0x10dbbc312 in content::TestBrowserThreadBundle::CreateThreads() test_browser_thread_bundle.cc:123
    #5 0x10dbbb822 in content::TestBrowserThreadBundle::Init() test_browser_thread_bundle.cc:109
    #6 0x1083c0840 in testing::internal::TestFactoryImpl<SupervisedUserServiceTest_ChangesIncludedSessionOnChangedSettings_Test>::CreateTest() supervised_user_service_unittest.cc:185
    #7 0x1086b1aee in testing::TestInfo::Run() gtest.cc:2644
    #8 0x1086b2f16 in testing::TestCase::Run() gtest.cc:2771
    #9 0x1086c6466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #10 0x1086c5a18 in testing::UnitTest::Run() gtest.cc:4256
    #11 0x10d85104e in base::TestSuite::Run() test_suite.cc:271
    #12 0x10d87df77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #13 0x10d87dc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #14 0x10d82dda1 in main run_all_unittests.cc:30
    #15 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

SUMMARY: AddressSanitizer: heap-use-after-free scheduler_worker_pool_impl.cc:64 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta)
Shadow bytes around the buggy address:
  0x1c2a000018c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c2a00001910:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001920: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001930: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001940: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2a00001960: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41677==ABORTING
Received signal 6
 [0x00010fc4857c]
 [0x00010fc482d5]
 [0x7fffd7494b3a]
 [0x000000000003]
 [0x7fffd7319420]
 [0x00012b85a866]
 [0x00012b859894]
 [0x00012b83ffb7]
 [0x00012b83fa12]
 [0x00012b84086b]
 [0x00010fdf0b4a]
 [0x00011044eed6]
 [0x00011044a4bc]
 [0x0001062886f1]
 [0x0001083c2163]
 [0x0001083c20de]
 [0x0001086b1aef]
 [0x0001086b2f17]
 [0x0001086c6467]
 [0x0001086c5a19]
 [0x00010d85104f]
 [0x00010d87df78]
 [0x00010d87dc04]
 [0x00010d82dda2]
 [0x7fffd7285235]
[end of stack trace]

And another one.

[4175/7610] ExtensionProtocolsTest.VerificationSeenForZeroByteFile (100 ms)
[ RUN      ] ExtensionReenablerUnitTest.TestReenablingDisabledExtension
=================================================================
==41360==ERROR: AddressSanitizer: heap-use-after-free on address 0x61500000c880 at pc 0x000115384b4a bp 0x7fff57354bd0 sp 0x7fff57354bc8
READ of size 8 at 0x61500000c880 thread T0
    #0 0x115384b49 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta) scheduler_worker_pool_impl.cc:64
    #1 0x1159e2ed5 in (anonymous namespace)::ScheduleTask(std::__1::unique_ptr<(anonymous namespace)::AfterStartupTask, std::__1::default_delete<(anonymous namespace)::AfterStartupTask> >) after_startup_task_utils.cc:73
    #2 0x1159de4bb in (anonymous namespace)::SetBrowserStartupIsComplete() after_startup_task_utils.cc:116
    #3 0x10b81c6f0 in extensions::ExtensionServiceTestBase::ExtensionServiceTestBase() extension_service_test_base.cc:98
    #4 0x10b781c65 in testing::internal::TestFactoryImpl<extensions::ExtensionReenablerUnitTest_TestReenablingDisabledExtension_Test>::CreateTest() extension_reenabler_unittest.cc:104
    #5 0x10dc45aee in testing::TestInfo::Run() gtest.cc:2644
    #6 0x10dc46f16 in testing::TestCase::Run() gtest.cc:2771
    #7 0x10dc5a466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #8 0x10dc59a18 in testing::UnitTest::Run() gtest.cc:4256
    #9 0x112de504e in base::TestSuite::Run() test_suite.cc:271
    #10 0x112e11f77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #11 0x112e11c03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #12 0x112dc1da1 in main run_all_unittests.cc:30
    #13 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

0x61500000c880 is located 0 bytes inside of 512-byte region [0x61500000c880,0x61500000ca80)
freed by thread T0 here:
    #0 0x130ddc9c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x619c2)
    #1 0x11538d3de in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() memory:2274
    #2 0x11538d5dd in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() task_scheduler_impl.cc:43
    #3 0x11538c6f6 in base::TaskScheduler::SetInstance(std::__1::unique_ptr<base::TaskScheduler, std::__1::default_delete<base::TaskScheduler> >) task_scheduler.cc:76
    #4 0x112dd290e in base::test::ScopedAsyncTaskScheduler::~ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:37
    #5 0x11314fd55 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() memory:2587
    #6 0x10b77aed2 in extensions::ExtensionProtocolsTest::~ExtensionProtocolsTest() extension_protocols_unittest.cc:181
    #7 0x10b779b3d in extensions::ExtensionProtocolsTest_AllowFrameRequests_Test::~ExtensionProtocolsTest_AllowFrameRequests_Test() extension_protocols_unittest.cc:433
    #8 0x10dc45ce3 in testing::TestInfo::Run() gtest.h:453
    #9 0x10dc46f16 in testing::TestCase::Run() gtest.cc:2771
    #10 0x10dc5a466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #11 0x10dc59a18 in testing::UnitTest::Run() gtest.cc:4256
    #12 0x112de504e in base::TestSuite::Run() test_suite.cc:271
    #13 0x112e11f77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #14 0x112e11c03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #15 0x112dc1da1 in main run_all_unittests.cc:30
    #16 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

previously allocated by thread T0 here:
    #0 0x130ddc3c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x613c2)
    #1 0x11538cca5 in base::internal::TaskSchedulerImpl::TaskSchedulerImpl(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::unique_ptr<base::internal::TaskTrackerPosix, std::__1::default_delete<base::internal::TaskTrackerPosix> >) ptr_util.h:56
    #2 0x11538c5b1 in base::TaskScheduler::Create(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >) ptr_util.h:56
    #3 0x112dd25bd in base::test::ScopedAsyncTaskScheduler::ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:24
    #4 0x113150312 in content::TestBrowserThreadBundle::CreateThreads() test_browser_thread_bundle.cc:123
    #5 0x11314f822 in content::TestBrowserThreadBundle::Init() test_browser_thread_bundle.cc:109
    #6 0x10b77a034 in testing::internal::TestFactoryImpl<extensions::ExtensionProtocolsTest_AllowFrameRequests_Test>::CreateTest() extension_protocols_unittest.cc:184
    #7 0x10dc45aee in testing::TestInfo::Run() gtest.cc:2644
    #8 0x10dc46f16 in testing::TestCase::Run() gtest.cc:2771
    #9 0x10dc5a466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #10 0x10dc59a18 in testing::UnitTest::Run() gtest.cc:4256
    #11 0x112de504e in base::TestSuite::Run() test_suite.cc:271
    #12 0x112e11f77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #13 0x112e11c03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #14 0x112dc1da1 in main run_all_unittests.cc:30
    #15 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

SUMMARY: AddressSanitizer: heap-use-after-free scheduler_worker_pool_impl.cc:64 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta)
Shadow bytes around the buggy address:
  0x1c2a000018c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a000018f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c2a00001910:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001920: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001930: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001940: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00001950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2a00001960: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41360==ABORTING
Received signal 6
 [0x0001151dc57c]
 [0x0001151dc2d5]
 [0x7fffd7494b3a]
 [0x000133fed551]
 [0x7fffd7319420]
 [0x000130df0866]
 [0x000130def894]
 [0x000130dd5fb7]
 [0x000130dd5a12]
 [0x000130dd686b]
 [0x000115384b4a]
 [0x0001159e2ed6]
 [0x0001159de4bc]
 [0x00010b81c6f1]
 [0x00010b781c66]
 [0x00010dc45aef]
 [0x00010dc46f17]
 [0x00010dc5a467]
 [0x00010dc59a19]
 [0x000112de504f]
 [0x000112e11f78]
 [0x000112e11c04]
 [0x000112dc1da2]
 [0x7fffd7285235]
[end of stack trace]

[ RUN      ] ExtensionMigratorTest.NoExistingOld
=================================================================
==41357==ERROR: AddressSanitizer: heap-use-after-free on address 0x6150004be680 at pc 0x000110940b4a bp 0x7fff5bd98bd0 sp 0x7fff5bd98bc8
READ of size 8 at 0x6150004be680 thread T0
    #0 0x110940b49 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta) scheduler_worker_pool_impl.cc:64
    #1 0x110f9eed5 in (anonymous namespace)::ScheduleTask(std::__1::unique_ptr<(anonymous namespace)::AfterStartupTask, std::__1::default_delete<(anonymous namespace)::AfterStartupTask> >) after_startup_task_utils.cc:73
    #2 0x110f9a4bb in (anonymous namespace)::SetBrowserStartupIsComplete() after_startup_task_utils.cc:116
    #3 0x106dd86f0 in extensions::ExtensionServiceTestBase::ExtensionServiceTestBase() extension_service_test_base.cc:98
    #4 0x106ce2cea in testing::internal::TestFactoryImpl<extensions::ExtensionMigratorTest_NoExistingOld_Test>::CreateTest() extension_migrator_unittest.cc:75
    #5 0x109201aee in testing::TestInfo::Run() gtest.cc:2644
    #6 0x109202f16 in testing::TestCase::Run() gtest.cc:2771
    #7 0x109216466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #8 0x109215a18 in testing::UnitTest::Run() gtest.cc:4256
    #9 0x10e3a104e in base::TestSuite::Run() test_suite.cc:271
    #10 0x10e3cdf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #11 0x10e3cdc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #12 0x10e37dda1 in main run_all_unittests.cc:30
    #13 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

0x6150004be680 is located 0 bytes inside of 512-byte region [0x6150004be680,0x6150004be880)
freed by thread T0 here:
    #0 0x12c3949c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x619c2)
    #1 0x1109493de in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() memory:2274
    #2 0x1109495dd in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() task_scheduler_impl.cc:43
    #3 0x1109486f6 in base::TaskScheduler::SetInstance(std::__1::unique_ptr<base::TaskScheduler, std::__1::default_delete<base::TaskScheduler> >) task_scheduler.cc:76
    #4 0x10e38e90e in base::test::ScopedAsyncTaskScheduler::~ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:37
    #5 0x10e70bd55 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() memory:2587
    #6 0x105f055df in BrowserWithTestWindowTest::~BrowserWithTestWindowTest() browser_with_test_window_test.cc:58
    #7 0x106cd9c89 in extensions::ExtensionMessageBubbleTest_TestShouldShowMethod_Test::~ExtensionMessageBubbleTest_TestShouldShowMethod_Test() extension_message_bubble_controller_unittest.cc:331
    #8 0x109201ce3 in testing::TestInfo::Run() gtest.h:453
    #9 0x109202f16 in testing::TestCase::Run() gtest.cc:2771
    #10 0x109216466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #11 0x109215a18 in testing::UnitTest::Run() gtest.cc:4256
    #12 0x10e3a104e in base::TestSuite::Run() test_suite.cc:271
    #13 0x10e3cdf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #14 0x10e3cdc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #15 0x10e37dda1 in main run_all_unittests.cc:30
    #16 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

previously allocated by thread T0 here:
    #0 0x12c3943c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x613c2)
    #1 0x110948ca5 in base::internal::TaskSchedulerImpl::TaskSchedulerImpl(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::unique_ptr<base::internal::TaskTrackerPosix, std::__1::default_delete<base::internal::TaskTrackerPosix> >) ptr_util.h:56
    #2 0x1109485b1 in base::TaskScheduler::Create(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >) ptr_util.h:56
    #3 0x10e38e5bd in base::test::ScopedAsyncTaskScheduler::ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:24
    #4 0x10e70c312 in content::TestBrowserThreadBundle::CreateThreads() test_browser_thread_bundle.cc:123
    #5 0x10e70b822 in content::TestBrowserThreadBundle::Init() test_browser_thread_bundle.cc:109
    #6 0x105f053e1 in BrowserWithTestWindowTest::BrowserWithTestWindowTest() browser_with_test_window_test.cc:48
    #7 0x106cdb565 in testing::internal::TestFactoryImpl<extensions::ExtensionMessageBubbleTest_TestShouldShowMethod_Test>::CreateTest() extension_message_bubble_controller_unittest.cc:163
    #8 0x109201aee in testing::TestInfo::Run() gtest.cc:2644
    #9 0x109202f16 in testing::TestCase::Run() gtest.cc:2771
    #10 0x109216466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #11 0x109215a18 in testing::UnitTest::Run() gtest.cc:4256
    #12 0x10e3a104e in base::TestSuite::Run() test_suite.cc:271
    #13 0x10e3cdf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #14 0x10e3cdc03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #15 0x10e37dda1 in main run_all_unittests.cc:30
    #16 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

SUMMARY: AddressSanitizer: heap-use-after-free scheduler_worker_pool_impl.cc:64 in base::internal::(anonymous namespace)::SchedulerParallelTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta)
Shadow bytes around the buggy address:
  0x1c2a00097c80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097c90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097ca0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097cb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097cc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c2a00097cd0:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097ce0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097cf0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097d00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00097d10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2a00097d20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==41357==ABORTING
Received signal 6
 [0x00011079857c]
 [0x0001107982d5]
 [0x7fffd7494b3a]
 [0x000135e0d551]
 [0x7fffd7319420]
 [0x00012c3a8866]
 [0x00012c3a7894]
 [0x00012c38dfb7]
 [0x00012c38da12]
 [0x00012c38e86b]
 [0x000110940b4a]
 [0x000110f9eed6]
 [0x000110f9a4bc]
 [0x000106dd86f1]
 [0x000106ce2ceb]
 [0x000109201aef]
 [0x000109202f17]
 [0x000109216467]
 [0x000109215a19]
 [0x00010e3a104f]
 [0x00010e3cdf78]
 [0x00010e3cdc04]
 [0x00010e37dda2]
 [0x7fffd7285235]
[end of stack trace]


And these too:
BrowserActionUnitTest.MultiIcons, ManagementApiUnitTest.ManagementSetEnabled, ExtensionMigratorTest.NoExistingOld, RequestContentScriptTest.MissingScripts, ComponentToolbarActionsFactoryTest.UnloadMigratedExtensions, RecentTabsSubMenuModelTest.MaxWidthNoDevices, ExtensionMessageBubbleBridgeUnitTest.TestGetExtraViewInfoMethodWithNormalSettingsOverrideExtension, 
Similar crash in another group of tests (ExternalProtocolHandlerTest):

Tests: ExternalProtocolHandlerTest.TestLaunchSchemeUnknownChromeDefault, ExternalProtocolHandlerTest.TestLaunchSchemeUnBlockedChromeNotDefault

[ RUN      ] ExternalProtocolHandlerTest.TestLaunchSchemeUnBlockedChromeNotDefault
=================================================================
==40974==ERROR: AddressSanitizer: heap-use-after-free on address 0x6150000af300 at pc 0x00011773df29 bp 0x7fff54f9b210 sp 0x7fff54f9b208
READ of size 8 at 0x6150000af300 thread T0
    #0 0x11773df28 in base::internal::(anonymous namespace)::SchedulerSequencedTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta) scheduler_worker_pool_impl.cc:103
    #1 0x1177268a7 in base::TaskRunner::PostTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>) task_runner.cc:47
    #2 0x11849e4f1 in shell_integration::DefaultWebClientWorker::StartCheckIsDefault() shell_integration.cc:140
    #3 0x117ffa257 in ExternalProtocolHandler::LaunchUrlWithDelegate(GURL const&, int, int, ui::PageTransition, bool, ExternalProtocolHandler::Delegate*) external_protocol_handler.cc:252
    #4 0x10b43385c in ExternalProtocolHandlerTest::DoTest(ExternalProtocolHandler::BlockState, shell_integration::DefaultWebClientState, bool, bool, bool) external_protocol_handler_unittest.cc:143
    #5 0x10fffcd30 in testing::Test::Run() gtest.cc:2471
    #6 0x10fffebc3 in testing::TestInfo::Run() gtest.cc:2653
    #7 0x10fffff16 in testing::TestCase::Run() gtest.cc:2771
    #8 0x110013466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #9 0x110012a18 in testing::UnitTest::Run() gtest.cc:4256
    #10 0x11519e04e in base::TestSuite::Run() test_suite.cc:271
    #11 0x1151caf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #12 0x1151cac03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #13 0x11517ada1 in main run_all_unittests.cc:30
    #14 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

0x6150000af300 is located 0 bytes inside of 512-byte region [0x6150000af300,0x6150000af500)
freed by thread T0 here:
    #0 0x1331949c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x619c2)
    #1 0x11774632e in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() memory:2274
    #2 0x1177465dd in base::internal::TaskSchedulerImpl::~TaskSchedulerImpl() task_scheduler_impl.cc:43
    #3 0x1177456f6 in base::TaskScheduler::SetInstance(std::__1::unique_ptr<base::TaskScheduler, std::__1::default_delete<base::TaskScheduler> >) task_scheduler.cc:76
    #4 0x11518b90e in base::test::ScopedAsyncTaskScheduler::~ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:37
    #5 0x115508d55 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() memory:2587
    #6 0x10b43db1a in ExternalProtocolHandlerTest_TestLaunchSchemeUnBlockedChromeDefault_Test::~ExternalProtocolHandlerTest_TestLaunchSchemeUnBlockedChromeDefault_Test() external_protocol_handler_unittest.cc:111
    #7 0x10fffece3 in testing::TestInfo::Run() gtest.h:453
    #8 0x10fffff16 in testing::TestCase::Run() gtest.cc:2771
    #9 0x110013466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #10 0x110012a18 in testing::UnitTest::Run() gtest.cc:4256
    #11 0x11519e04e in base::TestSuite::Run() test_suite.cc:271
    #12 0x1151caf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #13 0x1151cac03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #14 0x11517ada1 in main run_all_unittests.cc:30
    #15 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

previously allocated by thread T0 here:
    #0 0x1331943c2  (libclang_rt.asan_osx_dynamic.dylib:x86_64+0x613c2)
    #1 0x117745e74 in base::internal::TaskSchedulerImpl::TaskSchedulerImpl(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::unique_ptr<base::internal::TaskTrackerPosix, std::__1::default_delete<base::internal::TaskTrackerPosix> >) ptr_util.h:56
    #2 0x1177455b1 in base::TaskScheduler::Create(base::BasicStringPiece<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >) ptr_util.h:56
    #3 0x11518b5bd in base::test::ScopedAsyncTaskScheduler::ScopedAsyncTaskScheduler() scoped_async_task_scheduler.cc:24
    #4 0x115509312 in content::TestBrowserThreadBundle::CreateThreads() test_browser_thread_bundle.cc:123
    #5 0x115508822 in content::TestBrowserThreadBundle::Init() test_browser_thread_bundle.cc:109
    #6 0x10b442514 in testing::internal::TestFactoryImpl<ExternalProtocolHandlerTest_TestLaunchSchemeUnBlockedChromeDefault_Test>::CreateTest() external_protocol_handler_unittest.cc:114
    #7 0x10fffeaee in testing::TestInfo::Run() gtest.cc:2644
    #8 0x10fffff16 in testing::TestCase::Run() gtest.cc:2771
    #9 0x110013466 in testing::internal::UnitTestImpl::RunAllTests() gtest.cc:4648
    #10 0x110012a18 in testing::UnitTest::Run() gtest.cc:4256
    #11 0x11519e04e in base::TestSuite::Run() test_suite.cc:271
    #12 0x1151caf77 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, int, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) callback.h:80
    #13 0x1151cac03 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) unit_test_launcher.cc:458
    #14 0x11517ada1 in main run_all_unittests.cc:30
    #15 0x7fffd7285234 in start (libdyld.dylib:x86_64+0x5234)

SUMMARY: AddressSanitizer: heap-use-after-free scheduler_worker_pool_impl.cc:103 in base::internal::(anonymous namespace)::SchedulerSequencedTaskRunner::PostDelayedTask(tracked_objects::Location const&, base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>, base::TimeDelta)
Shadow bytes around the buggy address:
  0x1c2a00015e10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c2a00015e60:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015e90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x1c2a00015ea0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c2a00015eb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==40974==ABORTING
Received signal 6
 [0x00011759557c]
 [0x0001175952d5]
 [0x7fffd7494b3a]
 [0x0001362d9551]
 [0x7fffd7319420]
 [0x0001331a8866]
 [0x0001331a7894]
 [0x00013318dfb7]
 [0x00013318da12]
 [0x00013318e86b]
 [0x00011773df29]
 [0x0001177268a8]
 [0x00011849e4f2]
 [0x000117ffa258]
 [0x00010b43385d]
 [0x00010fffcd31]
 [0x00010fffebc4]
 [0x00010fffff17]
 [0x000110013467]
 [0x000110012a19]
 [0x00011519e04f]
 [0x0001151caf78]
 [0x0001151cac04]
 [0x00011517ada2]
 [0x7fffd7285235]
[end of stack trace]

@gab, last crashes encountered after your fix.

Comment 14 by gab@chromium.org, Jun 16 2017

Status: Started (was: Fixed)
Okay, was secretly hoping this was local to the extensions tests, I'll come up with a global fix.
@gab

I think, it might be to certain extent. I mean, there are 2 groups of tests that fail:
ExternalProtocolHandlerTest* and inheriting from ExtensionServiceTestBase.
Though, it seems like SchedulerWorkerPoolImpl should be able to deal with this.

Can't you ran and test it? If you need how to build with asan, check here:
https://www.chromium.org/developers/testing/addresssanitizer

Generally, just pass gn args:
is_asan = true
enable_nacl = false  # Necessary until NaCl GN build is more complete.
is_debug = false  # Release build.
dcheck_always_on = true

Running all unit tests takes just a few minutes.

Comment 16 by gab@chromium.org, Jun 16 2017

There's also ExtensionProtocolsTest* and I haven't looked further, but the core issue is that it's possible for one test to PostAfterStartupTask() and for that task to remain in the global queue and only be flushed by a call to SetBrowserStartupIsCompleteForTesting() in another test (after the destination, i.e. the previous test, is gone).

The solution is that TestBrowserThreadBundle needs to flush these tasks in the scope of each test. That's what I was going to do at first, until I realized it required plumbing from content/ to chrome/ and tried an easier route (which doesn't work as you've pointed out).

(I'm on Windows and won't bother spinning up a Linux ASAN build for this, I know precisely what the right fix is)

Comment 17 by gab@chromium.org, Aug 9 2017

Blocking: 753403

Comment 18 by gab@chromium.org, Aug 9 2017

I have a fix @ https://chromium-review.googlesource.com/c/538898/ which now passes everything but one io_thread_unittest (investigating).
Project Member

Comment 19 by bugdroid1@chromium.org, Aug 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d

commit c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d
Author: Gabriel Charette <gab@chromium.org>
Date: Thu Aug 10 18:21:49 2017

Always run after-startup-tasks in the scope of the unit test that posted them.

Bug:  732018 ,  753403 
Change-Id: I318e82f3aa2969d6e338a33905ec7cd256d32d9c
Reviewed-on: https://chromium-review.googlesource.com/538898
Reviewed-by: Avi Drissman <avi@chromium.org>
Reviewed-by: Reilly Grant <reillyg@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Charlie Harrison <csharrison@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#493468}
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/chrome/browser/chrome_content_browser_client.cc
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/chrome/browser/chrome_content_browser_client.h
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/chrome/browser/extensions/extension_service_test_base.cc
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/chrome/browser/io_thread_unittest.cc
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/chrome/browser/subresource_filter/subresource_filter_test_harness.cc
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/browser/BUILD.gn
[add] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/browser/after_startup_task_utils.cc
[add] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/browser/after_startup_task_utils.h
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/public/browser/content_browser_client.cc
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/public/browser/content_browser_client.h
[modify] https://crrev.com/c8cca9965f9ad583cad39ecd9fcd16e9dcea5e7d/content/public/test/test_browser_thread_bundle.cc

Comment 20 by gab@chromium.org, Aug 10 2017

Status: Fixed (was: Started)
Should be fixed now :)
@gab - I ran tests, they don't fail anymore. Hoorray!

Comment 22 by gab@chromium.org, Aug 16 2017

Awesome, thanks for verifying!

Sign in to add a comment