New issue
Advanced search Search tips

Issue 763051 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 810464
Owner: ----
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Test pair flake: TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread

Project Member Reported by scottmg@chromium.org, Sep 7 2017

Issue description

I've been soaking pairs of tests with this little script https://chromium-review.googlesource.com/c/chromium/src/+/646851 on a very slow VM.

I'm not sure if this one is the task scheduler or misuse of the api in the timer sequence test yet.

scottmg@humid:/usr/local/google/cr-perry/src$ cat out/asan/args.gn
is_asan = true
enable_nacl = false
is_debug = false
use_goma = true

scottmg@humid:/usr/local/google/cr-perry/src$ out/asan/base_unittests --gtest_filter=TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread --gtest_shuffle --gtest_repeat=100 --gtest_break_on_failure 2>&1 | tools/valgrind/asan/asan_symbolize.py
IMPORTANT DEBUGGING NOTE: batches of tests are run inside their
own process. For debugging a test inside a debugger, use the
--gtest_filter=<your_test_name> flag along with
--single-process-tests.
Using sharding settings from environment. This is shard 0/1
Using 1 parallel jobs.
Note: Google Test filter = TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread
Note: Randomizing tests' orders with a seed of 20486 .
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from TimerSequenceTest
[ RUN      ] TimerSequenceTest.OneShotTimerTaskOnPoolThread
[       OK ] TimerSequenceTest.OneShotTimerTaskOnPoolThread (4 ms)
[----------] 1 test from TimerSequenceTest (4 ms total)

[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest
[ RUN      ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed
[       OK ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (16 ms)
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest (16 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (20 ms total)
[  PASSED  ] 2 tests.
[1/2] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (16 ms)
[2/2] TimerSequenceTest.OneShotTimerTaskOnPoolThread (4 ms)
SUCCESS: all tests passed.
Note: Google Test filter = TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread
Note: Randomizing tests' orders with a seed of 20664 .
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from TimerSequenceTest
[ RUN      ] TimerSequenceTest.OneShotTimerTaskOnPoolThread
[       OK ] TimerSequenceTest.OneShotTimerTaskOnPoolThread (13 ms)
[----------] 1 test from TimerSequenceTest (13 ms total)

[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest
[ RUN      ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed
[       OK ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (58 ms)
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest (59 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (73 ms total)
[  PASSED  ] 2 tests.
[1/2] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (58 ms)
[2/2] TimerSequenceTest.OneShotTimerTaskOnPoolThread (13 ms)
SUCCESS: all tests passed.
Note: Google Test filter = TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread
Note: Randomizing tests' orders with a seed of 20846 .
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest
[ RUN      ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed
[       OK ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (23 ms)
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest (23 ms total)

[----------] 1 test from TimerSequenceTest
[ RUN      ] TimerSequenceTest.OneShotTimerTaskOnPoolThread
[       OK ] TimerSequenceTest.OneShotTimerTaskOnPoolThread (2 ms)
[----------] 1 test from TimerSequenceTest (2 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (27 ms total)
[  PASSED  ] 2 tests.
[1/2] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (23 ms)
[2/2] TimerSequenceTest.OneShotTimerTaskOnPoolThread (2 ms)
SUCCESS: all tests passed.
Note: Google Test filter = TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread
Note: Randomizing tests' orders with a seed of 21009 .
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest
[ RUN      ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed
[       OK ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (11 ms)
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest (11 ms total)

[----------] 1 test from TimerSequenceTest
[ RUN      ] TimerSequenceTest.OneShotTimerTaskOnPoolThread
[       OK ] TimerSequenceTest.OneShotTimerTaskOnPoolThread (2 ms)
[----------] 1 test from TimerSequenceTest (2 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test cases ran. (13 ms total)
[  PASSED  ] 2 tests.
[1/2] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (11 ms)
[2/2] TimerSequenceTest.OneShotTimerTaskOnPoolThread (2 ms)
SUCCESS: all tests passed.
Note: Google Test filter = TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread
Note: Randomizing tests' orders with a seed of 21160 .
[==========] Running 2 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest
[ RUN      ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed
[       OK ] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (22 ms)
[----------] 1 test from TaskSchedulerSingleThreadTaskRunnerManagerTest (22 ms total)

[----------] 1 test from TimerSequenceTest
[ RUN      ] TimerSequenceTest.OneShotTimerTaskOnPoolThread
=================================================================
==2172==ERROR: AddressSanitizer: heap-use-after-free on address 0x6150000055a8 at pc 0x000002432764 bp 0x7fdaa4875d10 sp 0x7fdaa4875d08
WRITE of size 4 at 0x6150000055a8 thread T3 (TaskSchedulerSi)
    #0 0x2432763 in fetch_add /usr/local/google/cr-perry/src/out/asan/../../buildtools/third_party/libc++/trunk/include/atomic:1014:17
    #1 0x2432763 in Barrier_AtomicIncrement /usr/local/google/cr-perry/src/out/asan/../../base/atomicops_internals_portable.h:91:0
    #2 0x2432763 in DecrementNumPendingUndelayedTasks /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/task_tracker.cc:521:0
    #3 0x2432763 in base::internal::TaskTracker::RunNextTask(base::internal::Sequence*) /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/task_tracker.cc:256:0
    #4 0x24245e0 in base::internal::SchedulerWorker::Thread::ThreadMain() /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_worker.cc:73:34
    #5 0x244e2e4 in base::(anonymous namespace)::ThreadFunc(void*) /usr/local/google/cr-perry/src/out/asan/../../base/threading/platform_thread_posix.cc:75:13
    #6 0x7fdaad4d3183 in start_thread /build/eglibc-SvCtMH/eglibc-2.19/nptl/pthread_create.c:312:0

0x6150000055a8 is located 40 bytes inside of 472-byte region [0x615000005580,0x615000005758)
freed by thread T0 here:
    #0 0x69edd2 in operator delete(void*) ??:0:0
    #1 0x228f9f8 in DeleteSelf_ /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/gtest.h:453:24
    #2 0x228f9f8 in HandleExceptionsInMethodIfSupported<testing::Test, void> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:0
    #3 0x228f9f8 in testing::TestInfo::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2659:0
    #4 0x2290c76 in testing::TestCase::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2772:28
    #5 0x22a6d96 in testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4677:43
    #6 0x22a6308 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:12
    #7 0x22a6308 in testing::UnitTest::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4285:0
    #8 0x2556892 in RUN_ALL_TESTS /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46
    #9 0x2556892 in base::TestSuite::Run() /usr/local/google/cr-perry/src/out/asan/../../base/test/test_suite.cc:270:0
    #10 0x2573f64 in Run /usr/local/google/cr-perry/src/out/asan/../../base/callback.h:92:12
    #11 0x2573f64 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::RepeatingCallback<int ()> const&, unsigned long, int, bool, base::RepeatingCallback<void ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:216:0
    #12 0x2573b66 in base::LaunchUnitTests(int, char**, base::RepeatingCallback<int ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:475:10
    #13 0x252ca3c in main /usr/local/google/cr-perry/src/out/asan/../../base/test/run_all_base_unittests.cc:12:10
    #14 0x7fdaac0a0f44 in __libc_start_main /build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:287:0

previously allocated by thread T0 here:
    #0 0x69e1b2 in operator new(unsigned long) ??:0:0
    #1 0x1b434ff in testing::internal::TestFactoryImpl<base::internal::TaskSchedulerSingleThreadTaskRunnerManagerTest_DifferentThreadsUsed_Test>::CreateTest() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/internal/gtest-internal.h:484:39
    #2 0x228f7ee in HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test *> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:12
    #3 0x228f7ee in testing::TestInfo::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2645:0
    #4 0x2290c76 in testing::TestCase::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2772:28
    #5 0x22a6d96 in testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4677:43
    #6 0x22a6308 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:12
    #7 0x22a6308 in testing::UnitTest::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4285:0
    #8 0x2556892 in RUN_ALL_TESTS /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46
    #9 0x2556892 in base::TestSuite::Run() /usr/local/google/cr-perry/src/out/asan/../../base/test/test_suite.cc:270:0
    #10 0x2573f64 in Run /usr/local/google/cr-perry/src/out/asan/../../base/callback.h:92:12
    #11 0x2573f64 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::RepeatingCallback<int ()> const&, unsigned long, int, bool, base::RepeatingCallback<void ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:216:0
    #12 0x2573b66 in base::LaunchUnitTests(int, char**, base::RepeatingCallback<int ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:475:10
    #13 0x252ca3c in main /usr/local/google/cr-perry/src/out/asan/../../base/test/run_all_base_unittests.cc:12:10
    #14 0x7fdaac0a0f44 in __libc_start_main /build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:287:0

Thread T3 (TaskSchedulerSi) created by T0 here:
    #0 0x65e7dd in __interceptor_pthread_create ??:0:0
    #1 0x244d5fa in base::(anonymous namespace)::CreateThread(unsigned long, bool, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) /usr/local/google/cr-perry/src/out/asan/../../base/threading/platform_thread_posix.cc:114:13
    #2 0x2423b31 in Initialize /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_worker.cc:118:5
    #3 0x2423b31 in base::internal::SchedulerWorker::Thread::Create(scoped_refptr<base::internal::SchedulerWorker>) /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_worker.cc:31:0
    #4 0x24236ba in base::internal::SchedulerWorker::Start() /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_worker.cc:206:13
    #5 0x241fb5e in CreateTaskRunnerWithTraitsImpl<base::internal::(anonymous namespace)::SchedulerWorkerDelegate> /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_single_thread_task_runner_manager.cc:452:13
    #6 0x241fb5e in base::internal::SchedulerSingleThreadTaskRunnerManager::CreateSingleThreadTaskRunnerWithTraits(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, base::TaskTraits const&, base::SingleThreadTaskRunnerThreadMode) /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_single_thread_task_runner_manager.cc:395:0
    #7 0x1b33cb1 in base::internal::TaskSchedulerSingleThreadTaskRunnerManagerTest_DifferentThreadsUsed_Test::TestBody() /usr/local/google/cr-perry/src/out/asan/../../base/task_scheduler/scheduler_single_thread_task_runner_manager_unittest.cc:104:13
    #8 0x228dc1c in HandleExceptionsInMethodIfSupported<testing::Test, void> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:12
    #9 0x228dc1c in testing::Test::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2472:0
    #10 0x228f8c4 in testing::TestInfo::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2654:11
    #11 0x2290c76 in testing::TestCase::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2772:28
    #12 0x22a6d96 in testing::internal::UnitTestImpl::RunAllTests() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4677:43
    #13 0x22a6308 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2456:12
    #14 0x22a6308 in testing::UnitTest::Run() /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4285:0
    #15 0x2556892 in RUN_ALL_TESTS /usr/local/google/cr-perry/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46
    #16 0x2556892 in base::TestSuite::Run() /usr/local/google/cr-perry/src/out/asan/../../base/test/test_suite.cc:270:0
    #17 0x2573f64 in Run /usr/local/google/cr-perry/src/out/asan/../../base/callback.h:92:12
    #18 0x2573f64 in base::(anonymous namespace)::LaunchUnitTestsInternal(base::RepeatingCallback<int ()> const&, unsigned long, int, bool, base::RepeatingCallback<void ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:216:0
    #19 0x2573b66 in base::LaunchUnitTests(int, char**, base::RepeatingCallback<int ()> const&) /usr/local/google/cr-perry/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:475:10
    #20 0x252ca3c in main /usr/local/google/cr-perry/src/out/asan/../../base/test/run_all_base_unittests.cc:12:10
    #21 0x7fdaac0a0f44 in __libc_start_main /build/eglibc-SvCtMH/eglibc-2.19/csu/libc-start.c:287:0

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/local/google/cr-perry/src/out/asan/base_unittests+0x2432763)
Shadow bytes around the buggy address:
  0x0c2a7fff8a60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8a70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8a80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8a90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8aa0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c2a7fff8ab0: fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8ac0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8ad0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c2a7fff8ae0: fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa
  0x0c2a7fff8af0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c2a7fff8b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2172==ABORTING
[1/2] TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed (22 ms)
[2/2] TimerSequenceTest.OneShotTimerTaskOnPoolThread (CRASHED)
1 test crashed:
    TimerSequenceTest.OneShotTimerTaskOnPoolThread (../../base/timer/timer_unittest.cc:810)
Summary of all test iterations:
1 test crashed:
    TimerSequenceTest.OneShotTimerTaskOnPoolThread (../../base/timer/timer_unittest.cc:810)
End of the summary.
Tests took 0 seconds.
scottmg@humid:/usr/local/google/cr-perry/src$
 
Summary: Test pairing flake: TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread (was: TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread failing)
Summary: Test pair flake: TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread (was: Test pairing flake: TaskSchedulerSingleThreadTaskRunnerManagerTest.DifferentThreadsUsed:TimerSequenceTest.OneShotTimerTaskOnPoolThread)

Comment 3 by gab@chromium.org, Feb 24 2018

Mergedinto: 810464
Status: Duplicate (was: Untriaged)

Sign in to add a comment