MemoryDumpSchedulerTest tests not synchronizing with MessageLoop shutdown |
|||||||
Issue descriptionLocal run, nothing else useful in the output, other than it timed out after 900s. [01044.156] 02619.02663> 1 test timed out: [01044.156] 02619.02663> MemoryDumpSchedulerTest.SingleTrigger (../../base/trace_event/memory_dump_scheduler_unittest.cc:57) See also crbug.com/751350 , the test setup is quite similar so perhaps there's something slightly off there.
,
Aug 2 2017
Here's a repro on plain Linux ASan with a couple sleeps added per https://chromium-review.googlesource.com/598928, and matches the explanation above, I think. I guess there must be some way to block for the main Sequence/MessageLoop/whatever-we-call-it-these-days to complete before the test exits. (?) [fuchsia-mdst-repro] sgraham@river:/work/cr/src$ ninja -C out/asan base_unittests -j1000 && out/asan/base_unittests --gtest_filter=MemoryDumpSchedulerTest.SingleTrigger 2>&1 | tools/valgrind/asan/asan_symbolize.py ninja: Entering directory `out/asan' [0->2/2 ~1] LINK ./base_unittests IMPORTANT DEBUGGING NOTE: batches of tests are run inside their own process. For debugging a test inside a debugger, use the --gtest_filter=<your_test_name> flag along with --single-process-tests. Using sharding settings from environment. This is shard 0/1 Using 1 parallel jobs. Note: Google Test filter = MemoryDumpSchedulerTest.SingleTrigger [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from MemoryDumpSchedulerTest [ RUN ] MemoryDumpSchedulerTest.SingleTrigger ================================================================= ==28239:28240==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f95c391fc20 at pc 0x00000235406d bp 0x7f95c262f470 sp 0x7f95c262f468 READ of size 8 at 0x7f95c391fc20 thread T1 (MemoryDumpSched) #0 0x235406c in operator-> /work/cr/src/out/asan/../../base/memory/ref_counted.h:547:12 #1 0x235406c in base::WaitableEvent::Signal() /work/cr/src/out/asan/../../base/synchronization/waitable_event_posix.cc:56:0 #2 0x1debd9c in operator() /work/cr/src/out/asan/../../base/trace_event/memory_dump_scheduler_unittest.cc:74:17 #3 0x1debd9c in Invoke<(lambda at ../../base/trace_event/memory_dump_scheduler_unittest.cc:71:11)> /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-generated-actions.h:74:0 #4 0x1debd9c in Perform<void, std::__1::tuple<base::trace_event::MemoryDumpLevelOfDetail> > /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-more-actions.h:61:0 #5 0x1debd9c in testing::PolymorphicAction<testing::internal::InvokeAction<base::trace_event::MemoryDumpSchedulerTest_SingleTrigger_Test::TestBody()::$_0> >::MonomorphicImpl<void (base::trace_event::MemoryDumpLevelOfDetail)>::Perform(std::__1::tuple<base::trace_event::MemoryDumpLevelOfDetail> const&) /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-actions.h:446:0 #6 0x1de5b6c in Perform /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-actions.h:395:19 #7 0x1de5b6c in testing::internal::ActionResultHolder<void>* testing::internal::ActionResultHolder<void>::PerformAction<void (base::trace_event::MemoryDumpLevelOfDetail)>(testing::Action<void (base::trace_event::MemoryDumpLevelOfDetail)> const&, testing::internal::Function<void (base::trace_event::MemoryDumpLevelOfDetail)>::ArgumentTuple const&) /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1447:0 #8 0x1de4238 in testing::internal::FunctionMockerBase<void (base::trace_event::MemoryDumpLevelOfDetail)>::UntypedPerformAction(void const*, void const*) const /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1547:12 #9 0x21f90a0 in testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith(void const*) /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/src/gmock-spec-builders.cc:410:15 #10 0x1ddbc7b in InvokeWith /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1590:40 #11 0x1ddbc7b in Invoke /work/cr/src/out/asan/../../third_party/googletest/src/googlemock/include/gmock/gmock-generated-function-mockers.h:101:0 #12 0x1ddbc7b in base::trace_event::(anonymous namespace)::CallbackWrapper::OnTick(base::trace_event::MemoryDumpLevelOfDetail) /work/cr/src/out/asan/../../base/trace_event/memory_dump_scheduler_unittest.cc:27:0 #13 0x23e11ce in Run /work/cr/src/out/asan/../../base/callback.h:80:12 #14 0x23e11ce in base::trace_event::MemoryDumpScheduler::Tick(unsigned int) /work/cr/src/out/asan/../../base/trace_event/memory_dump_scheduler.cc:99:0 #15 0x2232356 in Run /work/cr/src/out/asan/../../base/callback.h:91:12 #16 0x2232356 in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) /work/cr/src/out/asan/../../base/debug/task_annotator.cc:59:0 #17 0x22a7001 in base::MessageLoop::RunTask(base::PendingTask*) /work/cr/src/out/asan/../../base/message_loop/message_loop.cc:403:19 #18 0x22a8110 in base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) /work/cr/src/out/asan/../../base/message_loop/message_loop.cc:414:5 #19 0x22a9524 in base::MessageLoop::DoDelayedWork(base::TimeTicks*) /work/cr/src/out/asan/../../base/message_loop/message_loop.cc:561:10 #20 0x22b0312 in base::MessagePumpDefault::Run(base::MessagePump::Delegate*) /work/cr/src/out/asan/../../base/message_loop/message_pump_default.cc:37:27 #21 0x231f197 in base::RunLoop::Run() /work/cr/src/out/asan/../../base/run_loop.cc:111:14 #22 0x23b10bd in base::Thread::ThreadMain() /work/cr/src/out/asan/../../base/threading/thread.cc:338:3 #23 0x239ad94 in base::(anonymous namespace)::ThreadFunc(void*) /work/cr/src/out/asan/../../base/threading/platform_thread_posix.cc:71:13 #24 0x7f95c76556b9 in start_thread ??:0:0 Address 0x7f95c391fc20 is located in stack of thread T0 at offset 32 in frame #0 0x1ddaa5f in base::trace_event::MemoryDumpSchedulerTest_SingleTrigger_Test::TestBody() /work/cr/src/out/asan/../../base/trace_event/memory_dump_scheduler_unittest.cc:57:0 This frame has 16 object(s): [32, 40) 'evt' (line 61) <== Memory access at offset 32 is inside this variable [64, 96) 'config' (line 63) [128, 136) 'ref.tmp' (line 64) [160, 168) 'ref.tmp2' (line 65) [192, 193) 'sequence' (line 67) [208, 232) 'ref.tmp7' (line 68) [272, 296) 'ref.tmp12' (line 69) [336, 352) 'ref.tmp15' (line 69) [368, 400) 'agg.tmp26' [432, 440) 'agg.tmp27' [464, 472) 'time_ms' (line 83) [496, 504) 'ref.tmp30' (line 83) [528, 544) 'gtest_ar' (line 86) [560, 564) 'ref.tmp41' (line 86) [576, 584) 'ref.tmp43' (line 86) [608, 616) 'temp.lvalue' HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-use-after-return (/work/cr/src/out/asan/base_unittests+0x235406c) Shadow bytes around the buggy address: 0x0ff33871bf30: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bf40: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bf50: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bf60: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bf70: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 =>0x0ff33871bf80: f5 f5 f5 f5[f5]f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bf90: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bfa0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bfb0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bfc0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 0x0ff33871bfd0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Thread T1 (MemoryDumpSched) created by T0 here: #0 0x65a9ed in __interceptor_pthread_create ??:0:0 #1 0x239a0aa in base::(anonymous namespace)::CreateThread(unsigned long, bool, base::PlatformThread::Delegate*, base::PlatformThreadHandle*, base::ThreadPriority) /work/cr/src/out/asan/../../base/threading/platform_thread_posix.cc:110:13 #2 0x23afdd4 in base::Thread::StartWithOptions(base::Thread::Options const&) /work/cr/src/out/asan/../../base/threading/thread.cc:112:15 #3 0x23afa09 in base::Thread::Start() /work/cr/src/out/asan/../../base/threading/thread.cc:75:10 #4 0x1de377b in base::trace_event::MemoryDumpSchedulerTest::SetUp() /work/cr/src/out/asan/../../base/trace_event/memory_dump_scheduler_unittest.cc:42:17 #5 0x21c140c in HandleExceptionsInMethodIfSupported<testing::Test, void> /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2455:12 #6 0x21c140c in testing::Test::Run() /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2467:0 #7 0x21c3294 in testing::TestInfo::Run() /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2653:11 #8 0x21c4646 in testing::TestCase::Run() /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2771:28 #9 0x21da5e6 in testing::internal::UnitTestImpl::RunAllTests() /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4648:43 #10 0x21d9b57 in HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:2455:12 #11 0x21d9b57 in testing::UnitTest::Run() /work/cr/src/out/asan/../../third_party/googletest/src/googletest/src/gtest.cc:4256:0 #12 0x24a3f11 in RUN_ALL_TESTS /work/cr/src/out/asan/../../third_party/googletest/src/googletest/include/gtest/gtest.h:2237:46 #13 0x24a3f11 in base::TestSuite::Run() /work/cr/src/out/asan/../../base/test/test_suite.cc:270:0 #14 0x24c16ba in Run /work/cr/src/out/asan/../../base/callback.h:80:12 #15 0x24c16ba in base::(anonymous namespace)::LaunchUnitTestsInternal(base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&, unsigned long, int, bool, base::Callback<void (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) /work/cr/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:216:0 #16 0x24c12b6 in base::LaunchUnitTests(int, char**, base::Callback<int (), (base::internal::CopyMode)1, (base::internal::RepeatMode)1> const&) /work/cr/src/out/asan/../../base/test/launcher/unit_test_launcher.cc:462:10 #17 0x247810c in main /work/cr/src/out/asan/../../base/test/run_all_base_unittests.cc:22:10 #18 0x7f95c81b682f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291:0 ==28239:28240==ABORTING [1/1] MemoryDumpSchedulerTest.SingleTrigger (CRASHED) 1 test crashed: MemoryDumpSchedulerTest.SingleTrigger (../../base/trace_event/memory_dump_scheduler_unittest.cc:57) Tests took 5 seconds.
,
Aug 2 2017
,
Aug 2 2017
I think by inspection this will affect all of the MemoryDumpSchedulerTest tests.
,
Aug 2 2017
Thanks for tracking this down! Sorry, I was out today/yesterday, tomorrow hopefully I'll be back and can take a closer look at this.
,
Aug 2 2017
gab@ suggested RunLoop().RunUntilIdle() at the end of the tests (before |evt| goes out of scope) but also that maybe there's something that can be tweaked in the gmock setup so that this can't happen. (?) I'm not sure what the gmock changes would be though. Thanks for taking a look Hector!
,
Aug 2 2017
Hrm, I tried a RunLoop().RunUntilIdle() out of curiosity, but that crashes because there's no base::ThreadTaskRunnerHandle::Get(). But the stack in https://bugs.chromium.org/p/chromium/issues/detail?id=751350 is clearly in a MessageLoop. So I'm not sure how to wait for idle correctly. :/
,
Aug 2 2017
No ThreadTaskRunnerHandle in a MessageLoop is impossible unless it's not BindToCurrentThread() yet (highly unlikely)... Re. gmock: checkout WillOnce() Le mer. 2 août 2017 17:27, scottmg via monorail < monorail+v2.4143809049@chromium.org> a écrit :
,
Aug 2 2017
Oh, I was confused about which thread the task was running on. Doing a bg_thread_->FlushForTesting() at the end of the test body fixes the repro. I don't know if there's a better way to address this though.
,
Aug 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/447975f2f54a2fd264da3762b8f5470ad2d59da7 commit 447975f2f54a2fd264da3762b8f5470ad2d59da7 Author: Scott Graham <scottmg@chromium.org> Date: Thu Aug 03 02:14:43 2017 fuchsia: disable MemoryDumpSchedulerTest pending crbug.com/751748 TBR=wez@chromium.org Bug: 751748 , 738275 Change-Id: I775792a7441dca5fd90e410653cba4610115780a Reviewed-on: https://chromium-review.googlesource.com/599187 Commit-Queue: Scott Graham <scottmg@chromium.org> Commit-Queue: Wez <wez@chromium.org> Reviewed-by: Wez <wez@chromium.org> Reviewed-by: Scott Graham <scottmg@chromium.org> Cr-Commit-Position: refs/heads/master@{#491615} [modify] https://crrev.com/447975f2f54a2fd264da3762b8f5470ad2d59da7/testing/buildbot/filters/fuchsia.base_unittests.filter
,
Aug 3 2017
Ah yes indeed, you'll need to flush that thread. Ideally this would automatically work through RAII as ~Thread() does join prior to returning. But since the Thread is on the fixture it currently outlives the WaitableEvent. FWIW, the fixture is overkill with its use of the heap, it should just have everything on the stack and not bother to manually create and reset() things (letting member ordering do the right thing). If you move the WaitableEvent on the fixture (ahead of the Thread), you'll fix the issue as well. Le mer. 2 août 2017 22:15, bugdroid1 via monorail < monorail+v2.3275348242@chromium.org> a écrit :
,
Aug 3 2017
Did you mean something like this: https://chromium-review.googlesource.com/c/600213/1/base/trace_event/memory_dump_scheduler_unittest.cc?
,
Aug 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/cd7b37e96c0714bc27acb0e14bb7107d1f415935 commit cd7b37e96c0714bc27acb0e14bb7107d1f415935 Author: Hector Dearman <hjd@google.com> Date: Thu Aug 03 21:32:09 2017 memory-infra: Fix MemoryDumpSchedulerTest ASAN failure WaitableEvent needs to outlive Thread to avoid a use after return. Bug: 751748 Change-Id: I99181842fd81ccad71dba967deebf5d1014affc7 Reviewed-on: https://chromium-review.googlesource.com/600213 Reviewed-by: Primiano Tucci <primiano@chromium.org> Reviewed-by: Gabriel Charette <gab@chromium.org> Reviewed-by: Scott Graham <scottmg@chromium.org> Commit-Queue: Hector Dearman <hjd@chromium.org> Cr-Commit-Position: refs/heads/master@{#491851} [modify] https://crrev.com/cd7b37e96c0714bc27acb0e14bb7107d1f415935/base/trace_event/memory_dump_scheduler_unittest.cc [modify] https://crrev.com/cd7b37e96c0714bc27acb0e14bb7107d1f415935/testing/buildbot/filters/fuchsia.base_unittests.filter
,
Aug 4 2017
Just had another related failure on the Fuchsia tree: https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.fyi%2FFuchsia%2F8258%2F%2B%2Frecipes%2Fsteps%2Fbase_unittests%2F0%2Fstdout ... [00139.654] 02854.03075> [1868/2446] MemoryDumpManagerTest.SingleDumper (116 ms) [00139.654] 02854.03075> [1869/2446] MemoryDumpManagerTest.CheckMemoryDumpArgs (138 ms) [00139.654] 02854.03075> [1870/2446] MemoryDumpManagerTest.HeapProfilerSerializationState (85 ms) [00141.156] 01105.01144> <== fatal exception: process [38482] thread [38828] [00141.156] 01105.01144> <== undefined instruction, PC at 0x602563d5ae90 [00141.157] 01105.01144> CS: 0 RIP: 0x602563d5ae90 EFL: 0x202 CR2: 0 [00141.157] 01105.01144> RAX: 0xa RBX: 0x602563e1b040 RCX: 0 RDX: 0x602563e1b0cc [00141.157] 01105.01144> RSI: 0 RDI: 0x602563e1b040 RBP: 0x1922744fb30 RSP: 0x1922744f608 [00141.157] 01105.01144> R8: 0 R9: 0 R10: 0x1d R11: 0x1922744fa29 [00141.157] 01105.01144> R12: 0x1 R13: 0x2c5ae2a11220 R14: 0x2c5ae2a113b0 R15: 0x6cba784b50c8 [00141.157] 01105.01144> errc: 0 [00141.157] 01105.01144> bottom of user stack: [00141.158] 01105.01144> 0x000001922744f608: e277101f 00002c5a 00000008 00000030 |..w.Z,......0...| [00141.158] 01105.01144> 0x000001922744f618: 2744f6f0 00000192 2744f630 00000192 |..D'....0.D'....| [00141.158] 01105.01144> 0x000001922744f628: 784971c0 00006cba 785b99c0 00006cba |.qIx.l....[x.l..| [00141.158] 01105.01144> 0x000001922744f638: 2744fb30 00000192 2744f9e0 00000192 |0.D'......D'....| [00141.158] 01105.01144> 0x000001922744f648: 00000000 00000000 00000328 00000000 |........(.......| [00141.158] 01105.01144> 0x000001922744f658: 6cba785b 00000000 e2a11220 00002c5a |[x.l.... ...Z,..| [00141.158] 01105.01144> 0x000001922744f668: 2744fb30 00000192 2744f6d8 00000192 |0.D'......D'....| [00141.158] 01105.01144> 0x000001922744f678: e276eaf1 00002c5a 73eee000 00006c01 |..v.Z,.....s.l..| [00141.158] 01105.01144> 0x000001922744f688: e2605fbd 00002c5a 2744f8b1 00000192 |._`.Z,....D'....| [00141.158] 01105.01144> 0x000001922744f698: 2744f9e8 00000192 00000001 00000000 |..D'............| [00141.158] 01105.01144> 0x000001922744f6a8: e2a11220 00002c5a 2744fa28 00000192 | ...Z,..(.D'....| [00141.158] 01105.01144> 0x000001922744f6b8: 00000000 00000000 e2a111f8 00002c5a |............Z,..| [00141.158] 01105.01144> 0x000001922744f6c8: e276e9b8 00002c5a 2744fa29 00000192 |..v.Z,..).D'....| [00141.158] 01105.01144> 0x000001922744f6d8: e1eb86ea 00002c5a 2744f9e8 00000192 |....Z,....D'....| [00141.158] 01105.01144> 0x000001922744f6e8: e27a09af 00002c5a e2a5e140 00002c5a |..z.Z,..@...Z,..| [00141.159] 01105.01144> 0x000001922744f6f8: e27be3d9 00002c5a 00000000 00000000 |..{.Z,..........| [00141.159] 01105.01144> arch: x86_64 [00141.181] 01105.01144> dso: id=a00d1fa58fa66d498683b4f08ce73933dcdefd63 base=0x7b581a607000 name=<vDSO> [00141.181] 01105.01144> dso: id=e5a677de03279e111c67f6055ea442214513b237 base=0x602563d3e000 name=libc.so [00141.181] 01105.01144> dso: id=5c3659cd977cf48b base=0x2c5ae1be4000 name=app:/system/base_unittests [00141.181] 01105.01144> dso: id=788e800c7e74c2a011ad0431d596816740448a13 base=0x185a4a312000 name=libmxio.so [00141.181] 01105.01144> dso: id=6d8ac831560037a646934de1b2ab97562a5132e2 base=0xcef1d763000 name=liblaunchpad.so [00141.185] 01105.01144> bt#01: pc 0x602563d5ae90 sp 0x1922744f608 (libc.so,0x1ce90) [00141.193] 01105.01144> bt#02: pc 0x2c5ae277101f sp 0x1922744f610 (app:/system/base_unittests,0xb8d01f) [00141.196] 01105.01144> bt#03: pc 0x2c5ae27a09af sp 0x1922744f6f0 (app:/system/base_unittests,0xbbc9af) [00141.197] 01105.01144> bt#04: pc 0x2c5ae27be3d9 sp 0x1922744f700 (app:/system/base_unittests,0xbda3d9) [00141.198] 01105.01144> bt#05: pc 0x2c5ae2602da5 sp 0x1922744fb30 (app:/system/base_unittests,0xa1eda5) [00141.199] 01105.01144> bt#06: pc 0x2c5ae2846811 sp 0x1922744fb50 (app:/system/base_unittests,0xc62811) [00141.199] 01105.01144> bt#07: pc 0x2c5ae27cbd3b sp 0x1922744fbc0 (app:/system/base_unittests,0xbe7d3b) [00141.200] 01105.01144> bt#08: pc 0x2c5ae27eb6a8 sp 0x1922744fca0 (app:/system/base_unittests,0xc076a8) [00141.210] 01105.01144> bt#09: pc 0x2c5ae27ebcc2 sp 0x1922744fd60 (app:/system/base_unittests,0xc07cc2) [00141.210] 01105.01144> bt#10: pc 0x2c5ae27ec1bf sp 0x1922744fd80 (app:/system/base_unittests,0xc081bf) [00141.211] 01105.01144> bt#11: pc 0x2c5ae27ee61d sp 0x1922744fef0 (app:/system/base_unittests,0xc0a61d) [00141.213] 01105.01144> bt#12: pc 0x2c5ae280a50e sp 0x1922744ff20 (app:/system/base_unittests,0xc2650e) [00141.214] 01105.01144> bt#13: pc 0x2c5ae28380b5 sp 0x1922744ff60 (app:/system/base_unittests,0xc540b5) [00141.215] 01105.01144> bt#14: pc 0x2c5ae28312bc sp 0x1922744ffc0 (app:/system/base_unittests,0xc4d2bc) [00141.215] 01105.01144> bt#15: pc 0x602563d57a36 sp 0x1922744ffe0 (libc.so,0x19a36) [00141.223] 01105.01144> bt#16: pc 0x602563dcf44a sp 0x1922744fff0 (libc.so,0x9144a) [00141.224] 01105.01144> bt#17: pc 0 sp 0x19227450000 [00141.225] 01105.01144> bt#18: end ----- start symbolized stack #01: ?? ??:0 #02: abort_message at ??:? #03: __cxa_pure_virtual at ??:? #04: testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith(void const*) at /b/c/builder/linux/src/out/Release/../../third_party/googletest/src/googlemock/src/gmock-spec-builders.cc:376 #05: testing::internal::FunctionMockerBase<void (base::trace_event::MemoryDumpLevelOfDetail)>::InvokeWith(std::__1::tuple<base::trace_event::MemoryDumpLevelOfDetail> const&) at /b/c/builder/linux/src/out/Release/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1590 (inlined by) testing::internal::FunctionMocker<void (base::trace_event::MemoryDumpLevelOfDetail)>::Invoke(base::trace_event::MemoryDumpLevelOfDetail) at /b/c/builder/linux/src/out/Release/../../third_party/googletest/src/googlemock/include/gmock/gmock-generated-function-mockers.h:101 (inlined by) base::trace_event::(anonymous namespace)::CallbackWrapper::OnTick(base::trace_event::MemoryDumpLevelOfDetail) at /b/c/builder/linux/src/out/Release/../../base/trace_event/memory_dump_scheduler_unittest.cc:27 #06: base::Callback<void (base::trace_event::MemoryDumpLevelOfDetail), (base::internal::CopyMode)1, (base::internal::RepeatMode)1>::Run(base::trace_event::MemoryDumpLevelOfDetail) const & at /b/c/builder/linux/src/out/Release/../../base/callback.h:80 (inlined by) base::trace_event::MemoryDumpScheduler::Tick(unsigned int) at /b/c/builder/linux/src/out/Release/../../base/trace_event/memory_dump_scheduler.cc:99 #07: base::Callback<void (), (base::internal::CopyMode)0, (base::internal::RepeatMode)0>::Run() && at /b/c/builder/linux/src/out/Release/../../base/callback.h:92 (inlined by) base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) at /b/c/builder/linux/src/out/Release/../../base/debug/task_annotator.cc:59 #08: std::__1::vector<base::MessageLoop::TaskObserver*, std::__1::allocator<base::MessageLoop::TaskObserver*> >::empty() const at /b/c/builder/linux/src/out/Release/../../buildtools/third_party/libc++/trunk/include/vector:644 (inlined by) base::ObserverListBase<base::MessageLoop::TaskObserver>::begin() at /b/c/builder/linux/src/out/Release/../../base/observer_list.h:133 (inlined by) base::MessageLoop::RunTask(base::PendingTask*) at /b/c/builder/linux/src/out/Release/../../base/message_loop/message_loop.cc:404 #09: base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) at /b/c/builder/linux/src/out/Release/../../base/message_loop/message_loop.cc:414 #10: base::MessageLoop::DoDelayedWork(base::TimeTicks*) at /b/c/builder/linux/src/out/Release/../../base/message_loop/message_loop.cc:561 #11: base::MessagePumpDefault::Run(base::MessagePump::Delegate*) at /b/c/builder/linux/src/out/Release/../../base/message_loop/message_pump_default.cc:38 #12: base::RunLoop::Run() at /b/c/builder/linux/src/out/Release/../../base/run_loop.cc:112 #13: base::Lock::Acquire() at /b/c/builder/linux/src/out/Release/../../base/synchronization/lock.h:26 (inlined by) AutoLock at /b/c/builder/linux/src/out/Release/../../base/synchronization/lock.h:115 (inlined by) base::Thread::ThreadMain() at /b/c/builder/linux/src/out/Release/../../base/threading/thread.cc:341 #14: base::(anonymous namespace)::ThreadFunc(void*) at /b/c/builder/linux/src/out/Release/../../base/threading/platform_thread_posix.cc:73 #15: ?? ??:0 #16: ?? ??:0 ----- end symbolized stack [00141.280] 02854.03075> [1871/2446] MemoryDumpManagerTest.SummaryOnlyWhitelisting (67 ms) [00141.280] 02854.03075> [1872/2446] MemoryDumpManagerTest.UnregisterAndDeleteDumpProviderSoon (3 ms) [00141.290] 02854.03075> [1873/2446] MemoryDumpManagerTest.UnregisterAndDeleteDumpProviderSoonDuringDump (102 ms) [00141.290] 02854.03075> [1874/2446] MemoryDumpSchedulerTest.SingleTrigger (165 ms) [00141.290] 02854.03075> [ RUN ] MemoryDumpSchedulerTest.MultipleTriggers [00141.291] 02854.03075> [1875/2446] MemoryDumpSchedulerTest.MultipleTriggers (CRASHED) ...
,
Aug 4 2017
I'm not sure how gmock's awful magick works... Do the things in the scope of the test body have a lifetime shorter than the test fixture (where the thread lives now)?
,
Aug 11 2017
Got this one again locally. [00126.656] 01106.01145> <== fatal exception: process [41448] thread [41894] [00126.657] 01106.01145> <== undefined instruction, PC at 0x77f29190ecf0 [00126.657] 01106.01145> CS: 0 RIP: 0x77f29190ecf0 EFL: 0x202 CR2: 0 [00126.657] 01106.01145> RAX: 0xa RBX: 0x77f2919cf040 RCX: 0 RDX: 0x77f2919cf0cc [00126.657] 01106.01145> RSI: 0 RDI: 0x77f2919cf040 RBP: 0x39958b1d6270 RSP: 0x39958b1d5ea8 [00126.657] 01106.01145> R8: 0 R9: 0 R10: 0x1d R11: 0xc943df869f0 [00126.657] 01106.01145> R12: 0x39958b1d6278 R13: 0x6577368321f8 R14: 0x39958b1d60a0 R15: 0xc943de7e270 [00126.657] 01106.01145> errc: 0 [00126.657] 01106.01145> bottom of user stack: [00126.658] 01106.01145> 0x000039958b1d5ea8: 36555b1f 00006577 00000008 00000030 |.[U6we......0...| [00126.658] 01106.01145> 0x000039958b1d5eb8: 8b1d5f90 00003995 8b1d5ed0 00003995 |._...9...^...9..| [00126.658] 01106.01145> 0x000039958b1d5ec8: 3df159a0 00000c94 8b1d6270 00003995 |.Y.=....pb...9..| [00126.658] 01106.01145> 0x000039958b1d5ed8: 3df15a68 00000c94 8b1d63c0 00003995 |hZ.=.....c...9..| [00126.658] 01106.01145> 0x000039958b1d5ee8: 00000000 00000000 00000320 00000000 |........ .......| [00126.658] 01106.01145> 0x000039958b1d5ef8: 0d0dad98 00000000 3de7e270 00000c94 |........p..=....| [00126.658] 01106.01145> 0x000039958b1d5f08: 36833140 00006577 8b1d63c0 00003995 |@1.6we...c...9..| [00126.658] 01106.01145> 0x000039958b1d5f18: 8b1d5f48 00003995 3df3fbe0 00000c94 |H_...9.....=....| [00126.658] 01106.01145> 0x000039958b1d5f28: 3637d4a1 00006577 8b1d5fb1 00003995 |..76we..._...9..| [00126.659] 01106.01145> 0x000039958b1d5f38: 8b1d6270 00003995 8b1d6150 00003995 |pb...9..Pa...9..| [00126.659] 01106.01145> 0x000039958b1d5f48: 36833140 00006577 00000000 00000000 |@1.6we..........| [00126.659] 01106.01145> 0x000039958b1d5f58: 8b1d5faf 00003995 8b1d5fd0 00003995 |._...9..._...9..| [00126.659] 01106.01145> 0x000039958b1d5f68: 368321d0 00006577 36832340 00006577 |.!.6we..@#.6we..| [00126.659] 01106.01145> 0x000039958b1d5f78: 368321f8 00006577 368321d0 00006577 |.!.6we...!.6we..| [00126.659] 01106.01145> 0x000039958b1d5f88: 365851af 00006577 3687e288 00006577 |.QX6we.....6we..| [00126.659] 01106.01145> 0x000039958b1d5f98: 3658e964 00006577 00000000 00000000 |d.X6we..........| [00126.659] 01106.01145> arch: x86_64 [00126.664] 01106.01145> dso: id=2bffc5ef8297a42b31073e216f793b44b1bf0fc0 base=0x7ca4c32a3000 name=libmxio.so [00126.664] 01106.01145> dso: id=1afff68891bb5d557654642d4575fcaa5fae11a0 base=0x77f2918f2000 name=libc.so [00126.664] 01106.01145> dso: id=1c1086d35a6f5032 base=0x657735914000 name=app:/system/base_unittests [00126.665] 01106.01145> dso: id=61a92b63d8299597be907d7a9f8c699dca56be26 base=0x5a8f58c00000 name=liblaunchpad.so [00126.665] 01106.01145> dso: id=2131d8578686e3e8c3af56d58d914b0d1a0ac5d1 base=0x58c88d78c000 name=<vDSO> [00126.794] 02531.02568> [1831/2462] MemoryDumpManagerTest.RegistrationConsistency (206 ms) [00126.794] 02531.02568> [1832/2462] MemoryDumpManagerTest.RespectTaskRunnerAffinity (898 ms) [00126.794] 02531.02568> [1833/2462] MemoryDumpManagerTest.PostTaskForSequencedTaskRunner (161 ms) [00126.794] 02531.02568> [1834/2462] MemoryDumpManagerTest.DisableFailingDumpers (156 ms) [00126.794] 02531.02568> [1835/2462] MemoryDumpManagerTest.RegisterDumperWhileDumping (168 ms) [00126.795] 02531.02568> [1836/2462] MemoryDumpManagerTest.UnregisterDumperWhileDumping (14 ms) [00126.795] 02531.02568> [1837/2462] MemoryDumpManagerTest.UnregisterDumperFromThreadWhileDumping (106 ms) [00126.795] 02531.02568> [1838/2462] MemoryDumpManagerTest.TearDownThreadWhileDumping (114 ms) [00126.795] 02531.02568> [1839/2462] MemoryDumpManagerTest.TriggerDumpWithoutTracing (90 ms) [00126.795] 02531.02568> [1840/2462] MemoryDumpManagerTest.SummaryOnlyWhitelisting (44 ms) [00126.814] 01106.01145> bt#19: pc 0 sp 0x39958b1d7000 ----- start symbolized stack #01: base_unittests+0x1ccf0 #02: base_unittests+0xc41b1f #03: base_unittests+0xc711af #04: testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith(void const*) at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/src/gmock-spec-builders.cc:410 #05: InvokeWith at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1590 (inlined by) Invoke at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/include/gmock/gmock-generated-function-mockers.h:101 (inlined by) OnTick at /work/cr/src/out/fuch/../../base/trace_event/memory_dump_scheduler_unittest.cc:27 #06: Run at /work/cr/src/out/fuch/../../base/callback.h:80 (inlined by) Tick at /work/cr/src/out/fuch/../../base/trace_event/memory_dump_scheduler.cc:99 #07: Run at /work/cr/src/out/fuch/../../base/callback.h:92 (inlined by) RunTask at /work/cr/src/out/fuch/../../base/debug/task_annotator.cc:59 #08: empty at /work/cr/src/out/fuch/../../buildtools/third_party/libc++/trunk/include/vector:644 (inlined by) begin at /work/cr/src/out/fuch/../../base/observer_list.h:133 (inlined by) RunTask at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:411 #09: base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:421 #10: base::MessageLoop::DoDelayedWork(base::TimeTicks*) at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:568 #11: base::MessagePumpDefault::Run(base::MessagePump::Delegate*) at /work/cr/src/out/fuch/../../base/message_loop/message_pump_default.cc:38 #12: base::MessageLoop::Run() at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:350 #13: base::RunLoop::Run() at /work/cr/src/out/fuch/../../base/run_loop.cc:124 #14: base::Thread::Run(base::RunLoop*) at /work/cr/src/out/fuch/../../base/threading/thread.cc:255 #15: Acquire at /work/cr/src/out/fuch/../../base/synchronization/lock.h:45 (inlined by) AutoLock at /work/cr/src/out/fuch/../../base/synchronization/lock.h:115 (inlined by) ThreadMain at /work/cr/src/out/fuch/../../base/threading/thread.cc:341 #16: base::(anonymous namespace)::ThreadFunc(void*) at /work/cr/src/out/fuch/../../base/threading/platform_thread_posix.cc:73 #17: base_unittests+0x19896 #18: base_unittests+0x910fa ----- end symbolized stack Current theory is that the bg_thread_ should be destroyed before the on_tick_. I'll post a CL for that.
,
Aug 11 2017
On SingleTrigger too (before the posted patch) So hopefully we can be semi-confident this is a correct fix once it lands: [00126.403] 01106.01145> <== fatal exception: process [41410] thread [41725] [00126.403] 01106.01145> <== undefined instruction, PC at 0x7e6b56d2fcf0 [00126.404] 01106.01145> CS: 0 RIP: 0x7e6b56d2fcf0 EFL: 0x202 CR2: 0 [00126.404] 01106.01145> RAX: 0xa RBX: 0x7e6b56df0040 RCX: 0 RDX: 0x7e6b56df00cc [00126.404] 01106.01145> RSI: 0 RDI: 0x7e6b56df0040 RBP: 0x7ef49b22e1f8 RSP: 0x456a0593ea8 [00126.404] 01106.01145> R8: 0 R9: 0 R10: 0x1d R11: 0x456a05942b9 [00126.404] 01106.01145> R12: 0x1 R13: 0x1 R14: 0x456a05943c0 R15: 0x6003859d0270 [00126.404] 01106.01145> errc: 0 [00126.404] 01106.01145> bottom of user stack: [00126.405] 01106.01145> 0x00000456a0593ea8: 9af51b1f 00007ef4 00000008 00000030 |.....~......0...| [00126.405] 01106.01145> 0x00000456a0593eb8: a0593f90 00000456 a0593ed0 00000456 |.?Y.V....>Y.V...| [00126.405] 01106.01145> 0x00000456a0593ec8: 859b71c0 00006003 85ad8960 00006003 |.q...`..`....`..| [00126.405] 01106.01145> 0x00000456a0593ed8: a05943c0 00000456 a0594270 00000456 |.CY.V...pBY.V...| [00126.405] 01106.01145> 0x00000456a0593ee8: 00000000 00000000 00000328 00000000 |........(.......| [00126.405] 01106.01145> 0x00000456a0593ef8: 600385ad 00000000 00000000 00006003 |...`.........`..| [00126.405] 01106.01145> 0x00000456a0593f08: 9b017eb1 00000001 a05943f8 00000000 |.~.......CY.....| [00126.405] 01106.01145> 0x00000456a0593f18: 9af4f5d1 00007ef4 3ed0e000 000060a2 |.....~.....>.`..| [00126.405] 01106.01145> 0x00000456a0593f28: 9b017f43 00000000 a0594141 00000456 |C.......AAY.V...| [00126.405] 01106.01145> 0x00000456a0593f38: a0594278 00000456 00000001 00000000 |xBY.V...........| [00126.405] 01106.01145> 0x00000456a0593f48: 00000001 00000000 a05942b8 00000456 |.........BY.V...| [00126.405] 01106.01145> 0x00000456a0593f58: 00000000 00000000 9b22e1f8 00007ef4 |.........."..~..| [00126.405] 01106.01145> 0x00000456a0593f68: 9af4f498 00007ef4 a05942b9 00000456 |.....~...BY.V...| [00126.405] 01106.01145> 0x00000456a0593f78: 9a5f66aa 00007ef4 a0594278 00000456 |.f_..~..xBY.V...| [00126.405] 01106.01145> 0x00000456a0593f88: 9af811af 00007ef4 9b27a288 00007ef4 |.....~....'..~..| [00126.405] 01106.01145> 0x00000456a0593f98: 9af8a672 00007ef4 407e5eca 1e3ea095 |r....~...^~@..>.| [00126.406] 01106.01145> arch: x86_64 [00126.420] 01106.01145> dso: id=ea76b5512d327db5 base=0x7ef49a310000 name=app:/system/base_unittests [00126.420] 01106.01145> dso: id=1afff68891bb5d557654642d4575fcaa5fae11a0 base=0x7e6b56d13000 name=libc.so [00126.420] 01106.01145> dso: id=2131d8578686e3e8c3af56d58d914b0d1a0ac5d1 base=0x773452cc2000 name=<vDSO> [00126.420] 01106.01145> dso: id=2bffc5ef8297a42b31073e216f793b44b1bf0fc0 base=0x5a72c5b6f000 name=libmxio.so [00126.420] 01106.01145> dso: id=61a92b63d8299597be907d7a9f8c699dca56be26 base=0x33c3da503000 name=liblaunchpad.so [00126.472] 01106.01145> bt#19: pc 0 sp 0x456a0595000 ----- start symbolized stack #01: base_unittests+0x1ccf0 #02: base_unittests+0xc41b1f #03: base_unittests+0xc711af #04: testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith(void const*) at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/src/gmock-spec-builders.cc:376 #05: InvokeWith at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/include/gmock/gmock-spec-builders.h:1590 (inlined by) Invoke at /work/cr/src/out/fuch/../../third_party/googletest/src/googlemock/include/gmock/gmock-generated-function-mockers.h:101 (inlined by) OnTick at /work/cr/src/out/fuch/../../base/trace_event/memory_dump_scheduler_unittest.cc:27 #06: Run at /work/cr/src/out/fuch/../../base/callback.h:80 (inlined by) Tick at /work/cr/src/out/fuch/../../base/trace_event/memory_dump_scheduler.cc:99 #07: Run at /work/cr/src/out/fuch/../../base/callback.h:92 (inlined by) RunTask at /work/cr/src/out/fuch/../../base/debug/task_annotator.cc:59 #08: empty at /work/cr/src/out/fuch/../../buildtools/third_party/libc++/trunk/include/vector:644 (inlined by) begin at /work/cr/src/out/fuch/../../base/observer_list.h:133 (inlined by) RunTask at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:411 #09: base::MessageLoop::DeferOrRunPendingTask(base::PendingTask) at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:421 #10: base::MessageLoop::DoDelayedWork(base::TimeTicks*) at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:568 #11: base::MessagePumpDefault::Run(base::MessagePump::Delegate*) at /work/cr/src/out/fuch/../../base/message_loop/message_pump_default.cc:38 #12: base::MessageLoop::Run() at /work/cr/src/out/fuch/../../base/message_loop/message_loop.cc:350 #13: base::RunLoop::Run() at /work/cr/src/out/fuch/../../base/run_loop.cc:124 #14: base::Thread::Run(base::RunLoop*) at /work/cr/src/out/fuch/../../base/threading/thread.cc:255 #15: Acquire at /work/cr/src/out/fuch/../../base/synchronization/lock.h:45 (inlined by) AutoLock at /work/cr/src/out/fuch/../../base/synchronization/lock.h:115 (inlined by) ThreadMain at /work/cr/src/out/fuch/../../base/threading/thread.cc:341 #16: base::(anonymous namespace)::ThreadFunc(void*) at /work/cr/src/out/fuch/../../base/threading/platform_thread_posix.cc:73 #17: base_unittests+0x19896 #18: base_unittests+0x910fa ----- end symbolized stack [00126.491] 02940.02977> [1821/2462] MemoryDumpManagerTest.UnregisterAndDeleteDumpProviderSoon (43 ms) [00126.491] 02940.02977> [1822/2462] MemoryDumpManagerTest.UnregisterAndDeleteDumpProviderSoonDuringDump (161 ms) [00126.492] 02940.02977> [ RUN ] MemoryDumpSchedulerTest.SingleTrigger [00126.492] 02940.02977> Pure virtual function called!
,
Aug 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/58df80ac3ffa8ceb08eea641039a264754722be1 commit 58df80ac3ffa8ceb08eea641039a264754722be1 Author: Scott Graham <scottmg@chromium.org> Date: Mon Aug 14 15:46:59 2017 Ensure MemoryDumpSchedulerTest tears down thread before on_tick_ The crash in https://bugs.chromium.org/p/chromium/issues/detail?id=751748#c16 shows it in the on_tick_ I believe after the test body has been torn down. I believe this is because bg_thread_ needs to be torn down before on_tick_ so that the thread finishes running outstanding tasks before on_tick_ is torn down. Reorder the test fixture members to ensure this. Bug: 751748 Change-Id: Ib20396c2b1acc376686cc68b269c31592dbbe345 Reviewed-on: https://chromium-review.googlesource.com/612481 Reviewed-by: Hector Dearman <hjd@chromium.org> Reviewed-by: Primiano Tucci <primiano@chromium.org> Commit-Queue: Scott Graham <scottmg@chromium.org> Cr-Commit-Position: refs/heads/master@{#494055} [modify] https://crrev.com/58df80ac3ffa8ceb08eea641039a264754722be1/base/trace_event/memory_dump_scheduler_unittest.cc
,
Aug 30 2017
Still not quite right apparently :( https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium.fyi%2FFuchsia%2F9090%2F%2B%2Frecipes%2Fsteps%2Fbase_unittests%2F0%2Fstdout
,
Sep 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/48ef42e9021e1d4b24156733fca8c5e8f229fe35 commit 48ef42e9021e1d4b24156733fca8c5e8f229fe35 Author: Lalit Maganti <lalitm@google.com> Date: Fri Sep 08 14:20:00 2017 Increase period of ticks in start-stop memory scheduler test This test is flaky on Fuschia due to a timeout whose root cause is 6 ticks for the LIGHT detail invocations when less than 5 are expected. We believe that this is because the Fuschia scheduler is not scheduling the test thread for more than 1ms which allows the BG thread to get two ticks in. This causes the count to be greater than expected. By increasing the time between ticks, we should make it significantly more unlikely that a test will fail in this way. This slightly increases the time taken to test from 10 to 30ms but most other fixes would significantly increase the complexity of the code to remove the race entirely Bug: 751748 Change-Id: I876b483ef0d4983c1ae4e552581f06fb38b538fd Reviewed-on: https://chromium-review.googlesource.com/652997 Reviewed-by: Primiano Tucci <primiano@chromium.org> Commit-Queue: Lalit Maganti <lalitm@google.com> Cr-Commit-Position: refs/heads/master@{#500586} [modify] https://crrev.com/48ef42e9021e1d4b24156733fca8c5e8f229fe35/base/trace_event/memory_dump_scheduler_unittest.cc
,
Sep 8 2017
Hopefully that should fix it - I couldn't reproduce locally so we'll see if the trybots complain some more.
,
Jan 24 2018
Closing this as fixed based on #18 and #21; please open new bugs for new flakes/regressions. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by scottmg@chromium.org
, Aug 2 2017