"WebRtcInternalsTest.SendAllUpdateWithGetUserMedia" is flaky |
|||||||||
Issue description"WebRtcInternalsTest.SendAllUpdateWithGetUserMedia" is flaky. This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label. We have detected 3 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyPAsSBUZsYWtlIjFXZWJSdGNJbnRlcm5hbHNUZXN0LlNlbmRBbGxVcGRhdGVXaXRoR2V0VXNlck1lZGlhDA. Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs
,
Dec 20 2017
Seems like this is the first crash in the log (SendRemovePeerConnectionUpdate): [ RUN ] WebRtcInternalsTest.SendRemovePeerConnectionUpdate [7471:29955:1218/233831.362150:643506680488:FATAL:message_loop.cc(97)] Check failed: (pump_ && current() == this) || (!pump_ && current() != this). 0 content_unittests 0x0000000104a3501c base::debug::StackTrace::StackTrace(unsigned long) + 28 1 content_unittests 0x0000000104a59350 logging::LogMessage::~LogMessage() + 224 2 content_unittests 0x0000000104a70837 base::MessageLoop::~MessageLoop() + 215 3 content_unittests 0x0000000104a710e5 non-virtual thunk to base::MessageLoop::~MessageLoop() + 21 4 content_unittests 0x00000001028532df content::WebRtcEventLogManager::DisableLocalLoggingInternal(base::OnceCallback<void (bool)>) + 111 5 content_unittests 0x0000000102854431 base::internal::Invoker<base::internal::BindState<void (content::WebRtcEventLogManager::*)(base::OnceCallback<void (bool)>), base::internal::UnretainedWrapper<content::WebRtcEventLogManager>, base::OnceCallback<void (bool)> >, void ()>::RunOnce(base::internal::BindStateBase*) + 65 6 content_unittests 0x0000000104a35995 base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) + 261 7 content_unittests 0x0000000104ae1018 base::internal::TaskTracker::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) + 840 8 content_unittests 0x0000000104ae1e38 base::internal::TaskTrackerPosix::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) + 168 9 content_unittests 0x0000000103cab364 base::test::ScopedTaskEnvironment::TestTaskTracker::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) + 164 10 content_unittests 0x0000000104ae0188 base::internal::TaskTracker::RunNextTask(scoped_refptr<base::internal::Sequence>, base::internal::CanScheduleSequenceObserver*) + 472 11 content_unittests 0x0000000104ad79d3 base::internal::SchedulerWorker::Thread::ThreadMain() + 691 12 content_unittests 0x0000000104aec17f base::(anonymous namespace)::ThreadFunc(void*) + 95 13 libsystem_pthread.dylib 0x00007fff88534899 _pthread_body + 138 14 libsystem_pthread.dylib 0x00007fff8853472a _pthread_struct_init + 0 15 libsystem_pthread.dylib 0x00007fff88538fc9 thread_start + 13
,
Dec 20 2017
Elad - could this be related to this CL? https://chromium-review.googlesource.com/c/chromium/src/+/811647
,
Dec 20 2017
,
Dec 29 2017
Detected 3 new flakes for test/step "WebRtcInternalsTest.SendAllUpdateWithGetUserMedia". To see the actual flakes, please visit https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNyPAsSBUZsYWtlIjFXZWJSdGNJbnRlcm5hbHNUZXN0LlNlbmRBbGxVcGRhdGVXaXRoR2V0VXNlck1lZGlhDA. This message was posted automatically by the chromium-try-flakes app. Since flakiness is ongoing, the issue was moved back into Sheriff Bug Queue (unless already there).
,
Jan 2 2018
Looks like tasks are pending after test teardown. Msan stack trace:
[ RUN ] WebRtcInternalsTest.AddRemoveObserver
==228279==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0xc9bfd72 in size /home/maxmorin/chromium/src/out/lin/../../buildtools/third_party/libc++/trunk/include/string:889:17
#1 0xc9bfd72 in empty /home/maxmorin/chromium/src/out/lin/../../buildtools/third_party/libc++/trunk/include/string:904:0
#2 0xc9bfd72 in empty /home/maxmorin/chromium/src/out/lin/../../base/files/file_path.h:201:0
#3 0xc9bfd72 in content::WebRtcLocalEventLogManager::DisableLogging() /home/maxmorin/chromium/src/out/lin/../../content/browser/webrtc/webrtc_local_event_log_manager.cc:100:0
#4 0xc9a52e3 in content::WebRtcEventLogManager::DisableLocalLoggingInternal(base::OnceCallback<void (bool)>) /home/maxmorin/chromium/src/out/lin/../../content/browser/webrtc/webrtc_event_log_manager.cc:200:43
#5 0xc9a7eb0 in Invoke<content::WebRtcEventLogManager *, base::OnceCallback<void (bool)> > /home/maxmorin/chromium/src/out/lin/../../base/bind_internal.h:211:12
#6 0xc9a7eb0 in MakeItSo<void (content::WebRtcEventLogManager::*)(base::OnceCallback<void (bool)>), content::WebRtcEventLogManager *, base::OnceCallback<void (bool)> > /home/maxmorin/chromium/src/out/lin/../../base/bind_internal.h:294:0
#7 0xc9a7eb0 in RunImpl<void (content::WebRtcEventLogManager::*)(base::OnceCallback<void (bool)>), std::__1::tuple<base::internal::UnretainedWrapper<content::WebRtcEventLogManager>, base::OnceCallback<void (bool)> >, 0, 1> /home/maxmorin/chromium/src/out/lin/../../base/bind_internal.h:368:0
#8 0xc9a7eb0 in base::internal::Invoker<base::internal::BindState<void (content::WebRtcEventLogManager::*)(base::OnceCallback<void (bool)>), base::internal::UnretainedWrapper<content::WebRtcEventLogManager>, base::OnceCallback<void (bool)> >, void ()>::RunOnce(base::internal::BindStateBase*) /home/maxmorin/chromium/src/out/lin/../../base/bind_internal.h:336:0
#9 0xfb15d7a in Run /home/maxmorin/chromium/src/out/lin/../../base/callback.h:65:12
#10 0xfb15d7a in base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*) /home/maxmorin/chromium/src/out/lin/../../base/debug/task_annotator.cc:55:0
#11 0xfc87150 in base::internal::TaskTracker::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) /home/maxmorin/chromium/src/out/lin/../../base/task_scheduler/task_tracker.cc:410:30
#12 0xfc8ac80 in base::internal::TaskTrackerPosix::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) /home/maxmorin/chromium/src/out/lin/../../base/task_scheduler/task_tracker_posix.cc:23:16
#13 0xd21c358 in base::test::ScopedTaskEnvironment::TestTaskTracker::RunOrSkipTask(base::internal::Task, base::internal::Sequence*, bool) /home/maxmorin/chromium/src/out/lin/../../base/test/scoped_task_environment.cc:273:49
#14 0xfc84c28 in base::internal::TaskTracker::RunNextTask(scoped_refptr<base::internal::Sequence>, base::internal::CanScheduleSequenceObserver*) /home/maxmorin/chromium/src/out/lin/../../base/task_scheduler/task_tracker.cc:312:3
#15 0xfdd02e6 in base::internal::SchedulerWorker::Thread::ThreadMain() /home/maxmorin/chromium/src/out/lin/../../base/task_scheduler/scheduler_worker.cc:72:41
#16 0xfcb3385 in base::(anonymous namespace)::ThreadFunc(void*) /home/maxmorin/chromium/src/out/lin/../../base/threading/platform_thread_posix.cc:75:13
#17 0x7f4b18514183 in start_thread /build/eglibc-SvCtMH/eglibc-2.19/nptl/pthread_create.c:312:0
#18 0x7f4b1105bffc in clone /build/eglibc-SvCtMH/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111:0
Uninitialized value was created by an allocation of 'ref.tmp' in the stack frame of function '_ZN7content23TestBrowserThreadBundleD2Ev'
#0 0xd0d8d50 in content::TestBrowserThreadBundle::~TestBrowserThreadBundle() /home/maxmorin/chromium/src/out/lin/../../content/public/test/test_browser_thread_bundle.cc:32:0
SUMMARY: MemorySanitizer: use-of-uninitialized-value (/home/maxmorin/chromium/src/out/lin/content_unittests+0xc9bfd72)
Exiting
[1/12] WebRtcInternalsTest.AddRemoveObserver (CRASHED)
Attempting revert.
,
Jan 2 2018
Revert doesn't apply cleanly, so I'll disable all the tests instead.
,
Jan 2 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/226ae5da35e1cf8da99e551686e020fab613c15f commit 226ae5da35e1cf8da99e551686e020fab613c15f Author: maxmorin <maxmorin@chromium.org> Date: Tue Jan 02 12:50:43 2018 Disable all WebRtcInternalsTests. These tests are now very flaky. Bug: 775415, 787809 , 787938 , 791022 , 796047 Tbr: phoglund@chromium.org Notry: True Change-Id: I3ace2542e573d533aab6cfbc1558e4a7ebccce30 Reviewed-on: https://chromium-review.googlesource.com/846882 Reviewed-by: Max Morin <maxmorin@chromium.org> Reviewed-by: Patrik Höglund <phoglund@chromium.org> Commit-Queue: Max Morin <maxmorin@chromium.org> Cr-Commit-Position: refs/heads/master@{#526466} [modify] https://crrev.com/226ae5da35e1cf8da99e551686e020fab613c15f/content/browser/webrtc/webrtc_internals_unittest.cc
,
Jan 2 2018
Disabling the entire test suite is quite unfortunate. PTAL ASAP.
,
Jan 2 2018
I've come back from vacation today. I'm looking into it.
,
Jan 2 2018
This CL fixes the problem: https://chromium-review.googlesource.com/c/chromium/src/+/846994/1/content/browser/webrtc/webrtc_internals_unittest.cc As evidenced by the CL, the problem was only with the unit tests, so it does not need to be cherry-picked. (I am not sure if this means we'd want to decrease the bug's priority?)
,
Jan 4 2018
Can https://chromium-review.googlesource.com/846994 please be landed? This flaky test is affecting the commit queue, for example: https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/332426
,
Jan 4 2018
Adding mcasas@, who is an owner whose calendar claims is not OOO.
,
Jan 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c89aa9438e053b9a1e7d5667885e0db2f4754495 commit c89aa9438e053b9a1e7d5667885e0db2f4754495 Author: Elad Alon <eladalon@chromium.org> Date: Thu Jan 04 20:37:05 2018 Deflake WebRtcInternalsTest The tests were flaky because of a race with WebRtcEventLogManager::task_runner_. The problem is that SequencedTaskRunner's destructor does not block, allowing us to destroy a WebRtcEventLogManager objects while its |task_runner_| is still executing a task that operates on the WebRtcEventLogManager object. We solve this by making WebRtcEventLogManager work synchronously when it's used in WebRTCInternals' unit tests (but nowhere else). Bug: 796047 Change-Id: I76606b9394c1e5c02a3626fd8d3824bea587d1e4 Reviewed-on: https://chromium-review.googlesource.com/850697 Commit-Queue: Elad Alon <eladalon@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Cr-Commit-Position: refs/heads/master@{#527083} [modify] https://crrev.com/c89aa9438e053b9a1e7d5667885e0db2f4754495/content/browser/webrtc/webrtc_event_log_manager.cc [modify] https://crrev.com/c89aa9438e053b9a1e7d5667885e0db2f4754495/content/browser/webrtc/webrtc_event_log_manager.h [modify] https://crrev.com/c89aa9438e053b9a1e7d5667885e0db2f4754495/content/browser/webrtc/webrtc_internals_unittest.cc
,
Jan 4 2018
Max, could you re-enable the disabled tests?
,
Jan 10 2018
,
Jan 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e84a57647a5dc8838a1e1053d00c31deadef051d commit e84a57647a5dc8838a1e1053d00c31deadef051d Author: Elad Alon <eladalon@chromium.org> Date: Wed Jan 17 16:56:09 2018 Re-enable WebRtcInternalsTest WebRtcInternalsTest's tests have been deflaked; re-enabling. Bug: 796047 Change-Id: I943cd2ef61e17c71da6d8ff627534777c46c8582 Reviewed-on: https://chromium-review.googlesource.com/859779 Reviewed-by: Max Morin <maxmorin@chromium.org> Reviewed-by: Miguel Casas <mcasas@chromium.org> Commit-Queue: Max Morin <maxmorin@chromium.org> Cr-Commit-Position: refs/heads/master@{#529774} [modify] https://crrev.com/e84a57647a5dc8838a1e1053d00c31deadef051d/content/browser/webrtc/webrtc_internals_unittest.cc
,
Jan 24 2018
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by qyears...@chromium.org
, Dec 19 2017Labels: OS-Android OS-Mac