New issue
Advanced search Search tips

Issue 875278 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Chrome , Mac , Fuchsia
Pri: 1
Type: Bug



Sign in to add a comment

RTCRtpSenderTest base indirectly initializes TaskScheduler, breaking later tests in the same batch.

Project Member Reported by peter@chromium.org, Aug 17

Issue description

The RTCVideoDecoderAdapterTest.Lifecycle test has been flakily crashing on a bunch of bots and is blocking try-jobs on unrelated CLs. It seems to be an issue with getting a TaskScheduler instance -

https://ci.chromium.org/p/chromium/builders/luci.chromium.try/linux-chromeos-dbg/316
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/fuchsia_x64/87148
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/fuchsia_x64/87170
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/linux_chromium_dbg_ng/2752
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/mac_chromium_dbg_ng/523

The stack trace is as follows -

[ RUN      ] RTCVideoDecoderAdapterTest.Lifecycle
[15316:15316:0813/180511.190972:10171218023:FATAL:scoped_task_environment.cc(122)] Check failed: !TaskScheduler::GetInstance(). 
#0 0x7fa15cbdf4ad base::debug::StackTrace::StackTrace()
#1 0x7fa15c8eac7c base::debug::StackTrace::StackTrace()
#2 0x7fa15c954f6d logging::LogMessage::~LogMessage()
#3 0x0000037aaa03 base::test::ScopedTaskEnvironment::ScopedTaskEnvironment()
#4 0x0000022c5e40 content::RTCVideoDecoderAdapterTest::RTCVideoDecoderAdapterTest()
#5 0x0000022c5dcf content::RTCVideoDecoderAdapterTest_Lifecycle_Test::RTCVideoDecoderAdapterTest_Lifecycle_Test()
#6 0x0000022c5d97 testing::internal::TestFactoryImpl<>::CreateTest()
#7 0x00000132d6de _ZN7testing8internal12InvokeHelperIdNSt3__15tupleIJEEEE12InvokeMethodIN5media23VirtualAudioInputStreamEMS8_FdvEEEdPT_T0_RKS4_
#8 0x000002a97252 testing::internal::HandleExceptionsInMethodIfSupported<>()
#9 0x000002a7b4eb testing::TestInfo::Run()
#10 0x000002a7bfbf testing::TestCase::Run()
#11 0x000002a8e168 testing::internal::UnitTestImpl::RunAllTests()
#12 0x00000132c8ee _ZN7testing8internal12InvokeHelperIbNSt3__15tupleIJEEEE12InvokeMethodIN5media23VirtualAudioInputStreamEMS8_FbvEEEbPT_T0_RKS4_
#13 0x000002a98532 testing::internal::HandleExceptionsInMethodIfSupported<>()
#14 0x000002a8ddc7 testing::UnitTest::Run()
#15 0x0000037c0291 RUN_ALL_TESTS()
#16 0x0000037bcc5b base::TestSuite::Run()
#17 0x00000370be10 content::UnitTestTestSuite::Run()
#18 0x0000007de13d _ZN4base8internal13FunctorTraitsIMN7content26AppCacheRequestHandlerTestEFvvEvE6InvokeIS5_PS3_JEEEvT_OT0_DpOT1_
#19 0x0000007de084 _ZN4base8internal12InvokeHelperILb0EvE8MakeItSoIMN7content26AppCacheRequestHandlerTestEFvvEJPS5_EEEvOT_DpOT0_
#20 0x000002517205 _ZN4base8internal7InvokerINS0_9BindStateIMN7content17UnitTestTestSuiteEFivEJNS0_17UnretainedWrapperIS4_EEEEEFivEE7RunImplIS6_NSt3__15tupleIJS8_EEEJLm0EEEEiOT_OT0_NSD_16integer_sequenceImJXspT1_EEEE
#21 0x000002517149 _ZN4base8internal7InvokerINS0_9BindStateIMN7content17UnitTestTestSuiteEFivEJNS0_17UnretainedWrapperIS4_EEEEEFivEE7RunOnceEPNS0_13BindStateBaseE
#22 0x0000037eb34e _ZNO4base12OnceCallbackIFivEE3RunEv
#23 0x0000037e4510 base::(anonymous namespace)::LaunchUnitTestsInternal()
#24 0x0000037e4385 base::LaunchUnitTests()
#25 0x000002517046 main
#26 0x7fa145fcaf45 __libc_start_main
#27 0x000000709fca _start
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/2638dc394233d3f2cb550de92ac945ddb9263deb

commit 2638dc394233d3f2cb550de92ac945ddb9263deb
Author: Peter Beverloo <peter@chromium.org>
Date: Fri Aug 17 15:44:03 2018

Disable RTCVideoDecoderAdapterTest.Lifecycle

It's been crashing flakily on various platforms, and is blocking
try-bots on unrelated CLs.

TBR=sandersd@chromium.org
NOTRY=true (disabling a test)

Bug:  875278 
Change-Id: I93ebe231e45bdc813e39f465056dfc6bd1cde920
Reviewed-on: https://chromium-review.googlesource.com/1179892
Commit-Queue: Peter Beverloo <peter@chromium.org>
Reviewed-by: Peter Beverloo <peter@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584085}
[modify] https://crrev.com/2638dc394233d3f2cb550de92ac945ddb9263deb/content/renderer/media/webrtc/rtc_video_decoder_adapter_unittest.cc

Summary: RTCVideoDecoderAdapterTest tests flakily crashing (was: RTCVideoDecoderAdapterTest.Lifecycle flakily crashing)
Turning this into the uber bug.
Cc: sande...@chromium.org
 Issue 875301  has been merged into this issue.
 Issue 875336  has been merged into this issue.
 Issue 875390  has been merged into this issue.
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 17

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/695b883a3dad5ebfd1fe9cd31e752407765e9619

commit 695b883a3dad5ebfd1fe9cd31e752407765e9619
Author: Jennifer Apacible <apacible@chromium.org>
Date: Fri Aug 17 19:08:05 2018

Disable all RTCVideoDecoderAdapterTests as they are flakily crashing.

TBR=sandersd@chromium.org
NOTRY=true (disabling a test)

Bug:  875278 
Change-Id: I26c070fc09e5c7798fcbed324286c7b4e7ece379
Reviewed-on: https://chromium-review.googlesource.com/1180247
Reviewed-by: apacible <apacible@chromium.org>
Commit-Queue: apacible <apacible@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584142}
[modify] https://crrev.com/695b883a3dad5ebfd1fe9cd31e752407765e9619/content/renderer/media/webrtc/rtc_video_decoder_adapter_unittest.cc

Labels: M-70
AFAICT this is not a bug in these tests themselves, but rather an issue with a preceding test initializing TaskScheduler, so that ScopedTaskEnvironment then fails; I'm repro'ing the issue locally to establish which tests are running in the failing batch process before the crash.
Cc: gab@chromium.org w...@chromium.org
Components: Internals>TaskScheduler
+gab in case he has tips on finding the culprit...
Owner: w...@chromium.org
I'm looking into what is "leaking" a TaskScheduler now, will update here and assign back when done.
Status: Started (was: Assigned)
Summary: RTCRtpSenderTest base indirectly initializes TaskScheduler, breaking later tests in the same batch. (was: RTCVideoDecoderAdapterTest tests flakily crashing)
 Issue 875491  has been merged into this issue.
Fix for the errant test suite is up at: https://chromium-review.googlesource.com/c/chromium/src/+/1180408

Unfortunately this still barfs on one Fuchsia test, which we lack output for because of  issue 875113  - have a fix up for that as well (https://chromium-review.googlesource.com/c/chromium/src/+/1180599) which I'm rebasing the fix for this on, in the hope of making progress...
Project Member

Comment 13 by bugdroid1@chromium.org, Aug 18

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9eb10b3c39d3309dfe72a7c6439d5f7d20013a1e

commit 9eb10b3c39d3309dfe72a7c6439d5f7d20013a1e
Author: Wez <wez@chromium.org>
Date: Sat Aug 18 03:29:18 2018

Replace MessageLoop with ScopedTaskEnvironment in RTCRtpSenderTest.

This test base instantiates a content::ChildProcess to provide the
expected environment for the code-under-test. ChildProcess expects to
either run in the context of ScopedTaskEnvironment, in tests, or to
need to initialize the TaskScheduler itself.

Add a ScopedTaskEnvironment to the test base, otherwise the ChildProcess
TaskScheduler will "leak" and break later tests.

Re-enables the RTCVideoDecoderAdapterTest tests, that had been
broken by this issue.

TBR=hbos

Bug:  875278 ,  875301 ,  875336 
Change-Id: I8a7ee1f42e62241d48720c41897bdcadbcd26bdf
Reviewed-on: https://chromium-review.googlesource.com/1180408
Commit-Queue: Wez <wez@chromium.org>
Reviewed-by: Wez <wez@chromium.org>
Cr-Commit-Position: refs/heads/master@{#584298}
[modify] https://crrev.com/9eb10b3c39d3309dfe72a7c6439d5f7d20013a1e/content/renderer/media/webrtc/rtc_rtp_receiver_unittest.cc
[modify] https://crrev.com/9eb10b3c39d3309dfe72a7c6439d5f7d20013a1e/content/renderer/media/webrtc/rtc_rtp_sender_unittest.cc
[modify] https://crrev.com/9eb10b3c39d3309dfe72a7c6439d5f7d20013a1e/content/renderer/media/webrtc/rtc_video_decoder_adapter_unittest.cc

Status: Fixed (was: Started)

Sign in to add a comment