WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQualityH264 flaky crash |
||||||
Issue descriptionChromium WebRTC Mac Tester bot crashes flakily on WebRtcVideoQualityBrowserTests/WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQualityH264/1. https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester As far as I can tell, first seen May 13 7:31: https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/54704 Output: (Lots of the "failed to encode frame" error.) [9958:37635:0515/230439:ERROR:video_sender.cc(317)] Failed to encode frame. Error code: -1 BrowserTestBase signal handler received SIGTERM. Backtrace: [9958:37635:0515/230439:ERROR:video_sender.cc(317)] Failed to encode frame. Error code: -1 0 browser_tests 0x00000001072f01b3 _ZN4base5debug10StackTraceC1Ev + 19 1 browser_tests 0x00000001073cf6e1 _ZN7content12_GLOBAL__N_1L27DumpStackTraceSignalHandlerEi + 65 2 libsystem_platform.dylib 0x00007fff8e08df1a _sigtramp + 26 3 browser_tests 0x0000000107d9b8e9 _ZN7Browser22NavigationStateChangedEPN7content11WebContentsENS0_15InvalidateTypesE + 105 4 CoreFoundation 0x00007fff84095eb4 __CFRunLoopServiceMachPort + 212 5 CoreFoundation 0x00007fff8409537b __CFRunLoopRun + 1371 6 CoreFoundation 0x00007fff84094bd8 CFRunLoopRunSpecific + 296 7 HIToolbox 0x00007fff89ad056f RunCurrentEventLoopInMode + 235 8 HIToolbox 0x00007fff89ad02ea ReceiveNextEventCommon + 431 9 HIToolbox 0x00007fff89ad012b _BlockUntilNextEventMatchingListInModeWithFilter + 71 10 AppKit 0x00007fff86a7e8ab _DPSNextEvent + 978 11 AppKit 0x00007fff86a7de58 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 346 12 AppKit 0x00007fff86a73af3 -[NSApplication run] + 594 13 browser_tests 0x00000001072e296f _ZN4base24MessagePumpNSApplication5DoRunEPNS_11MessagePump8DelegateE + 351 14 browser_tests 0x00000001072e1edf _ZN4base24MessagePumpCFRunLoopBase3RunEPNS_11MessagePump8DelegateE + 127 15 browser_tests 0x000000010731d097 _ZN4base11MessageLoop10RunHandlerEv + 215 16 browser_tests 0x000000010733d733 _ZN4base7RunLoop3RunEv + 51 17 browser_tests 0x000000010caccabf _ZN7content14RunThisRunLoopEPN4base7RunLoopE + 79 18 browser_tests 0x00000001073d14cb _ZN7content12_GLOBAL__N_119ExecuteScriptHelperEPNS_15RenderFrameHostERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEPNS3_10unique_ptrIN4base5ValueENS3_14default_deleteISE_EEEE + 267 19 browser_tests 0x00000001073d1d27 _ZN7content29ExecuteScriptAndExtractStringERKNS_17ToRenderFrameHostERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEPS9_ + 119 20 browser_tests 0x000000010669b149 _ZN4test16PollingWaitUntilERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEES8_PN7content11WebContentsEi + 281 21 browser_tests 0x00000001066aab52 _ZN29WebRtcVideoQualityBrowserTest16TestVideoQualityERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE + 994 22 browser_tests 0x00000001066ab663 _ZN62WebRtcVideoQualityBrowserTest_MANUAL_TestVideoQualityH264_Test19RunTestOnMainThreadEv + 83 23 browser_tests 0x00000001072da499 _ZN20InProcessBrowserTest23RunTestOnMainThreadLoopEv + 169 24 browser_tests 0x00000001073cf4cb _ZN7content15BrowserTestBase28ProxyRunTestOnMainThreadLoopEv + 235 25 browser_tests 0x0000000106d75c09 _ZN22ChromeBrowserMainParts25PreMainMessageLoopRunImplEv + 3945 26 browser_tests 0x0000000106d74b99 _ZN22ChromeBrowserMainParts21PreMainMessageLoopRunEv + 121 27 browser_tests 0x000000010b27838f _ZN7content15BrowserMainLoop21PreMainMessageLoopRunEv + 143 28 browser_tests 0x000000010b5f3530 _ZN7content17StartupTaskRunner14RunAllTasksNowEv + 48 29 browser_tests 0x000000010b276cb4 _ZN7content15BrowserMainLoop18CreateStartupTasksEv + 644 30 browser_tests 0x000000010b27ad7b _ZN7content21BrowserMainRunnerImpl10InitializeERKNS_18MainFunctionParamsE + 635 31 browser_tests 0x000000010b2744c5 _ZN7content11BrowserMainERKNS_18MainFunctionParamsE + 149 32 browser_tests 0x000000010ca942a2 _ZN7content21ContentMainRunnerImpl3RunEv + 802 33 browser_tests 0x000000010ca932e6 _ZN7content11ContentMainERKNS_17ContentMainParamsE + 54 34 browser_tests 0x00000001073cf244 _ZN7content15BrowserTestBase5SetUpEv + 1332 35 browser_tests 0x00000001072d97d4 _ZN20InProcessBrowserTest5SetUpEv + 308 36 browser_tests 0x00000001078e646b _ZN7testing4Test3RunEv + 299 37 browser_tests 0x00000001078e709a _ZN7testing8TestInfo3RunEv + 410 38 browser_tests 0x00000001078e7663 _ZN7testing8TestCase3RunEv + 451 39 browser_tests 0x00000001078edd09 _ZN7testing8internal12UnitTestImpl11RunAllTestsEv + 825 40 browser_tests 0x00000001078ed99d _ZN7testing8UnitTest3RunEv + 269 41 browser_tests 0x00000001073ac1f6 _ZN4base9TestSuite3RunEv + 150 42 browser_tests 0x00000001072bf48f _ZN21ChromeTestSuiteRunner12RunTestSuiteEiPPc + 31 43 browser_tests 0x000000010cac99d2 _ZN7content11LaunchTestsEPNS_20TestLauncherDelegateEiiPPc + 386 44 browser_tests 0x0000000106bb410a main + 90 45 browser_tests 0x000000010606ee34 start + 52 [104/104] WebRtcVideoQualityBrowserTests/WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQualityH264/1 (TIMED OUT) 1 test timed out: WebRtcVideoQualityBrowserTests/WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQualityH264/1 (../../chrome/browser/media/webrtc_video_quality_browsertest.cc:359)
,
May 17 2016
Sorry, this isn't an area that I'm familiar with.
,
May 23 2016
I am not sure what triggers this flaky failure. My best guess is that something in the virtual machine destroys the VTSession. In that case we return platform error[0]. Is it possible to run these in Debug? Also, adding pbos@. I thought VideoEncoderSoftwareFallbackWrapper would be triggered after these errors. WDYT? [0] https://code.google.com/p/chromium/codesearch#chromium/src/media/gpu/vt_video_encode_accelerator_mac.cc&l=297
,
May 24 2016
If you mean reconfiguring or adding bots, I guess we can, but it's a lot of work and maintenance. Why would Debug help you? The stack trace above appears to be symbolized.
,
May 24 2016
emircan@: Has to return WEBRTC_VIDEO_CODEC_FALLBACK_SOFTWARE to request a fallback in current code: https://chromium.googlesource.com/external/webrtc/+/d36df89d40bde3c62ee5cbff841933e50b3c007b/webrtc/video/video_encoder.cc#147 But either way it shouldn't crash here. Or is this stack trace just a timeout?
,
Jun 1 2016
Moving this nonessential bug to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 10 2016
This issue is Pri-1 but has already been moved once. Lowering the priority and moving to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 18 2016
Ping
,
Jul 22 2016
I cannot reproduce a crash locally on Macbook Air nor Mac Mini. I am still stuck on this. Also, I am not sure if this is a MANUAL_TestVideoQualityH264 related issue any more. Here are the last 100 bugs Build#57123-Build#57222. In this range there were 4 exceptions and 1 failure. https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester?numbuilds=100 https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/57222/steps/browser_tests/logs/stdio Hangs at MANUAL_RunsAudioVideoCall60SecsAndLogsInternalMetricsVp9 https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/57214/steps/browser_tests/logs/stdio Fails 97 tests https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/57174/steps/browser_tests/logs/stdio Hangs at MANUAL_TestVideoQualityVp8 https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/57153/steps/browser_tests/logs/stdio Hangs at MANUAL_TestVideoQualityH264/0 https://build.chromium.org/p/chromium.webrtc/builders/Mac%20Tester/builds/57135/steps/browser_tests/logs/stdio Hangs at MANUAL_TestVideoQualityH264/1 We no longer see the "Failed to encode frame" message shown in the first post at all any more. Considering this, can we mark this as fixed? I think there is an issue regarding the generic MANUAL_TestVideoQuality* sequence though. phoglung@, pbos@ WDYT?
,
Jul 22 2016
I looked at the list above. The Exceptions are all "the connection was lost in a non-clean fashion", which means the bot fell over. That's an infra problem if anything and hardly related to any tests. Well, unless the tests causes the bot to kernel panic and die, but I doubt that. The sole failure appears to be a legit failure where someone added a bad CHECK from the Chromium side (that patch seems to have gotten reverted very quickly). The test appears to be running, so appears it got fixed by someone. Yay! Closing the bug.
,
Aug 1 2016
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by grunell@chromium.org
, May 16 2016