WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQuality* tests broken on chromium.webrc |
|||||||
Issue descriptionSmallest among first failing builds: https://ci.chromium.org/buildbot/chromium.webrtc/Win10%20Tester/31493 WebRTC roll in that range: https://webrtc.googlesource.com/src.git/+log/efbcb31cb67e..1946a3f0fe2c Example Log: Completed barcode decoding. [0827/042751.062:ERROR:y4m_file_reader.cc(79)] File C:\b\c\b\Win10_Tester\src\chrome\test\data\webrtc/resources\reference_video_1280x720_30fps.yuv does not start with YUV4MPEG2 header [0827/042751.062:ERROR:y4m_file_reader.cc(79)] File C:\Users\CHROME~1\AppData\Local\Temp\scoped_dir2540_9985\captured_video.yuv does not start with YUV4MPEG2 header Error opening video files Flag 'stats_file_ref' is not recognized Flag 'stats_file_test' is not recognized Flag 'width' is not recognized Flag 'height' is not recognized You have entered: chartjson_result_file=, label=720p_VP9, reference_file=C:\b\c\b\Win10_Tester\src\chrome\test\data\webrtc/resources\reference_video_1280x720_30fps.yuv, test_file=C:\Users\CHROME~1\AppData\Local\Temp\scoped_dir2540_9985\captured_video.yuv, ../../chrome/browser/media/webrtc/webrtc_video_quality_browsertest.cc(316): error: Value of: CompareVideosAndPrintResult( MakeLabel(test_config_.test_name, video_codec), test_config_.width, test_config_.height, GetWorkingDir().Append(kCapturedYuvFileName), test::GetReferenceFilesDir() .Append(test_config_.reference_video) .AddExtension(test::kYuvFileExtension), GetWorkingDir().Append(kStatsFileName)) Actual: false Expected: true Stack trace: Backtrace: StackTraceGetter::CurrentStackTrace [0x01E97E48+40] testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop [0x01EA08A5+69] testing::internal::AssertHelper::operator= [0x01EA04F0+48] WebRtcVideoQualityBrowserTest::TestVideoQuality [0x01419A13+2435] WebRtcVideoQualityBrowserTest_MANUAL_TestVideoQualityVp9_Test::RunTestOnMainThread [0x01419B3E+62] content::BrowserTestBase::ProxyRunTestOnMainThreadLoop [0x03794875+373] ChromeBrowserMainParts::PreMainMessageLoopRunImpl [0x043745DF+3887] ChromeBrowserMainParts::PreMainMessageLoopRun [0x043735EB+139] content::BrowserMainLoop::PreMainMessageLoopRun [0x027C8754+68] content::StartupTaskRunner::RunAllTasksNow [0x02A85394+36] content::BrowserMainLoop::CreateStartupTasks [0x027C73DA+554] content::BrowserMainRunnerImpl::Initialize [0x027CA826+102] content::BrowserMain [0x027C5A6A+138] content::RunBrowserProcessMain [0x03557D44+84] content::ContentMainRunnerImpl::Run [0x03558485+389] content::ContentServiceManagerMainDelegate::RunEmbedderProcess [0x03557A03+19] service_manager::Main [0x03C29132+354] content::ContentMain [0x03557C93+51] content::BrowserTestBase::SetUp [0x03794632+1554] [2540:8976:0827/042751.234:INFO:user_input_monitor_win.cc(170)] RegisterRawInputDevices() failed for RIDEV_REMOVE: The parameter is incorrect. (0x57) [1444:4352:0827/042751.238:ERROR:gles2_cmd_decoder_passthrough_doers.cc(4211)] NOT IMPLEMENTED [2540:8976:0827/042751.257:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown. [2540:8976:0827/042751.257:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.
,
Aug 27
,
Aug 27
I'm pretty sure the problem is somewhere here; I also suspect there may be several problems: 5c2de6b by magjed@webrtc.org on 2018-08-24 17:36 (local) Fix a bug in barcode_decoder.py 9bb55fc by magjed@webrtc.org on 2018-08-24 15:21 (local) Reland "Update video_quality_analysis to align videos instead of using barcodes" 3e169ac by magjed@webrtc.org on 2018-08-24 14:45 (local) Revert "Update video_quality_analysis to align videos instead of using barcodes" d65e143 by magjed@webrtc.org on 2018-08-24 11:26 (local) Update video_quality_analysis to align videos instead of using barcodes 40de15d by magjed@webrtc.org on 2018-08-23 19:51 (local) Android PeerConnectionFactory: Build without video codecs by default b2c0e8f by magjed@webrtc.org on 2018-08-23 17:35 (local) Add tool for aliging video files 1b5e5db by magjed@webrtc.org on 2018-08-23 13:04 (local) Remove SequencedTaskChecker from Y4mFileReader 404be7f by magjed@webrtc.org on 2018-08-23 11:56 (local) Add Y4mFileReader
,
Aug 27
for webrtc@523b4c43303951cff837bf0269bde00b7b48acbe (which is right before https://webrtc.googlesource.com/src.git/+/d65e143801a7aaa9affdb939ea836aec1955cdcc) after also re-building frame_analyzer, I'm getting a different error: Decoding barcodes from PNG files with /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/zxing... Generating stats file: stats_ref.txt Completed barcode decoding. Converting YUV file to PNG images (may take a while)... /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/ffmpeg -s 640x360 -i /tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv -f image2 -vcodec png /tmp/tmpia64t2/frame_%04d.png Decoding barcodes from PNG files with /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/zxing... Generating stats file: /tmp/.org.chromium.Chromium.ybH7GY/stats.txt Completed barcode decoding. [0827/171053.655860:ERROR:y4m_file_reader.cc(79)] File /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/reference_video_640x360_30fps.yuv does not start with YUV4MPEG2 header [0827/171053.655938:ERROR:y4m_file_reader.cc(79)] File /tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv does not start with YUV4MPEG2 header Error opening video files You have entered: chartjson_result_file=, height=360, label=360p_VP8, reference_file=/usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/reference_video_640x360_30fps.yuv, stats_file_ref=stats_ref.txt, stats_file_test=/tmp/.org.chromium.Chromium.ybH7GY/stats.txt, test_file=/tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv, width=640, [81147:81147:0827/171053.664575:ERROR:webrtc_video_quality_browsertest.cc(255)] Failed to compare videos; see output above to see what the error was. ../../chrome/browser/media/webrtc/webrtc_video_quality_browsertest.cc:316: Failure Value of: CompareVideosAndPrintResult( MakeLabel(test_config_.test_name, video_codec), test_config_.width, test_config_.height, GetWorkingDir().Append(kCapturedYuvFileName), test::GetReferenceFilesDir() .Append(test_config_.reference_video) .AddExtension(test::kYuvFileExtension), GetWorkingDir().Append(kStatsFileName)) Actual: false Expected: true Stack trace: #0 0x00000342bb6c testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop() #1 0x00000342b549 testing::internal::AssertHelper::operator=() #2 0x00000229020c WebRtcVideoQualityBrowserTest::TestVideoQuality() #3 0x00000228f49d WebRtcVideoQualityBrowserTest_MANUAL_TestVideoQualityVp8_Test::RunTestOnMainThread() #4 0x00000564a588 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop() #5 0x000005242bde ChromeBrowserMainParts::PreMainMessageLoopRunImpl() #6 0x000005241881 ChromeBrowserMainParts::PreMainMessageLoopRun() #7 0x000003f3f881 content::BrowserMainLoop::PreMainMessageLoopRun() #8 0x0000042bf877 content::StartupTaskRunner::RunAllTasksNow() #9 0x000003f3deed content::BrowserMainLoop::CreateStartupTasks() #10 0x000003f421b3 content::BrowserMainRunnerImpl::Initialize() #11 0x000003f3bd84 content::BrowserMain() #12 0x0000050a2d69 content::ContentMainRunnerImpl::Run() #13 0x000006bb338b service_manager::Main() #14 0x0000050a10d1 content::ContentMain() #15 0x00000564a17c content::BrowserTestBase::SetUp() #16 0x0000051ee4cb InProcessBrowserTest::SetUp() [81182:81182:0827/171054.020080:WARNING:x11_util.cc(1391)] X error received: serial 313, error_code 3 (BadWindow), request_code 4, minor_code 0 (Unknown) [81147:81147:0827/171054.051825:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown. [81147:81147:0827/171054.051892:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.
,
Aug 27
Ok, so webrtc@63af828a1baa96d67214861d7164b31c68b1f013 which is right before " Add Y4mFileReader" works if I don't forget to rebuild frame_analyzer. The problem is there are too many CLs landed here to revert. This needs to be fixed ASAP. +Kari for re-routing, since Magnus is OOO.
,
Aug 27
+Sami
,
Aug 27
I don't really know this code. I made a CL reverting all the CLs touching the code. If that doesn't help, I don't really know what to do. https://webrtc-review.googlesource.com/c/src/+/96320
,
Aug 27
I don't know how to start the auto-roller so that this gets rolled in also, could someone do that?
,
Aug 27
Thank you Sami! I'll restart auto-roller as soon as your CL lands.
,
Aug 27
The following revision refers to this bug: https://webrtc.googlesource.com/src.git/+/0673bc9204928f9c5566e8e816e22689c866681b commit 0673bc9204928f9c5566e8e816e22689c866681b Author: Sami Kalliomäki <sakal@webrtc.org> Date: Mon Aug 27 16:50:54 2018 Revert CLs affecting video quality toolchain. Speculatively fixes Chromium test for cut: crbug.com/877968 Reverts CLs: https://webrtc-review.googlesource.com/c/src/+/94772 https://webrtc-review.googlesource.com/c/src/+/95648 https://webrtc-review.googlesource.com/c/src/+/94773 https://webrtc-review.googlesource.com/c/src/+/96000 https://webrtc-review.googlesource.com/c/src/+/95949 Revert "Add Y4mFileReader" This reverts commit 404be7f302358e6be16aadeba8bc8f8aba348c0f. Revert "Remove SequencedTaskChecker from Y4mFileReader" This reverts commit 1b5e5db842971340eb9128985ddbaf0225a9d0b1. Revert "Add tool for aliging video files" This reverts commit b2c0e8f60fad10e2786e5e131136a0da1299d883. Revert "Reland "Update video_quality_analysis to align videos instead of using barcodes"" This reverts commit 9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a. Revert "Fix a bug in barcode_decoder.py" This reverts commit 5c2de6b3ce079cff52c411a2c02ce6553a38dc79. TBR=magjed@webrtc.org, phoglund@webrtc.org, phensman@webrtc.org Bug: chromium:877968 , webrtc:9642 Change-Id: I784d0598fd0370eec38d758b9fa0b38e4b3423be Reviewed-on: https://webrtc-review.googlesource.com/96320 Reviewed-by: Sami Kalliomäki <sakal@webrtc.org> Commit-Queue: Sami Kalliomäki <sakal@webrtc.org> Cr-Commit-Position: refs/heads/master@{#24458} [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/examples/androidtests/video_quality_loopback_test.py [add] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/resources/video_quality_analysis_frame.txt.sha1 [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/BUILD.gn [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/barcode_tools/barcode_decoder.py [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/frame_analyzer.cc [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.cc [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.h [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_unittest.cc [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.cc [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.h [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.cc [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.h [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner_unittest.cc [modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/psnr_ssim_analyzer/psnr_ssim_analyzer.cc [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.cc [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.h [delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader_unittest.cc
,
Aug 27
Rolls resumed
,
Aug 28
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/5e0dbfde1a29748372815765e1bae39526b003cc commit 5e0dbfde1a29748372815765e1bae39526b003cc Author: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Tue Aug 28 00:39:42 2018 Roll src/third_party/webrtc 1946a3f0fe2c..d8111e2169c9 (24 commits) https://webrtc.googlesource.com/src.git/+log/1946a3f0fe2c..d8111e2169c9 git log 1946a3f0fe2c..d8111e2169c9 --date=short --no-merges --format='%ad %ae %s' 2018-08-27 nisse@webrtc.org Delete unused class MockPeerConnection. 2018-08-27 sakal@webrtc.org Revert CLs affecting video quality toolchain. 2018-08-27 mhoro@webrtc.org Add rtt_mult_experiment to evaluate video robustness vs. latency 2018-08-27 ssilkin@webrtc.org Prevent duplicates in VP9 RTP p_diff list. 2018-08-27 mbonadei@webrtc.org Simplify assertion. 2018-08-27 titovartem@webrtc.org Rename VideoQualityTestFixtureInterface::Params.pipe into config. 2018-08-27 titovartem@webrtc.org Minor improve od documentation for network simulation. 2018-08-27 devicentepena@webrtc.org AEC3: audioproc_f: adding the read of the parameter fixed_capture_delay_samples 2018-08-27 alessiob@webrtc.org Limiter reset when fixed gain controller gain set. 2018-08-27 ilnik@webrtc.org Reland "Move FakeCodec to separate target and behave like real encoder." 2018-08-27 philipel@webrtc.org Convert VP8 descriptor to generic descriptor. 2018-08-27 peah@webrtc.org AEC3: Adaptive handling of echo path with strong high-frequency gain 2018-08-27 kthelgason@webrtc.org Remove deprecated type alias for RtpVideoCodecTypes. 2018-08-27 gustaf@webrtc.org AEC3: No ERLE uncertainty with diverged filter 2018-08-27 ssilkin@webrtc.org Add copy and move constructors to RateStatistics. 2018-08-27 nisse@webrtc.org Revert "Delete leftover includes and declarations for MediaConstraintsInterface" 2018-08-27 nisse@webrtc.org Add lock annotations in StreamStatisticianImpl 2018-08-27 nisse@webrtc.org Delete leftover includes and declarations for MediaConstraintsInterface 2018-08-27 ilnik@webrtc.org Add experiment for boosted qp at lowest stream for screenshare 2018-08-27 nisse@webrtc.org Delete unused AudioCodingModule methods. 2018-08-27 nisse@webrtc.org Delete AudioCodingModule::LeastRequiredDelayMs and related NetEq code. 2018-08-27 ilnik@webrtc.org Revert "Reland "Move FakeCodec to separate target and behave like real encoder."" 2018-08-27 kwiberg@webrtc.org Add a list of allowed and disallowed Abseil things 2018-08-27 srte@webrtc.org Adds debug printing of units in unit tests. Created with: gclient setdep -r src/third_party/webrtc@d8111e2169c9 The AutoRoll server is located here: https://autoroll.skia.org/r/webrtc-chromium-autoroll Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. CQ_INCLUDE_TRYBOTS=luci.chromium.try:linux_chromium_archive_rel_ng;master.tryserver.chromium.mac:mac_chromium_archive_rel_ng BUG=chromium:None,chromium:877968,chromium:None,chromium:none,chromium:none,chromium:None,chromium:None,chromium:none TBR=webrtc-chromium-sheriffs-robots@google.com Change-Id: I844e3f663773208f386748fb63448a8e2da819e4 Reviewed-on: https://chromium-review.googlesource.com/1192182 Reviewed-by: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#586524} [modify] https://crrev.com/5e0dbfde1a29748372815765e1bae39526b003cc/DEPS
,
Aug 28
Thanks Sami - all the bots a green now.
,
Sep 11
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by olka@chromium.org
, Aug 27