New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 877968 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit 15 days ago
Closed: Aug 28
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Mac
Pri: 1
Type: Bug



Sign in to add a comment

WebRtcVideoQualityBrowserTest.MANUAL_TestVideoQuality* tests broken on chromium.webrc

Project Member Reported by olka@chromium.org, Aug 27

Issue description

Smallest among first failing builds: https://ci.chromium.org/buildbot/chromium.webrtc/Win10%20Tester/31493


WebRTC roll in that range: https://webrtc.googlesource.com/src.git/+log/efbcb31cb67e..1946a3f0fe2c


Example Log:
Completed barcode decoding.
[0827/042751.062:ERROR:y4m_file_reader.cc(79)] File C:\b\c\b\Win10_Tester\src\chrome\test\data\webrtc/resources\reference_video_1280x720_30fps.yuv does not start with YUV4MPEG2 header
[0827/042751.062:ERROR:y4m_file_reader.cc(79)] File C:\Users\CHROME~1\AppData\Local\Temp\scoped_dir2540_9985\captured_video.yuv does not start with YUV4MPEG2 header
Error opening video files
Flag 'stats_file_ref' is not recognized
Flag 'stats_file_test' is not recognized
Flag 'width' is not recognized
Flag 'height' is not recognized
You have entered:
chartjson_result_file=, label=720p_VP9, reference_file=C:\b\c\b\Win10_Tester\src\chrome\test\data\webrtc/resources\reference_video_1280x720_30fps.yuv, test_file=C:\Users\CHROME~1\AppData\Local\Temp\scoped_dir2540_9985\captured_video.yuv, 
../../chrome/browser/media/webrtc/webrtc_video_quality_browsertest.cc(316): error: Value of: CompareVideosAndPrintResult( MakeLabel(test_config_.test_name, video_codec), test_config_.width, test_config_.height, GetWorkingDir().Append(kCapturedYuvFileName), test::GetReferenceFilesDir() .Append(test_config_.reference_video) .AddExtension(test::kYuvFileExtension), GetWorkingDir().Append(kStatsFileName))
  Actual: false
Expected: true
Stack trace:
Backtrace:
	StackTraceGetter::CurrentStackTrace [0x01E97E48+40]
	testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop [0x01EA08A5+69]
	testing::internal::AssertHelper::operator= [0x01EA04F0+48]
	WebRtcVideoQualityBrowserTest::TestVideoQuality [0x01419A13+2435]
	WebRtcVideoQualityBrowserTest_MANUAL_TestVideoQualityVp9_Test::RunTestOnMainThread [0x01419B3E+62]
	content::BrowserTestBase::ProxyRunTestOnMainThreadLoop [0x03794875+373]
	ChromeBrowserMainParts::PreMainMessageLoopRunImpl [0x043745DF+3887]
	ChromeBrowserMainParts::PreMainMessageLoopRun [0x043735EB+139]
	content::BrowserMainLoop::PreMainMessageLoopRun [0x027C8754+68]
	content::StartupTaskRunner::RunAllTasksNow [0x02A85394+36]
	content::BrowserMainLoop::CreateStartupTasks [0x027C73DA+554]
	content::BrowserMainRunnerImpl::Initialize [0x027CA826+102]
	content::BrowserMain [0x027C5A6A+138]
	content::RunBrowserProcessMain [0x03557D44+84]
	content::ContentMainRunnerImpl::Run [0x03558485+389]
	content::ContentServiceManagerMainDelegate::RunEmbedderProcess [0x03557A03+19]
	service_manager::Main [0x03C29132+354]
	content::ContentMain [0x03557C93+51]
	content::BrowserTestBase::SetUp [0x03794632+1554]
[2540:8976:0827/042751.234:INFO:user_input_monitor_win.cc(170)] RegisterRawInputDevices() failed for RIDEV_REMOVE: The parameter is incorrect. (0x57)
[1444:4352:0827/042751.238:ERROR:gles2_cmd_decoder_passthrough_doers.cc(4211)] NOT IMPLEMENTED
[2540:8976:0827/042751.257:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.
[2540:8976:0827/042751.257:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.
 
Cc: mbonadei@chromium.org
Rolls paused.

I've been trying to bisect, but the tests started to fail locally for me even on previously successful builds.
Cc: guidou@chromium.org
I'm pretty sure the problem is somewhere here; I also suspect there may be several problems:

5c2de6b	by	magjed@webrtc.org	
on 2018-08-24 17:36 (local)
Fix a bug in barcode_decoder.py
9bb55fc	by	magjed@webrtc.org	
on 2018-08-24 15:21 (local)
Reland "Update video_quality_analysis to align videos instead of using barcodes"
3e169ac	by	magjed@webrtc.org	
on 2018-08-24 14:45 (local)
Revert "Update video_quality_analysis to align videos instead of using barcodes"
d65e143	by	magjed@webrtc.org	
on 2018-08-24 11:26 (local)
Update video_quality_analysis to align videos instead of using barcodes
40de15d	by	magjed@webrtc.org	
on 2018-08-23 19:51 (local)
Android PeerConnectionFactory: Build without video codecs by default
b2c0e8f	by	magjed@webrtc.org	
on 2018-08-23 17:35 (local)
Add tool for aliging video files
1b5e5db	by	magjed@webrtc.org	
on 2018-08-23 13:04 (local)
Remove SequencedTaskChecker from Y4mFileReader
404be7f	by	magjed@webrtc.org	
on 2018-08-23 11:56 (local)
Add Y4mFileReader
for webrtc@523b4c43303951cff837bf0269bde00b7b48acbe (which is right before https://webrtc.googlesource.com/src.git/+/d65e143801a7aaa9affdb939ea836aec1955cdcc) after also re-building frame_analyzer, I'm getting a different error:

Decoding barcodes from PNG files with /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/zxing...
Generating stats file: stats_ref.txt
Completed barcode decoding.
Converting YUV file to PNG images (may take a while)...
/usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/ffmpeg -s 640x360 -i /tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv -f image2 -vcodec png /tmp/tmpia64t2/frame_%04d.png
Decoding barcodes from PNG files with /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/tools/linux/zxing...
Generating stats file: /tmp/.org.chromium.Chromium.ybH7GY/stats.txt
Completed barcode decoding.
[0827/171053.655860:ERROR:y4m_file_reader.cc(79)] File /usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/reference_video_640x360_30fps.yuv does not start with YUV4MPEG2 header
[0827/171053.655938:ERROR:y4m_file_reader.cc(79)] File /tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv does not start with YUV4MPEG2 header
Error opening video files
You have entered:
chartjson_result_file=, height=360, label=360p_VP8, reference_file=/usr/local/google/home/olka/chromium-int/src/chrome/test/data/webrtc/resources/reference_video_640x360_30fps.yuv, stats_file_ref=stats_ref.txt, stats_file_test=/tmp/.org.chromium.Chromium.ybH7GY/stats.txt, test_file=/tmp/.org.chromium.Chromium.ybH7GY/captured_video.yuv, width=640, 

[81147:81147:0827/171053.664575:ERROR:webrtc_video_quality_browsertest.cc(255)] Failed to compare videos; see output above to see what the error was.
../../chrome/browser/media/webrtc/webrtc_video_quality_browsertest.cc:316: Failure
Value of: CompareVideosAndPrintResult( MakeLabel(test_config_.test_name, video_codec), test_config_.width, test_config_.height, GetWorkingDir().Append(kCapturedYuvFileName), test::GetReferenceFilesDir() .Append(test_config_.reference_video) .AddExtension(test::kYuvFileExtension), GetWorkingDir().Append(kStatsFileName))
  Actual: false
Expected: true
Stack trace:
#0 0x00000342bb6c testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop()
#1 0x00000342b549 testing::internal::AssertHelper::operator=()
#2 0x00000229020c WebRtcVideoQualityBrowserTest::TestVideoQuality()
#3 0x00000228f49d WebRtcVideoQualityBrowserTest_MANUAL_TestVideoQualityVp8_Test::RunTestOnMainThread()
#4 0x00000564a588 content::BrowserTestBase::ProxyRunTestOnMainThreadLoop()
#5 0x000005242bde ChromeBrowserMainParts::PreMainMessageLoopRunImpl()
#6 0x000005241881 ChromeBrowserMainParts::PreMainMessageLoopRun()
#7 0x000003f3f881 content::BrowserMainLoop::PreMainMessageLoopRun()
#8 0x0000042bf877 content::StartupTaskRunner::RunAllTasksNow()
#9 0x000003f3deed content::BrowserMainLoop::CreateStartupTasks()
#10 0x000003f421b3 content::BrowserMainRunnerImpl::Initialize()
#11 0x000003f3bd84 content::BrowserMain()
#12 0x0000050a2d69 content::ContentMainRunnerImpl::Run()
#13 0x000006bb338b service_manager::Main()
#14 0x0000050a10d1 content::ContentMain()
#15 0x00000564a17c content::BrowserTestBase::SetUp()
#16 0x0000051ee4cb InProcessBrowserTest::SetUp()

[81182:81182:0827/171054.020080:WARNING:x11_util.cc(1391)] X error received: serial 313, error_code 3 (BadWindow), request_code 4, minor_code 0 (Unknown)
[81147:81147:0827/171054.051825:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.
[81147:81147:0827/171054.051892:WARNING:pref_notifier_impl.cc(23)] Pref observer found at shutdown.

Owner: kthelgason@chromium.org
Ok, so webrtc@63af828a1baa96d67214861d7164b31c68b1f013 which is right before "
Add Y4mFileReader" works if I don't forget to rebuild frame_analyzer.

The problem is there are too many CLs landed here to revert. This needs to be fixed ASAP.

+Kari for re-routing, since Magnus is OOO.
Cc: sakal@chromium.org
+Sami
I don't really know this code. I made a CL reverting all the CLs touching the code. If that doesn't help, I don't really know what to do.

https://webrtc-review.googlesource.com/c/src/+/96320
I don't know how to start the auto-roller so that this gets rolled in also, could someone do that?
Owner: olka@chromium.org
Thank you Sami! I'll restart auto-roller as soon as your CL lands.
Project Member

Comment 10 by bugdroid1@chromium.org, Aug 27

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/0673bc9204928f9c5566e8e816e22689c866681b

commit 0673bc9204928f9c5566e8e816e22689c866681b
Author: Sami Kalliomäki <sakal@webrtc.org>
Date: Mon Aug 27 16:50:54 2018

Revert CLs affecting video quality toolchain.

Speculatively fixes Chromium test for cut:  crbug.com/877968 

Reverts CLs:
https://webrtc-review.googlesource.com/c/src/+/94772
https://webrtc-review.googlesource.com/c/src/+/95648
https://webrtc-review.googlesource.com/c/src/+/94773
https://webrtc-review.googlesource.com/c/src/+/96000
https://webrtc-review.googlesource.com/c/src/+/95949

Revert "Add Y4mFileReader"

This reverts commit 404be7f302358e6be16aadeba8bc8f8aba348c0f.

Revert "Remove SequencedTaskChecker from Y4mFileReader"

This reverts commit 1b5e5db842971340eb9128985ddbaf0225a9d0b1.

Revert "Add tool for aliging video files"

This reverts commit b2c0e8f60fad10e2786e5e131136a0da1299d883.

Revert "Reland "Update video_quality_analysis to align videos instead of using barcodes""

This reverts commit 9bb55fc09b6bfa00cba7779c37ad6c39b4206f7a.

Revert "Fix a bug in barcode_decoder.py"

This reverts commit 5c2de6b3ce079cff52c411a2c02ce6553a38dc79.

TBR=magjed@webrtc.org, phoglund@webrtc.org, phensman@webrtc.org

Bug:  chromium:877968 , webrtc:9642
Change-Id: I784d0598fd0370eec38d758b9fa0b38e4b3423be
Reviewed-on: https://webrtc-review.googlesource.com/96320
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Commit-Queue: Sami Kalliomäki <sakal@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24458}
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/examples/androidtests/video_quality_loopback_test.py
[add] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/resources/video_quality_analysis_frame.txt.sha1
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/BUILD.gn
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/barcode_tools/barcode_decoder.py
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/frame_analyzer.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_lib.h
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/reference_less_video_analysis_unittest.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis.h
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/frame_analyzer/video_quality_analysis_unittest.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner.h
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/frame_analyzer/video_temporal_aligner_unittest.cc
[modify] https://crrev.com/0673bc9204928f9c5566e8e816e22689c866681b/rtc_tools/psnr_ssim_analyzer/psnr_ssim_analyzer.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.cc
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader.h
[delete] https://crrev.com/f9fc171568ecd6308649405a814146913f38955e/rtc_tools/y4m_file_reader_unittest.cc

Rolls resumed
Project Member

Comment 12 by bugdroid1@chromium.org, Aug 28

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5e0dbfde1a29748372815765e1bae39526b003cc

commit 5e0dbfde1a29748372815765e1bae39526b003cc
Author: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Aug 28 00:39:42 2018

Roll src/third_party/webrtc 1946a3f0fe2c..d8111e2169c9 (24 commits)

https://webrtc.googlesource.com/src.git/+log/1946a3f0fe2c..d8111e2169c9


git log 1946a3f0fe2c..d8111e2169c9 --date=short --no-merges --format='%ad %ae %s'
2018-08-27 nisse@webrtc.org Delete unused class MockPeerConnection.
2018-08-27 sakal@webrtc.org Revert CLs affecting video quality toolchain.
2018-08-27 mhoro@webrtc.org Add rtt_mult_experiment to evaluate video robustness vs. latency
2018-08-27 ssilkin@webrtc.org Prevent duplicates in VP9 RTP p_diff list.
2018-08-27 mbonadei@webrtc.org Simplify assertion.
2018-08-27 titovartem@webrtc.org Rename VideoQualityTestFixtureInterface::Params.pipe into config.
2018-08-27 titovartem@webrtc.org Minor improve od documentation for network simulation.
2018-08-27 devicentepena@webrtc.org AEC3: audioproc_f: adding the read of the parameter fixed_capture_delay_samples
2018-08-27 alessiob@webrtc.org Limiter reset when fixed gain controller gain set.
2018-08-27 ilnik@webrtc.org Reland "Move FakeCodec to separate target and behave like real encoder."
2018-08-27 philipel@webrtc.org Convert VP8 descriptor to generic descriptor.
2018-08-27 peah@webrtc.org AEC3: Adaptive handling of echo path with strong high-frequency gain
2018-08-27 kthelgason@webrtc.org Remove deprecated type alias for RtpVideoCodecTypes.
2018-08-27 gustaf@webrtc.org AEC3: No ERLE uncertainty with diverged filter
2018-08-27 ssilkin@webrtc.org Add copy and move constructors to RateStatistics.
2018-08-27 nisse@webrtc.org Revert "Delete leftover includes and declarations for MediaConstraintsInterface"
2018-08-27 nisse@webrtc.org Add lock annotations in StreamStatisticianImpl
2018-08-27 nisse@webrtc.org Delete leftover includes and declarations for MediaConstraintsInterface
2018-08-27 ilnik@webrtc.org Add experiment for boosted qp at lowest stream for screenshare
2018-08-27 nisse@webrtc.org Delete unused AudioCodingModule methods.
2018-08-27 nisse@webrtc.org Delete AudioCodingModule::LeastRequiredDelayMs and related NetEq code.
2018-08-27 ilnik@webrtc.org Revert "Reland "Move FakeCodec to separate target and behave like real encoder.""
2018-08-27 kwiberg@webrtc.org Add a list of allowed and disallowed Abseil things
2018-08-27 srte@webrtc.org Adds debug printing of units in unit tests.


Created with:
  gclient setdep -r src/third_party/webrtc@d8111e2169c9

The AutoRoll server is located here: https://autoroll.skia.org/r/webrtc-chromium-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:linux_chromium_archive_rel_ng;master.tryserver.chromium.mac:mac_chromium_archive_rel_ng

BUG=chromium:None,chromium:877968,chromium:None,chromium:none,chromium:none,chromium:None,chromium:None,chromium:none
TBR=webrtc-chromium-sheriffs-robots@google.com

Change-Id: I844e3f663773208f386748fb63448a8e2da819e4
Reviewed-on: https://chromium-review.googlesource.com/1192182
Reviewed-by: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: webrtc-chromium-autoroll <webrtc-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#586524}
[modify] https://crrev.com/5e0dbfde1a29748372815765e1bae39526b003cc/DEPS

Owner: sakal@chromium.org
Status: Fixed (was: Assigned)
Thanks Sami - all the bots a green now.
Components: Blink>WebRTC>Video

Sign in to add a comment