New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 5 users
Status: Archived
Owner:
Closed: Aug 4
Cc:
Components:
NextAction: ----
OS: Windows , Mac
Pri: 2
Type: Bug



Sign in to add a comment
VideoCaptureExternalTest.FrameRate flaky on Windows and Mac
Project Member Reported by pbos@webrtc.org, Apr 28 2014 Back to list
Project Member Comment 1 by bugdroid1@chromium.org, Apr 29 2014
The following revision refers to this bug:
  http://code.google.com/p/webrtc/source/detail?r=5994

------------------------------------------------------------------
r5994 | pbos@webrtc.org | 2014-04-28T11:27:36.047002Z

Changed paths:
   M http://code.google.com/p/webrtc/source/diff?path=/trunk/webrtc/modules/video_capture/test/video_capture_unittest.cc&spec=svn5994&r_previous=5993&r=5994&format=side

Disable capture test for FrameRate on Windows.

Flaky on Windows, has been for a while.

R=kjellander@webrtc.org
TBR=mflodman@webrtc.org
BUG= 3270 

Review URL: https://webrtc-codereview.appspot.com/19389004
-----------------------------------------------------------------
Comment 2 by vrk@webrtc.org, Oct 14 2014
Labels: Area-Video
Project Member Comment 3 by tnakamura@webrtc.org, Nov 4 2015
This bug hasn't been modified for more than a year. Is this still a valid open issue?
Project Member Comment 4 by mflodman@webrtc.org, Mar 1 2016
Cc: mflodman@webrtc.org
Labels: EngTriaged
Owner: sprang@webrtc.org
Project Member Comment 5 by kjellander@webrtc.org, Mar 30 2016
Summary: VideoCaptureExternalTest.FrameRate flaky on Windows and Mac (was: VideoCaptureExternalTest.FrameRate flaky on Windows)
In  bug 5709 , jbauch@ reported new issues similar to this one.

I suggest we:
1) Change EXPECT_TRUE to EXPECT_GT and EXPECT_LT so we can see what the values are when it fails (at https://code.google.com/p/chromium/codesearch#chromium/src/third_party/webrtc/modules/video_capture/test/video_capture_unittest.cc&l=509)
2) Increase the tolerance to reduce flakiness (possibly after gathering more data after submitting the above).
Project Member Comment 6 by kjellander@webrtc.org, Mar 30 2016
Labels: OS-Windows OS-Mac
Project Member Comment 7 by kjellander@webrtc.org, Mar 30 2016
Cc: kjellander@webrtc.org
 Issue 5709  has been merged into this issue.
Project Member Comment 8 by henrik.lundin@webrtc.org, Apr 1 2016
This is still flaking on Mac, since it was not disabled on that platform. Preparing a CL to disable on Mac too.

Latest failing run:

https://uberchromegw.corp.google.com/i/client.webrtc/builders/Mac64%20Release%20%5Blarge%20tests%5D/builds/7125/steps/video_capture_tests/logs/stdio

[ RUN      ] VideoCaptureExternalTest.FrameRate
../../webrtc/modules/video_capture/test/video_capture_unittest.cc:529: Failure
Value of: capture_feedback_.frame_rate() >= 25 && capture_feedback_.frame_rate() <= 33
  Actual: false
Expected: true
No of timing warnings 86
[  FAILED  ] VideoCaptureExternalTest.FrameRate (6549 ms)


Project Member Comment 9 by pbos@webrtc.org, Apr 1 2016
If we want to keep this test I think it makes more sense to make this a perf test that reports RESULT= rather than have it flake on thresholds not being large enough.
Project Member Comment 10 by bugdroid1@chromium.org, Apr 1 2016
The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/cfebcca51bba77949b2c5a1fdddca5e17d8e993f

commit cfebcca51bba77949b2c5a1fdddca5e17d8e993f
Author: henrik.lundin <henrik.lundin@webrtc.org>
Date: Fri Apr 01 10:23:14 2016

Disable VideoCaptureExternalTest.FrameRate on Mac

The test is flaky.

BUG= webrtc:3270 
TBR=mflodman@webrtc.org
NOTRY=true

Review URL: https://codereview.webrtc.org/1851023002

Cr-Commit-Position: refs/heads/master@{#12196}

[modify] https://crrev.com/cfebcca51bba77949b2c5a1fdddca5e17d8e993f/webrtc/modules/video_capture/test/video_capture_unittest.cc

Project Member Comment 11 by mflodman@webrtc.org, Aug 4
Status: Archived
[Bulk edit] This issue was created more than a year ago and hasn't been modified the last six months -> archiving.

If this is still a valid issue that should be open, please reopen again.
Sign in to add a comment