New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 673683 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Last visit > 30 days ago
Closed: Dec 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

24%-31.1% regression in webrtc_perf_tests at 15441:15511

Project Member Reported by brandtr@chromium.org, Dec 13 2016

Issue description

See the link to graphs below.
 
Owner: ivoc@chromium.org
Status: Available (was: Untriaged)
Hey ivoc,

Would you mind having a look at this alert? I can't find a reasonable culprit CL in the blamelist, and I think you know more about this test than I do :)

Note that all of the alerts are on the Nexus9 bot, so it may very well be a flakey result. (This should not be one of the bots with new devices, as per kjellander's email to the webrtc-perf-sheriffs email list.)

Comment 3 by ivoc@chromium.org, Dec 13 2016

Status: Started (was: Available)
Hmm, that is very strange, the code that's used in this test did not change at all, so I don't see how this can possibly be caused by any changes to our code. I think there is a problem with these tests that causes them to be unreliable (Per had similar problems with the level estimator perf tests). I will try to make some changes to the way these tests work to improve their reliability.
Thanks for looking into it! I have added some more alerts that came in today to the group report.
Project Member

Comment 5 by bugdroid1@chromium.org, Dec 14 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/930959d261451af05c31db944ecc440808eb89f8

commit 930959d261451af05c31db944ecc440808eb89f8
Author: ivoc <ivoc@webrtc.org>
Date: Wed Dec 14 13:48:16 2016

Improvements to the reliability of the echo detector perf test.

BUG= chromium:673683 

Review-Url: https://codereview.webrtc.org/2568883004
Cr-Commit-Position: refs/heads/master@{#15606}

[modify] https://crrev.com/930959d261451af05c31db944ecc440808eb89f8/webrtc/modules/audio_processing/residual_echo_detector_complexity_unittest.cc

Comment 6 by ivoc@chromium.org, Dec 14 2016

I hope this change will make the perf stats more robust, however, since this modifies the behavior of the stats it's expected to see a lot of alerts popping up. 
Status: Verified (was: Started)

Sign in to add a comment