New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 713507 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

109.7% regression in webrtc_perf_tests at 17661:17661

Project Member Reported by peah@chromium.org, Apr 20 2017

Issue description

See the link to graphs below.
 

Comment 1 by peah@chromium.org, Apr 20 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=713507

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggrT6tgoM


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus9

Comment 2 by peah@chromium.org, Apr 20 2017

Owner: peah@chromium.org
Closing as WontFix as the values have recovered to what they were before.

Comment 3 by peah@chromium.org, Apr 20 2017

Status: WontFix (was: Untriaged)

Comment 4 by ivoc@chromium.org, May 8 2017

Cc: ivoc@chromium.org
Status: Assigned (was: WontFix)
Hi Per, I attached a new alert to this bug, since the values have now again shifted to a different level without a clear cause. Could you have another look at this? 
Do you know what could cause these changes? Is the test just flaky or unreliable?
Looks like it got better again, and just now got worse again.

Comment 6 by peah@chromium.org, May 26 2017

Based on the newly perceived change, and on similar issues in the past, I think it is clear that these tests are flaky. I'll address that in a CL.
Project Member

Comment 7 by bugdroid1@chromium.org, Jul 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/4235d78b574aa2923b2e01a9ab35873544b33dfe

commit 4235d78b574aa2923b2e01a9ab35873544b33dfe
Author: peah <peah@webrtc.org>
Date: Mon Jul 03 16:11:22 2017

Disabling flaky complexity tests for the audio processing module.

The complexity test for the audio processing module have long proven
to give false alarms of complexity regressions for which no related
changes can be identified. Attempts to address that has improved the
that, but the tests do still give false alarms.

This CL deactivates the complexity tests until a better way of
testing this is available.

BUG= chromium:713507 , webrtc:5846,webrtc:6685,webrtc:7712

Review-Url: https://codereview.webrtc.org/2897403006
Cr-Commit-Position: refs/heads/master@{#18879}

[modify] https://crrev.com/4235d78b574aa2923b2e01a9ab35873544b33dfe/webrtc/modules/audio_processing/audio_processing_performance_unittest.cc
[modify] https://crrev.com/4235d78b574aa2923b2e01a9ab35873544b33dfe/webrtc/modules/audio_processing/level_controller/level_controller_complexity_unittest.cc

Comment 8 by peah@chromium.org, Oct 24 2017

Status: Fixed (was: Assigned)

Sign in to add a comment