New issue
Advanced search Search tips

Issue 606765 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android , Mac
Pri: 2
Type: Bug-Regression



Sign in to add a comment

3.8% regression in webrtc_perf_tests at 12491:12491

Project Member Reported by hlundin@chromium.org, Apr 26 2016

Issue description

This is caused by https://chromium.googlesource.com/external/webrtc/+/97ba30eedf99e84de33200b7546d5d578fe358e6. The same regression is seen on other platforms as well, but they haven't fired any alerts (yet).
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=606765

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg5LDxrgoM


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus5
Cc: ossu@webrtc.org peah@chromium.org
Labels: -M-50 M-52 OS-Android OS-Linux OS-Mac
ossu@, please, take a look. Is it reasonable that your change causes this regression?

Comment 3 by ossu@webrtc.org, Apr 26 2016

I will take a look. It really shouldn't, since I've tried to avoid changing the implementation when converting to C++, but perhaps something else has snuck in.

Comment 4 by ossu@webrtc.org, Apr 26 2016

I think I've pinned it down to the calls to DecoderDatabase::IsComfortNoise that were added to distinguish between a missing decoder and a null AudioDecoder because comfort noise is no longer an AudioDecoder.

On my machine, running NetEqPerformanceTest.RunClean takes around 2900 ms before any of my changes, and closer to 3000 ms after. The differences aren't huge compared to the variance between runs, but with my changes, I cannot get it to run in less than ~2950 ms, before it would dip below 2900 ms.

Anyhow: IsComfortNoise, in turn, calls IsType four times, each with a different possible variant of CNG that it could be (narrowband, wideband, etc.). IsType does a std::map::find() to search for a payload type (the key).

Since NetEqPerformanceTest doesn't involve  CNG, I tried having IsComfortNoise just return false and I got measurements below 2900 ms. After adding an atomic counter into IsType, I see the following numbers:
- Before any of my changes (pre-CL), IsType gets called 17666480 times.
- After applying what is essentially the first patch set of my CL, IsType gets called 37666192 times - slightly more than double.
- Short-circuiting IsComfortNoise to just return false, IsType gets called 9666524 times.

The test seems to simulate 10 million millseconds of audio, so IsType gets called on average roughly four times, twice and once per millisecond of audio. Even the old ways seem a bit excessive, though a good start would probably be to do just one lookup of the type, then comparing the found data (if any) against all four CNG types, rather than have one separate lookup per CNG type.

It should probably be added that this is for a std::map with just one member in it, at the moment of testing.


Thanks for investigating! I'll noodle on if I should make a fix, or just call it "acceptable losses".

Comment 6 by ossu@webrtc.org, Apr 27 2016

I have a simple fix that changes the four IsType in IsComfortNoise to just one map lookup. It seems to bring the measurements in line with before the CNG CL. It actually shaves about one million calls off of that value. I'll put it up shortly.

Comment 7 by ossu@webrtc.org, Apr 27 2016

Proposed a fix here: https://codereview.webrtc.org/1923763003/
Status: Fixed (was: Assigned)
Fix landed in WebRTC 12533 (https://crrev.com/6353723b029de581b1d0393ddf49c79f34b8f37c).
The graphs seem to have recovered (more or less).

Sign in to add a comment