3.8% regression in webrtc_perf_tests at 12491:12491 |
|||
Issue descriptionThis is caused by https://chromium.googlesource.com/external/webrtc/+/97ba30eedf99e84de33200b7546d5d578fe358e6. The same regression is seen on other platforms as well, but they haven't fired any alerts (yet).
,
Apr 26 2016
ossu@, please, take a look. Is it reasonable that your change causes this regression?
,
Apr 26 2016
I will take a look. It really shouldn't, since I've tried to avoid changing the implementation when converting to C++, but perhaps something else has snuck in.
,
Apr 26 2016
I think I've pinned it down to the calls to DecoderDatabase::IsComfortNoise that were added to distinguish between a missing decoder and a null AudioDecoder because comfort noise is no longer an AudioDecoder. On my machine, running NetEqPerformanceTest.RunClean takes around 2900 ms before any of my changes, and closer to 3000 ms after. The differences aren't huge compared to the variance between runs, but with my changes, I cannot get it to run in less than ~2950 ms, before it would dip below 2900 ms. Anyhow: IsComfortNoise, in turn, calls IsType four times, each with a different possible variant of CNG that it could be (narrowband, wideband, etc.). IsType does a std::map::find() to search for a payload type (the key). Since NetEqPerformanceTest doesn't involve CNG, I tried having IsComfortNoise just return false and I got measurements below 2900 ms. After adding an atomic counter into IsType, I see the following numbers: - Before any of my changes (pre-CL), IsType gets called 17666480 times. - After applying what is essentially the first patch set of my CL, IsType gets called 37666192 times - slightly more than double. - Short-circuiting IsComfortNoise to just return false, IsType gets called 9666524 times. The test seems to simulate 10 million millseconds of audio, so IsType gets called on average roughly four times, twice and once per millisecond of audio. Even the old ways seem a bit excessive, though a good start would probably be to do just one lookup of the type, then comparing the found data (if any) against all four CNG types, rather than have one separate lookup per CNG type. It should probably be added that this is for a std::map with just one member in it, at the moment of testing.
,
Apr 27 2016
Thanks for investigating! I'll noodle on if I should make a fix, or just call it "acceptable losses".
,
Apr 27 2016
I have a simple fix that changes the four IsType in IsComfortNoise to just one map lookup. It seems to bring the measurements in line with before the CNG CL. It actually shaves about one million calls off of that value. I'll put it up shortly.
,
Apr 27 2016
Proposed a fix here: https://codereview.webrtc.org/1923763003/
,
Apr 29 2016
Fix landed in WebRTC 12533 (https://crrev.com/6353723b029de581b1d0393ddf49c79f34b8f37c). The graphs seem to have recovered (more or less). |
|||
►
Sign in to add a comment |
|||
Comment 1 by hlundin@chromium.org
, Apr 26 2016