New issue
Advanced search Search tips

Issue 689919 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Closed: Mar 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

9.3%-114.5% regression in webrtc_perf_tests at 16425:16428

Project Member Reported by hlundin@chromium.org, Feb 8 2017

Issue description

Main culprit pointed to by perf tool is https://chromium.googlesource.com/external/webrtc/+/5f4712686550ba1d069c5e4c456ffcabe7ccba97 by ilnik@.

Could possibly also be https://chromium.googlesource.com/external/webrtc/+/77ce9a55415a673422d424ed862be142d5e277ef by henrika@.

I looked at the corresponding graphs for Linux and Windows, and they did not exhibit the same regression.
 

Comment 2 by ilnik@chromium.org, Feb 8 2017

Cc: ilnik@chromium.org
Owner: henrika@chromium.org
My change should only affect full_stack_video tests and are OS independent.

Since broken tests are audio tests I believe commit by henrika@ may be the main culprit, as it affects audio processing.
hlundin@: what does these tests measure?

My CL does not touch the audio path at all. It only moves some stat calculations used for logging
from a task queue into audiocallbacks. And the calculations are trivial (only counters).

In addition, my changes are platform agnostic and should have the same effect on all platforms if any effect existed.

Not sure where to go from here.
hlundin@: my CL landed in 16429 and judging from the graph that is after the jump. Do you agree?
hlundin@: if I speculatively revert my change. Would that show up directly in the perf tool bot?

Comment 6 by ilnik@chromium.org, Feb 8 2017

Henrika@: From what I see, you introduced some locks and stats calculations in the audio device buffer. This might cause change in the tests. Especially, if tests did not count logging time before. Since increase is not huge in absolute values (in some tests its about 1us), it may not be a regression at all but just change in the way metrics are calculated because some code now is executed in the different thread.
Re #5: Yes, a speculative revert would show up pretty soon. Probably within hours.
I have committed a revert here:

https://codereview.webrtc.org/2684913003/

Please note that my change should only improve things since I have shown with profiling that the old code added overhead by calling PostTask on a task queue for each audio packet and that does come with a cost. I am surprised if the limited usage of locks here would have a negative impact.

Let's wait with all the details until the revert has landed. In any case, the intention of my change was to do good.
Cc: -hlundin@chromium.org henrika@chromium.org
Owner: hlundin@chromium.org
Henrik L: could you help me keep track and report back what the effect of my revert was. Thanks.
I will look at that. Thanks!

Cr-Commit-Position: refs/heads/master@{#16490}
Henrik A: You are off the hook. It looks like your CL is not to blame. Feel free to re-land.
Reverted the revert. My CL is in again.
Should I revert my commit?
It makes even less sense, as it touches only video performance tests.
Reverted my CL. Let's see.
Cc: -ilnik@chromium.org hlundin@chromium.org
Owner: ilnik@chromium.org
Revert broke some other tests, so I relanded changes later. But it looks like it really was my commit.
I Will look into that further.

Comment 19 by ilnik@chromium.org, Feb 13 2017

Cc: ilnik@chromium.org
Owner: solenberg@chromium.org
I found the broken tests in /webrtc/modules/audio_processing/audio_processing_performance_unnitest.cc

All my changes are contained within VideoQualityTest which is not referenced or used at all anywhere outside of video performance test. My changes are linked in the webrtc_perf_tests but aren't called at all. My code just couldn't cause regression unless there are some general problems like executable became too big or some other Mac magic.

solenberg@ could you please try to reproduce the bug?
Cc: solenberg@chromium.org
Owner: peah@chromium.org
Passing the hot potato to peah@, who wrote audio_processing_performance_unittest.cc.

Labels: -Restrict-View-Google
Removing R-V-G label (was set upon bug filing because the bot was incorrectly configured as internal)

Comment 22 by peah@chromium.org, Mar 6 2017

Status: WontFix (was: Assigned)
It does not make sense that these tests should fail due to that. But the size of the regression is so small so that it is not a concern.

Closing as WontFix. 

Sign in to add a comment