New issue
Advanced search Search tips

Issue 635886 link

Starred by 3 users

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

22.1% regression in webrtc_perf_tests at 13635:13637

Project Member Reported by peah@chromium.org, Aug 9 2016

Issue description

There is a 22.1 % regression in the ramp_up_down_3streams_rtc/second_rampup performance test.
 

Comment 1 by peah@chromium.org, Aug 9 2016

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=635886

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICglpHStQoM


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus72

Comment 2 by peah@chromium.org, Aug 9 2016

Owner: aleloi@chromium.org
aleloi: Could this be related to your CL https://codereview.webrtc.org/2206223002 ?

Comment 3 by ale...@webrtc.org, Aug 9 2016

I can't think of why that would be the case. The CL just routes calls from media_engine to voice_engine along a different path. I could be mistaken of course.

I'm currently running the test locally before and after my change. Is this the right test? (Never used the perf tests yet).

./out/Debug/webrtc_perf_tests --gtest_filter=FullStackTest.ForemanCif500kbps



Comment 4 by ale...@webrtc.org, Aug 9 2016

The test printed this:
[after] RESULT media_bitrate: foreman_cif_500kbps = {472848.355932, 46091.035785} bps
[before] RESULT media_bitrate: foreman_cif_500kbps = {483912.440678, 51826.326657} bps

The numbers were higher before. Is the difference large enough for it to be a regression?

Comment 5 by ale...@webrtc.org, Aug 9 2016

More data from 10 runs before and after my commit:

[after]
RESULT media_bitrate: foreman_cif_500kbps = {482117.118644, 52575.638891} bps
RESULT media_bitrate: foreman_cif_500kbps = {467676.864407, 50595.862686} bps
RESULT media_bitrate: foreman_cif_500kbps = {474071.033898, 45396.520069} bps
RESULT media_bitrate: foreman_cif_500kbps = {455254.322034, 45942.820188} bps
RESULT media_bitrate: foreman_cif_500kbps = {473287.508475, 46354.195718} bps
RESULT media_bitrate: foreman_cif_500kbps = {482059.728814, 52115.174141} bps
RESULT media_bitrate: foreman_cif_500kbps = {460246.067797, 48671.677389} bps
RESULT media_bitrate: foreman_cif_500kbps = {453636.847458, 45919.465630} bps
RESULT media_bitrate: foreman_cif_500kbps = {480199.000000, 48824.984509} bps
RESULT media_bitrate: foreman_cif_500kbps = {467532.525424, 57975.279313} bps


[before]

RESULT media_bitrate: foreman_cif_500kbps = {460843.898305, 49189.237499} bps
RESULT media_bitrate: foreman_cif_500kbps = {470074.322034, 50331.156245} bps
RESULT media_bitrate: foreman_cif_500kbps = {474943.813559, 50938.498567} bps
RESULT media_bitrate: foreman_cif_500kbps = {470452.000000, 50528.778188} bps
RESULT media_bitrate: foreman_cif_500kbps = {459460.677966, 47288.986803} bps
RESULT media_bitrate: foreman_cif_500kbps = {484727.440678, 51458.606900} bps
RESULT media_bitrate: foreman_cif_500kbps = {480594.406780, 50636.787068} bps
RESULT media_bitrate: foreman_cif_500kbps = {457423.881356, 54395.768977} bps
RESULT media_bitrate: foreman_cif_500kbps = {472221.169492, 50318.283210} bps
RESULT media_bitrate: foreman_cif_500kbps = {479274.118644, 51903.356893} bps


I will look at it more closely tomorrow.

Comment 6 by peah@chromium.org, Aug 15 2016

Cc: peah@chromium.org
 Issue 635885  has been merged into this issue.

Comment 7 by peah@chromium.org, Aug 22 2016

Owner: peah@chromium.org
It does not look as if your CL was the culprit for this. I'll take the issue and see if I can find another reason.

Comment 8 by peah@chromium.org, Aug 22 2016

Cc: -peah@chromium.org holmer@chromium.org
holmer@: This seems related to bit-rate selection. But I cannot find anything in the list of CLs 
https://chromium.googlesource.com/external/webrtc/+log/bfc7a30c663dbc7ecad145b8e834ebb47b3f75ec..90860af8811ccb524516c9003cdade85d2305f0e)
and
https://chromium.googlesource.com/chromium/src/+log/33f87688e4..f65883a9c0
that is related to this.

Do you know of anything that could have caused this?
Also, do you have a feeling for how bad that regressions are?





Comment 9 by peah@chromium.org, Aug 24 2016

Cc: -holmer@chromium.org peah@chromium.org
Owner: holmer@chromium.org
holmer@: Could this be caused by your CL https://codereview.webrtc.org/2201093006 ?
Status: WontFix (was: Assigned)
Yes, probably. I think the detection was a bit off in this case, so that it detected it earlier than it should.

Sign in to add a comment