Issue metadata
Sign in to add a comment
|
22.1% regression in webrtc_perf_tests at 13635:13637 |
||||||||||||||||||||
Issue descriptionThere is a 22.1 % regression in the ramp_up_down_3streams_rtc/second_rampup performance test.
,
Aug 9 2016
aleloi: Could this be related to your CL https://codereview.webrtc.org/2206223002 ?
,
Aug 9 2016
I can't think of why that would be the case. The CL just routes calls from media_engine to voice_engine along a different path. I could be mistaken of course. I'm currently running the test locally before and after my change. Is this the right test? (Never used the perf tests yet). ./out/Debug/webrtc_perf_tests --gtest_filter=FullStackTest.ForemanCif500kbps
,
Aug 9 2016
The test printed this:
[after] RESULT media_bitrate: foreman_cif_500kbps = {472848.355932, 46091.035785} bps
[before] RESULT media_bitrate: foreman_cif_500kbps = {483912.440678, 51826.326657} bps
The numbers were higher before. Is the difference large enough for it to be a regression?
,
Aug 9 2016
More data from 10 runs before and after my commit:
[after]
RESULT media_bitrate: foreman_cif_500kbps = {482117.118644, 52575.638891} bps
RESULT media_bitrate: foreman_cif_500kbps = {467676.864407, 50595.862686} bps
RESULT media_bitrate: foreman_cif_500kbps = {474071.033898, 45396.520069} bps
RESULT media_bitrate: foreman_cif_500kbps = {455254.322034, 45942.820188} bps
RESULT media_bitrate: foreman_cif_500kbps = {473287.508475, 46354.195718} bps
RESULT media_bitrate: foreman_cif_500kbps = {482059.728814, 52115.174141} bps
RESULT media_bitrate: foreman_cif_500kbps = {460246.067797, 48671.677389} bps
RESULT media_bitrate: foreman_cif_500kbps = {453636.847458, 45919.465630} bps
RESULT media_bitrate: foreman_cif_500kbps = {480199.000000, 48824.984509} bps
RESULT media_bitrate: foreman_cif_500kbps = {467532.525424, 57975.279313} bps
[before]
RESULT media_bitrate: foreman_cif_500kbps = {460843.898305, 49189.237499} bps
RESULT media_bitrate: foreman_cif_500kbps = {470074.322034, 50331.156245} bps
RESULT media_bitrate: foreman_cif_500kbps = {474943.813559, 50938.498567} bps
RESULT media_bitrate: foreman_cif_500kbps = {470452.000000, 50528.778188} bps
RESULT media_bitrate: foreman_cif_500kbps = {459460.677966, 47288.986803} bps
RESULT media_bitrate: foreman_cif_500kbps = {484727.440678, 51458.606900} bps
RESULT media_bitrate: foreman_cif_500kbps = {480594.406780, 50636.787068} bps
RESULT media_bitrate: foreman_cif_500kbps = {457423.881356, 54395.768977} bps
RESULT media_bitrate: foreman_cif_500kbps = {472221.169492, 50318.283210} bps
RESULT media_bitrate: foreman_cif_500kbps = {479274.118644, 51903.356893} bps
I will look at it more closely tomorrow.
,
Aug 15 2016
,
Aug 22 2016
It does not look as if your CL was the culprit for this. I'll take the issue and see if I can find another reason.
,
Aug 22 2016
holmer@: This seems related to bit-rate selection. But I cannot find anything in the list of CLs https://chromium.googlesource.com/external/webrtc/+log/bfc7a30c663dbc7ecad145b8e834ebb47b3f75ec..90860af8811ccb524516c9003cdade85d2305f0e) and https://chromium.googlesource.com/chromium/src/+log/33f87688e4..f65883a9c0 that is related to this. Do you know of anything that could have caused this? Also, do you have a feeling for how bad that regressions are?
,
Aug 24 2016
holmer@: Could this be caused by your CL https://codereview.webrtc.org/2201093006 ?
,
Sep 7 2016
Yes, probably. I think the detection was a bit off in this case, so that it detected it earlier than it should. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by peah@chromium.org
, Aug 9 2016