New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 821835 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Mar 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

8.1%-17.2% regression in webrtc_perf_tests at 22376:22376

Project Member Reported by philipel@chromium.org, Mar 14 2018

Issue description

Are these changes expected? PTAL
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Mar 14 2018

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=821835

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=30846ca04921d703935ced76c6b75a4402f1481ff5f55075263e471867454944


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus5-kitkat
webrtc-android-tests-nexus9
Cc: emir...@chromium.org
Owner: phoglund@chromium.org
As far as I can tell, simulcast_vp8_3sl_high test shouldn't have anything related to my CL in the range. Also, my CL is a reland and original land or revert didn't cause any regressions.
https://webrtc.googlesource.com/src/+log/35576ac007c82e824943087e72dd33693fffb7ac..03e6ec9db0ec894e12f96713269a0c7b63502f60

However, there is a very clear signal at this rev, 4 more regression in other tests as well. I checked the revisions before and after but couldn't find anything related. Can this be a infra related issue, i.e. this test running side by side with the test added in my CL and effecting perf results? phoglund@ can you also take a look?


Status: WontFix (was: Assigned)
Well, you are twiddling around in the full stack tests, so it was probably your CL. I'm not aware of any infra changes around that time. The absolute values have just settled into new levels after you somehow disturbed the equilibrium. That's fine. It had been different had you changed the production code, but now since it's just test code, it's a WontFix.
It shot back up now to another level, again the changes seem unrelated and my CL isn't in that range:
https://chromeperf.appspot.com/report?sid=93da1d9febbf3bce19f518e8ad12c4171a487434e84ea35bd116f10125f1a614&rev=22376
Not, it's not your CL _that_ time, but I sure think https://webrtc.googlesource.com/src/+/853715c9a90611486aa191219a103a4acd7a664f looks suspicious.

If you feel this test is giving too many false positives, look in the blame for the test and discuss with the test authors. Perhaps the test is too sensitive or measures the wrong thing.

Sign in to add a comment