New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 618226 link

Starred by 0 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Jun 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug-Regression



Sign in to add a comment

72.7%-479.2% regression in webrtc_perf_tests at 13037:13037

Project Member Reported by hlundin@chromium.org, Jun 8 2016

Issue description

This comes with https://chromium.googlesource.com/external/webrtc/+/46b89b9a3f3f92acea250ab9e34a479040101145, which changes how a number of tests report their results.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=618226

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3MqCrQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvNKjrAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvMjcuwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3Pm7uAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3M69qAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3PmtqQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3MrYtAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3JHCvQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvLSVuQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3P22sgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3OyIsgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvPCtsQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3IrqoQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3I7wvAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3OugtQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3Ir0vwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3P2iowoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3KzDvAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3LnFuwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3MHLvAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3KvCoAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3KvYvgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3Ln0ogoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3MybsAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgvPCYtgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3MyNoQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgnJ-xsAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3LngvQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3L2woQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgnO-FuwoM


Bot(s) for this bug's original alert(s):

webrtc-android-tests-nexus5
webrtc-android-tests-nexus72
webrtc-android-tests-nexus9
webrtc-linux-large-tests
webrtc-mac-large-tests
webrtc-win-large-tests
Labels: -M-51 M-53 OS-All
Owner: danilchap@chromium.org
Danil, please, take a look and consider whether these deviations are expected from your change or not (https://chromium.googlesource.com/external/webrtc/+/46b89b9a3f3f92acea250ab9e34a479040101145).
Cc: pbos@chromium.org holmer@chromium.org
The deviations are caused by my change. I had wrong assumption about old behavior:
(before cl) when some stat is reported 100 times, the _last_ value is shown on the graph.
(after cl) When same stat is reported as a list if 100 values, the _average_ value is shown on the graph.

I'll now investigate which behavior is intended for each of mentioned stats.
If we're tracking the average I think we should compute the average based on samples after the synchronizer has converged, and not include the initial samples.
Project Member

Comment 5 by bugdroid1@chromium.org, Jun 16 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/webrtc.git/+/371b43b86a6a1bf0fb8a9059309a67a6e16a9542

commit 371b43b86a6a1bf0fb8a9059309a67a6e16a9542
Author: Danil Chapovalov <danilchap@webrtc.org>
Date: Thu Jun 16 07:58:44 2016

Changes synchronization offset perfomance tracking
to be logged after converge only.

BUG= webrtc:5032 ,  chromium:618226 
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2046943004 .

Cr-Commit-Position: refs/heads/master@{#13160}

[modify] https://crrev.com/371b43b86a6a1bf0fb8a9059309a67a6e16a9542/webrtc/call/call_perf_tests.cc

Labels: -performance-sheriff Performance-Sheriff
Status: Fixed (was: Assigned)
synchronization and bitrate_kbps metrics are now back to pre-aggregate values.
RampUp metrics (UpDownUpThreeStreamsRtx and second_rampup) are not related to this bug.

Sign in to add a comment