New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 598026 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Jul 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

10.8%-56.8% regression in performance_browser_tests at 383087:383126

Project Member Reported by sullivan@chromium.org, Mar 25 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=598026

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsNGusAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsIO5uQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsKqD-AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsN-krwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsP3NtgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg8LDDtgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsInvygkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPThpAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsLb-vwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPThpAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsNLtsQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsIjJqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPThpAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg0M_y3wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsLnD3goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg8LDZpwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg0PryxgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsOHAsQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPT6uQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg8LCuugoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPT6uQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsKG7oAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsO_5pQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsPScqAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsKGtvgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsJ2QnQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsJmFyAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgsLbQpgoM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
chromium-rel-win7-gpu-ati
chromium-rel-win7-gpu-nvidia
chromium-rel-win7-x64-dual
chromium-rel-win8-dual
Project Member

Comment 2 by 42576172...@developer.gserviceaccount.com, Mar 25 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision                Mean Value  Std. Dev.   Num Values  Good?
chromium@383095         37.924306   0.210981    18          good
chromium@383125         38.152037   0.673711    12          bad

Bisect job ran on: win_perf_bisect
Bug ID: 598026

Test Command: .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu
Test Metric: TabCapturePerformance_comp_gpu_webrtc/CaptureSucceeded
Relative Change: 0.59%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6437
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017190478619654736


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=598026

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
I kicked off a few more bisects on the most-regressed metrics.
Owner: marpan@chromium.org
Seems to be the libvpx roll 09915545c43bac0776a0ad366960501eeb298995. marpan, are you right to take a look at this?

Comment 5 by marpan@chromium.org, Apr 11 2016

Cc: m...@chromium.org

Comment 6 by m...@chromium.org, Apr 11 2016

The CastV2Performance tests are explicitly testing end-to-end screen mirroring, and the "Encode" line in the graphs is measuring just the real-time encode (VP8) run time.  It's pretty clear from the graphs that encode performance has regressed by about 15% on all machines.

Comment 7 by marpan@chromium.org, Apr 11 2016

Cc: yunqingwang@chromium.org
Ping.

Comment 9 by marpan@google.com, Apr 22 2016

We looked into the libvpx roll in #5, and the change that caused the CastV2Perf regression seems to be VP8 change (only VP8 change in that roll): https://chromium.googlesource.com/webm/libvpx.git/+/b198bcd528ef53b5b292255449f791d3f787496f

But this VP8 change was needed to fix a deadlock/freeze issue for Hangouts: 
https://buganizer.corp.google.com/issues/27232610

So reverting it is not a good option. We have not found another suitable fix.


How would you like to proceed? Do we need to accept the regression to avoid the deadlock issue, or should we keep it open and investigate an optimization?

Comment 11 by marpan@google.com, May 6 2016

I think we need to accept the perf regression, as we need to avoid the deadlock issue. We don't currently have a fix for the perf regression.
Cc: marpan@chromium.org
Owner: rsch...@chromium.org
Assigning this to rschoen@chromium.org to take a decision.
rschoen@, Should this be marked as won't fix as the regression is expected?
Labels: -performance-sheriff Performance-Sheriff
Status: WontFix (was: Assigned)
Marking WontFix, as we don't have any other solution.

Sign in to add a comment