New issue
Advanced search Search tips

Issue 645594 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Sep 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

84.5% regression in performance_browser_tests at 414195:414252

Project Member Reported by rsch...@chromium.org, Sep 9 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=645594

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICggfz5vQkM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-dual
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Sep 10 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@414194  -46.5579  18.4928  18  good
chromium@414252  -46.543   18.1908  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 645594

Test Command: .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu
Test Metric: CastV2Performance_gpu_30fps_fast/av_sync
Relative Change: 4.20%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6920
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001970539036785392


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5830415238037504

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Trying a wider bisect.
Owner: m...@chromium.org
Just noticed this bisect produced negative values. miu, is that supposed to happen for this test?

Comment 7 by m...@chromium.org, Sep 13 2016

Status: WontFix (was: Assigned)
Yes, A/V sync can be in either direction (audio ahead of video vs. audio behind video): https://cs.chromium.org/chromium/src/chrome/browser/extensions/api/cast_streaming/performance_test.cc?rcl=0&l=229

Closer to zero is always better.

Looks like we need to update the test config for this measurement. I'll note this in bug 567848.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Sep 13 2016

Cc: enne@chromium.org
Owner: enne@chromium.org

=== Auto-CCing suspected CL author enne@chromium.org ===

Hi enne@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Use vsync manager regardless of begin frame settings
Author  : enne
Commit description:
  
There was some hope that the ui::CompositorVSyncManager could be removed
once --enable-begin-frame-scheduling was turned on.  Unfortunately, the
vsync manager is used by two additional systems that can't be easily
removed: DelegatedFrameHost::AttemptFrameSubscriberCapture as well as
components/exo/wayland/server.cc.  The latter may go away eventually.
If so, then the frame capture could then be changed to more simple
polling as needed instead of vsync manager and observer system that
needs to be plumbed everywhere.

In the medium term, the vsync manager needs to live on in all paths.
This means that BrowserCompositorOutputSurface and ui::Compositor always
need to update the vsync manager with changes.  Conditionals for
whether or not to send vsync information to renderers just get moved
down the pipeline from the output surface to DelegatedFrameHost.

Review-Url: https://codereview.chromium.org/2277883002
Cr-Commit-Position: refs/heads/master@{#414186}
Commit  : 0875b32a9ef74d66951b77930ca46f1a3bf08043
Date    : Wed Aug 24 23:00:35 2016


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@414099  -61.9975  2.96861  12  good
chromium@414154  -57.1983  11.6532  18  good
chromium@414182  -58.5021  11.4318  18  good
chromium@414184  -61.7354  3.97209  5   good
chromium@414185  -62.215   1.31901  5   good
chromium@414186  -49.8642  13.9957  18  bad    <--
chromium@414189  -46.4777  18.756   18  bad
chromium@414196  -50.4746  12.9603  18  bad
chromium@414209  -51.5913  4.9264   18  bad
chromium@414319  -46.2776  23.0175  8   bad

Bisect job ran on: win_perf_bisect
Bug ID: 645594

Test Command: .\src\out\Release\performance_browser_tests.exe --test-launcher-print-test-stdio=always --enable-gpu
Test Metric: CastV2Performance_gpu_30fps_fast/av_sync
Relative Change: 9.59%
Score: 99.5

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6926
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001681781384059616


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5274054167101440

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by m...@chromium.org, Sep 13 2016

Nothing to see here. enne's change actually improved performance here. False alarm because the alert is set up as "lesser is better" whereas it should be "closer to zero is better."

Sign in to add a comment