New issue
Advanced search Search tips

Issue 629296 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 623804
Owner:
Closed: Jul 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

13.7% regression in page_cycler.intl_ar_fa_he at 405386:405430

Project Member Reported by m...@chromium.org, Jul 18 2016

Issue description

See the link to graphs below.
 

Comment 1 by m...@chromium.org, Jul 18 2016

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=629296

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgipeWrgoM


Bot(s) for this bug's original alert(s):

chromium-rel-mac10

Comment 3 by m...@chromium.org, Jul 18 2016

Mergedinto: 629262
Status: Duplicate (was: Assigned)
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, Jul 19 2016

Cc: waff...@chromium.org
Owner: waff...@chromium.org

=== Auto-CCing suspected CL author waffles@chromium.org ===

Hi waffles@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Fix component-installer-controlled Flash path.
Author  : waffles
Commit description:
  
BUG=627959

Review-Url: https://codereview.chromium.org/2150573004
Cr-Commit-Position: refs/heads/master@{#405419}
Commit  : 3436cd4cbccb4c573185e2faefc248727e1baecb
Date    : Thu Jul 14 02:50:17 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@405385  731.523  5.09495  5  good
chromium@405408  731.647  5.74776  5  good
chromium@405414  743.469  22.7897  5  good
chromium@405417  738.368  8.00279  5  good
chromium@405418  741.638  7.03726  5  good
chromium@405419  834.44   9.70063  5  bad    <--
chromium@405430  823.317  6.49139  5  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 629296

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.intl_ar_fa_he
Test Metric: warm_times/page_load_time
Relative Change: 12.55%
Score: 99.9

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2214
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006759492789771056


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5831564143886336

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Can you tell me what page_cycler.intl_ar_fa_he is doing?

Comment 6 by m...@chromium.org, Jul 19 2016

Not sure (I'm just the perf sheriff, not the test owner). However, `git grep page_cycler` reveals the test code lives here: tools/perf/benchmarks/page_cycler.py 

From the code comments, it looks like this performance test is simply loading-up a whole bunch of pages and measuring how long they take to load. So, does anything in your change lengthen the critical path to renderer start-up time or page load time? Or, does your change cause some other code downstream to behave worse?


My change could introduce a browser start-up time regression (because we look in an additional place to see if we can load a Flash plugin from there) but I don't see how it could cause a renderer start-up time regression, except maybe on test pages that include Adobe Flash, assuming that the browser process is already up and running.

It's especially interesting that this change triggered the issue, because all this will affect is the *successful* loading of Flash if there's a component-updated implementation available.

I don't think we can just revert the change because that will break Flash for most users. Can we adjust the baseline for now and I'll look into this issue going forward?

Comment 8 by ericrk@chromium.org, Jul 21 2016

Status: Assigned (was: Duplicate)
Looking into a lot of the smoothness regressions that are also grouped in telemetry as part of this bug - would like to understand these better. Running an additional bisect on those regressions (we may have grouped two unrelated things together?)

FYI, this was duped to 629262 which was in turn duped to this bug - un-duping this to remove the cycle.
For instance, the perf reports linked to this bug also includes a 1500% regression in smoothness mean_input_event_latency, which is pretty bad - running a bisect on this now.
Cc: ericrk@chromium.org
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jul 21 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N  Good?
chromium@405235  21.5265  0.759764  8  good
chromium@405913  46.1843  54.2837   6  bad

Bisect job ran on: win_perf_bisect
Bug ID: 629296

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.top_25_smooth
Test Metric: mean_input_event_latency/http___answers.yahoo.com
Relative Change: 8.70%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6722
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006511810627160784


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4936333743620096

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Mergedinto: -629262 623804
Status: Duplicate (was: Assigned)
(Re-duping properly.)

Sign in to add a comment