Issue metadata
Sign in to add a comment
|
13.7% regression in page_cycler.intl_ar_fa_he at 405386:405430 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 18 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006759492789771056
,
Jul 18 2016
,
Jul 19 2016
=== Auto-CCing suspected CL author waffles@chromium.org === Hi waffles@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Fix component-installer-controlled Flash path. Author : waffles Commit description: BUG=627959 Review-Url: https://codereview.chromium.org/2150573004 Cr-Commit-Position: refs/heads/master@{#405419} Commit : 3436cd4cbccb4c573185e2faefc248727e1baecb Date : Thu Jul 14 02:50:17 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@405385 731.523 5.09495 5 good chromium@405408 731.647 5.74776 5 good chromium@405414 743.469 22.7897 5 good chromium@405417 738.368 8.00279 5 good chromium@405418 741.638 7.03726 5 good chromium@405419 834.44 9.70063 5 bad <-- chromium@405430 823.317 6.49139 5 bad Bisect job ran on: mac_10_10_perf_bisect Bug ID: 629296 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.intl_ar_fa_he Test Metric: warm_times/page_load_time Relative Change: 12.55% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2214 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006759492789771056 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5831564143886336 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 19 2016
Can you tell me what page_cycler.intl_ar_fa_he is doing?
,
Jul 19 2016
Not sure (I'm just the perf sheriff, not the test owner). However, `git grep page_cycler` reveals the test code lives here: tools/perf/benchmarks/page_cycler.py From the code comments, it looks like this performance test is simply loading-up a whole bunch of pages and measuring how long they take to load. So, does anything in your change lengthen the critical path to renderer start-up time or page load time? Or, does your change cause some other code downstream to behave worse?
,
Jul 19 2016
My change could introduce a browser start-up time regression (because we look in an additional place to see if we can load a Flash plugin from there) but I don't see how it could cause a renderer start-up time regression, except maybe on test pages that include Adobe Flash, assuming that the browser process is already up and running. It's especially interesting that this change triggered the issue, because all this will affect is the *successful* loading of Flash if there's a component-updated implementation available. I don't think we can just revert the change because that will break Flash for most users. Can we adjust the baseline for now and I'll look into this issue going forward?
,
Jul 21 2016
Looking into a lot of the smoothness regressions that are also grouped in telemetry as part of this bug - would like to understand these better. Running an additional bisect on those regressions (we may have grouped two unrelated things together?) FYI, this was duped to 629262 which was in turn duped to this bug - un-duping this to remove the cycle.
,
Jul 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006511810627160784
,
Jul 21 2016
For instance, the perf reports linked to this bug also includes a 1500% regression in smoothness mean_input_event_latency, which is pretty bad - running a bisect on this now.
,
Jul 21 2016
,
Jul 21 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@405235 21.5265 0.759764 8 good chromium@405913 46.1843 54.2837 6 bad Bisect job ran on: win_perf_bisect Bug ID: 629296 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.top_25_smooth Test Metric: mean_input_event_latency/http___answers.yahoo.com Relative Change: 8.70% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6722 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006511810627160784 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=4936333743620096 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you! |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by m...@chromium.org
, Jul 18 2016