Issue metadata
Sign in to add a comment
|
5.2% regression in startup.warm.blank_page at 379759:379773 |
||||||||||||||||||||
Issue descriptionChromiumPerf/chromium-rel-mac10/startup.warm.blank_page / first_main_frame_load_time
,
Mar 9 2016
=== Auto-CCing suspected CL author tzik@chromium.org === Hi tzik@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Replace base::Tuple in //ipc with std::tuple Author : tzik Commit description: BUG= 554987 Review URL: https://codereview.chromium.org/1770013002 Cr-Commit-Position: refs/heads/master@{#379759} Commit : 55e3e4d3c52bf99bd8a710e55b1dcccea5e3acc6 Date : Tue Mar 08 05:50:35 2016 ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@379758 576.757895 2.337399 5 good chromium@379759 587.642105 3.768289 5 bad chromium@379760 585.842105 3.242934 5 bad chromium@379761 587.957895 2.113602 5 bad chromium@379763 588.336842 5.019988 5 bad chromium@379767 594.715789 20.651729 5 bad chromium@379773 584.315789 1.653503 5 bad Bisect job ran on: mac_10_10_perf_bisect Bug ID: 593188 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests startup.warm.blank_page Test Metric: first_main_frame_load_time/first_main_frame_load_time Relative Change: 1.31% Score: 99.8 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2029 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9018709932033130464 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 9 2016
Really...? My change does mostly nothing to the executable. It just expanded a type alias and replacing MakeRefTuple with a std equivalent. The delta of MakeRefTuple and std::tie is _NOEXCEPT_ addition only.
,
Mar 9 2016
Sorry for the noise, tzik. The bisect was looking for a 5% regression, and only found 1.3% at your CL, which is likely within the noise. We're working on making the bisect work better in cases like these. For the original regression, looking at the other mac bots it's a little strange: https://chromeperf.appspot.com/report?sid=34998f4d05383e5dfebcf44d8cbe9ce500cc2a97d3c09eeaa83e5a3a660fb895&rev=379773 The Mac 10 bot shows a 5% regression, Mac 11 and Retina show no regression, and the older Mac with a HDD shows a ~2% regression at a slightly earlier range. I'm going to kick off two additional bisects--one on the HDD mac, and one with an expanded range.
,
Mar 9 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@379708 1265.736842 12.62724 24 good chromium@379728 1267.940109 12.662616 29 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 593188 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests startup.warm.blank_page Test Metric: first_main_frame_load_time/first_main_frame_load_time Relative Change: 0.55% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/444 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9018660218083875280 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 9 2016
=== Auto-CCing suspected CL author jbudorick@chromium.org === Hi jbudorick@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add jbudorick to //testing/android/OWNERS. Author : jbudorick Commit description: BUG= Review URL: https://codereview.chromium.org/1775753002 Cr-Commit-Position: refs/heads/master@{#379709} Commit : 0bd9473b9553929068a7a2dcf4e50ccee084043b Date : Tue Mar 08 03:04:01 2016 ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@379708 571.440546 3.157394 27 good chromium@379709 578.915789 2.976966 5 bad chromium@379710 580.484211 1.971191 5 bad chromium@379712 585.157895 6.104583 5 bad chromium@379715 579.394737 3.101991 4 bad chromium@379724 583.810526 5.128353 5 bad chromium@379739 588.210526 23.599732 5 bad chromium@379773 574.866496 4.762825 41 bad Bisect job ran on: mac_10_10_perf_bisect Bug ID: 593188 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests startup.warm.blank_page Test Metric: first_main_frame_load_time/first_main_frame_load_time Relative Change: 0.03% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2031 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9018660211869699216 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 9 2016
try again, bisect bot.
,
Mar 15 2016
I don't think bisect when through, so I kicked-off another one https://chromeperf.appspot.com/buildbucket_job_status/9018085828378799456
,
Mar 15 2016
===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Replace base::Tuple in //ipc with std::tuple Author : tzik Commit description: BUG= 554987 Review URL: https://codereview.chromium.org/1770013002 Cr-Commit-Position: refs/heads/master@{#379759} Commit : 55e3e4d3c52bf99bd8a710e55b1dcccea5e3acc6 Date : Tue Mar 08 05:50:35 2016 ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@379758 578.614035 3.170093 6 good chromium@379759 588.378947 2.988297 5 bad chromium@379760 589.578947 2.912957 6 bad chromium@379761 591.263158 1.759423 5 bad chromium@379763 588.978947 1.616484 5 bad chromium@379767 585.447368 2.538349 8 bad chromium@379773 584.921053 2.805811 8 bad Bisect job ran on: mac_10_10_perf_bisect Bug ID: 593188 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests startup.warm.blank_page Test Metric: first_main_frame_load_time/first_main_frame_load_time Relative Change: 1.10% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2042 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9018085828378799456 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 25 2016
This has since recovered. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by ericwilligers@chromium.org
, Mar 9 2016