Issue metadata
Sign in to add a comment
|
28.9%-36.1% regression in speedometer at 435864:435964 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Dec 5 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994110572339879632
,
Dec 5 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence. Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@435863 884.242 453.433 50 good chromium@435964 860.235 168.33 50 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 671177 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer Test Metric: AngularJS-TodoMVC/AngularJS-TodoMVC Relative Change: 2.72% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/623 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994110572339879632 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5003433642295296 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 5 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994108616826603920
,
Dec 5 2016
===== BISECT JOB RESULTS ===== Status: failed === Bisection aborted === The bisect was aborted because Bisect cannot identify a culprit: Please contact the the team (see below) if you believe this is in error. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@435863 383.767 334.178 50 good chromium@435876 403.145 166.258 50 good chromium@435883 416.123 319.67 50 unknown chromium@435889 427.089 376.686 50 bad chromium@435914 404.212 115.343 50 bad chromium@435964 404.383 180.371 50 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 671177 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer Test Metric: FlightJS-TodoMVC/FlightJS-TodoMVC Relative Change: 5.37% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/624 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994108616826603920 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5818593470578688 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 6 2016
,
Dec 8 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8993843734441711088
,
Dec 8 2016
=== PERF REGRESSION === === Auto-CCing suspected CL author pkalinnikov@chromium.org === Hi pkalinnikov@chromium.org, the bisect results pointed to your CL, please take a look at the results. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add high-precision timing histograms. Author : pkalinnikov Commit description: BUG=609747 Review-Url: https://codereview.chromium.org/2503283003 Cr-Commit-Position: refs/heads/master@{#435890} Commit : 7c84dd6c2273cea47031237046ec001ac7fb75ea Date : Fri Dec 02 09:07:18 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@435863 503.72 205.978 50 good chromium@435889 531.628 486.721 50 good chromium@435890 491.257 70.7267 50 bad <-- chromium@435891 488.275 73.9187 50 bad chromium@435893 493.689 193.847 50 bad chromium@435896 489.856 138.197 50 bad chromium@435902 494.269 125.575 50 bad chromium@435914 491.643 137.152 50 bad chromium@435964 492.882 124.03 50 bad Bisect job ran on: winx64_zen_perf_bisect Bug ID: 671177 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer Test Metric: BackboneJS-TodoMVC/BackboneJS-TodoMVC Relative Change: 2.15% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64_zen_perf_bisect/builds/628 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993843734441711088 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5260521358491648 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 12 2016
I think the CL 2503283003 is not relevant to this performance regression for 2 reasons: 1. The component that is modified in the CL is not active, when the aforementioned benchmark runs, therefore it can't influence its results. To prove it, I added CHECK(false) to DocumentSubresourceFilter constructor, and run the benchmark, and it worked successfully without failures. 2. The performance graph (https://chromeperf.appspot.com/group_report?bug_id=671177) reached its normal level after a while, so that spike was probably flaky and unrelated to my CL. Michael, what do you think? Can we mark this as resolved?
,
Dec 13 2016
,
Dec 15 2016
Yeah, probably zenbooks missbehaving.
,
Apr 10 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982667341478765424
,
Apr 10 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: winx64_zen_perf_bisect Benchmark : speedometer Metric : AngularJS-TodoMVC/AngularJS-TodoMVC Revision Result N chromium@435863 897.899 +- 190.669 21 good chromium@435964 975.763 +- 745.913 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests speedometer Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8982667341478765424 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5003433642295296 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you! |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by hablich@chromium.org
, Dec 5 2016