Issue metadata
Sign in to add a comment
|
12.6%-45.6% regression in blink_perf.dom at 496951:497059 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Aug 29 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969912156655506256
,
Aug 29 2017
=== Auto-CCing suspected CL author chaopeng@chromium.org === Hi chaopeng@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : chaopeng Commit : e8aa016b91432b7e599e19f75e60bc7b02e19f69 Date : Thu Aug 24 15:20:19 2017 Subject: Add UX experimental feature flash all scrollbars for any scroll update Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.dom Metric : inner_html_with_selection/inner_html_with_selection Change : 29.77% | 13.7981666667 -> 17.9056666667 Revision Result N chromium@496950 13.7982 +- 1.64144 6 good chromium@497005 14.8283 +- 1.02699 6 good chromium@497032 14.082 +- 1.38046 6 good chromium@497046 15.1893 +- 0.602892 6 good chromium@497053 15.406 +- 1.75563 6 good chromium@497054 18.2091 +- 4.35296 9 bad <-- chromium@497055 17.7705 +- 1.82023 6 bad chromium@497056 18.1162 +- 2.77756 6 bad chromium@497059 17.9057 +- 2.4112 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969912156655506256 For feedback, file a bug with component Speed>Bisection
,
Aug 30 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969811916367600128
,
Aug 30 2017
+perezju
I can not reproduce this performance change locally, also my patch is not related to selection and all behind flag. I don't think it related to my patch.
With my patch:
"per_page_values": [
{
"std": 0.6020527385536018,
"name": "inner_html_with_selection",
"type": "list_of_scalar_values",
"important": true,
"values": [
40.60500000000002,
41.47999999999979,
41.194999999999936,
40.17500000000018,
40.13999999999987
],
"units": "ms",
"page_id": 0
}
]
Without my patch:
"per_page_values": [
{
"std": 1.1684305713220686,
"name": "inner_html_with_selection",
"type": "list_of_scalar_values",
"important": true,
"values": [
40.40499999999997,
43.625,
42.02000000000021,
41.49500000000012,
41.585000000000036
],
"units": "ms",
"page_id": 0
}
]
,
Aug 30 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8969809199236276832
,
Aug 30 2017
Sorry, the benchmark is a bit noisy and it looks like it just picked up the CL by accident. Kicked off (hopefully) a cleaner bisect.
,
Aug 31 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : chaopeng Commit : e8aa016b91432b7e599e19f75e60bc7b02e19f69 Date : Thu Aug 24 15:20:19 2017 Subject: Add UX experimental feature flash all scrollbars for any scroll update Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.dom Metric : inner_html_with_selection/inner_html_with_selection Change : 36.42% | 13.4656666667 -> 18.3698333333 Revision Result N chromium@496950 13.4657 +- 0.957766 6 good chromium@497005 15.8773 +- 2.34304 6 good chromium@497032 14.6183 +- 1.60151 6 good chromium@497046 15.7617 +- 1.13221 6 good chromium@497053 15.7108 +- 2.42597 9 good chromium@497054 18.9937 +- 3.45689 6 bad <-- chromium@497055 18.082 +- 1.55704 6 bad chromium@497056 17.9237 +- 2.4516 9 bad chromium@497059 18.3698 +- 0.764697 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969811916367600128 For feedback, file a bug with component Speed>Bisection
,
Aug 31 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.dom Metric : remove_child_with_selection/remove_child_with_selection Change : 11.54% | 46.2286666667 -> 51.5641666667 Suspected Commit Range 2 commits in range https://chromium.googlesource.com/chromium/src/+log/3686eccdb7575bcf4d1630c3572625f3d64beb0f..32c9c46b63856450fb00bbc2bfa51e181da8e908 Revision Result N chromium@495876 46.2287 +- 1.55717 6 good chromium@495962 48.8947 +- 0.432022 6 good chromium@495973 49.6602 +- 0.679199 6 good chromium@495979 50.3794 +- 2.19092 14 good chromium@495980 --- --- build failure chromium@495981 51.6289 +- 3.72879 14 bad chromium@495982 51.4701 +- 2.92767 9 bad chromium@495984 52.2512 +- 2.33869 6 bad chromium@496005 50.8135 +- 2.38753 6 bad chromium@496047 51.5642 +- 3.7847 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8969809199236276832 For feedback, file a bug with component Speed>Bisection
,
Oct 18 2017
+jbroman, owner of blink_perf.dom: These bisects (except #9, which appears to be in the wrong range) consistently report r497054 as the culprit, but as chaopeng says the feature is behind a flag and I can't figure out how it would affect the benchmark. any ideas?
,
Oct 18 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965373099992050992
,
Oct 18 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965372959496925472
,
Oct 18 2017
That does seem quite bizarre. I've kicked off another couple bisects under the assumption that the actual regression is earlier, where the graphs first tick up. It's annoying that some of these regressions come with a massive increase in variation, such that a run after the regression can spuriously show as "good". It might be nice if the bisect bots could do a few runs and then bisect on z-score=1 or something like that.
,
Oct 18 2017
Thanks! +dtu, simonhatch for bisect suggestion in #13
,
Oct 19 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Michael Lippautz Commit : 146d898b6e27a3900dae50b497e24674d7d2adfb Date : Mon Aug 21 09:24:08 2017 Subject: [wrapper-tracing] NodeListsNodeData Align Oilpan and wrapper tracing Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.dom Metric : inner_html_with_selection/inner_html_with_selection Change : 45.09% | 10.9668333333 -> 15.9113333333 Revision Result N chromium@495856 10.9668 +- 0.815618 6 good chromium@495880 11.2608 +- 0.846219 6 good chromium@495886 11.9138 +- 1.65413 6 good chromium@495889 11.7737 +- 1.30782 6 good chromium@495890 11.6597 +- 0.275839 6 good chromium@495891 15.6857 +- 2.41175 6 bad <-- chromium@495892 17.0717 +- 1.10067 6 bad chromium@495904 16.09 +- 1.42803 6 bad chromium@495952 16.1535 +- 2.01659 6 bad chromium@496047 15.9113 +- 1.98468 6 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965372959496925472 For feedback, file a bug with component Speed>Bisection
,
Oct 19 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Michael Lippautz Commit : 146d898b6e27a3900dae50b497e24674d7d2adfb Date : Mon Aug 21 09:24:08 2017 Subject: [wrapper-tracing] NodeListsNodeData Align Oilpan and wrapper tracing Bisect Details Configuration: android_nexus7_perf_bisect Benchmark : blink_perf.dom Metric : remove_child_with_selection/remove_child_with_selection Change : 7.76% | 46.198 -> 49.7823333333 Revision Result N chromium@495856 46.198 +- 1.79744 6 good chromium@495874 46.4253 +- 3.32489 6 good chromium@495883 45.7097 +- 2.16327 6 good chromium@495887 45.8978 +- 2.65584 9 good chromium@495889 46.832 +- 6.26433 9 good chromium@495890 46.5237 +- 3.00879 6 good chromium@495891 50.2983 +- 0.630776 9 bad <-- chromium@495925 49.9777 +- 1.80762 6 bad chromium@495993 50.9255 +- 1.87514 6 bad chromium@496130 50.6167 +- 3.47965 6 bad chromium@496403 49.9247 +- 3.22245 6 bad chromium@496950 49.7823 +- 1.87053 6 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965373099992050992 For feedback, file a bug with component Speed>Bisection |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Aug 29 2017