Issue metadata
Sign in to add a comment
|
10.8% regression in v8.runtimestats.browsing_desktop at 487411:487550 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 24 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8973205590037464688
,
Jul 24 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8973205581417077264
,
Jul 24 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : V8-Only:duration_avg/browse_news/browse_news_hackernews Revision Result N chromium@487410 963.886 +- 161.324 21 good chromium@487550 982.656 +- 159.901 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.hackernews v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8973205581417077264 For feedback, file a bug with component Speed>Bisection
,
Jul 24 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : V8-Only:duration_avg/browse_news/browse_news_hackernews Revision Result N chromium@487410 871.177 +- 144.031 21 good chromium@487550 873.42 +- 126.821 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.hackernews v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8973205590037464688 For feedback, file a bug with component Speed>Bisection
,
Jul 24 2017
,
Jul 29 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972692430341891424
,
Jul 29 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972692415170350816
,
Jul 29 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972692400841878704
,
Jul 30 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : Total:duration_avg/browse_social/browse_social_facebook_infinite_scroll Revision Result N chromium@487410 6526.5 +- 3920.73 21 good chromium@487550 6305.06 +- 716.624 21 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8972692415170350816 For feedback, file a bug with component Speed>Bisection
,
Jul 30 2017
=== Auto-CCing suspected CL author fmalita@chromium.org === Hi fmalita@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Florin Malita Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1 Date : Tue Jul 18 17:13:13 2017 Subject: Enable Skia's integral-translate bilerp optimization Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : V8-Only:duration_avg/browse_social/browse_social_facebook_infinite_scroll Change : 6.83% | 4807.10933333 -> 5135.2045 Revision Result N chromium@487410 4807.11 +- 107.388 6 good chromium@487480 4831.46 +- 551.666 9 good chromium@487498 4828.93 +- 376.666 9 good chromium@487503 4780.92 +- 403.603 9 good chromium@487505 4823.43 +- 270.203 9 good chromium@487506 4812.03 +- 427.417 9 good chromium@487507 5083.28 +- 806.553 9 bad <-- chromium@487515 5129.79 +- 269.583 9 bad chromium@487550 5135.2 +- 89.2919 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8972692400841878704 For feedback, file a bug with component Speed>Bisection
,
Jul 30 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Florin Malita Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1 Date : Tue Jul 18 17:13:13 2017 Subject: Enable Skia's integral-translate bilerp optimization Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : JavaScript:duration_avg/browse_social/browse_social_facebook_infinite_scroll Change : 5.37% | 2067.61983333 -> 2178.68516667 Revision Result N chromium@487410 2067.62 +- 63.9053 6 good chromium@487480 2068.25 +- 51.8151 6 good chromium@487498 2059.75 +- 63.7764 6 good chromium@487503 2064.64 +- 124.121 6 good chromium@487505 2045.34 +- 129.248 14 good chromium@487506 2053.42 +- 297.076 14 good chromium@487507 2141.14 +- 325.199 14 bad <-- chromium@487515 2166.27 +- 109.581 6 bad chromium@487550 2178.69 +- 79.5018 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8972692430341891424 For feedback, file a bug with component Speed>Bisection
,
Jul 31 2017
I'm suspicious of the blame here: 1) https://chromium-review.googlesource.com/c/575675 should only affect raterization (no JS/v8 impact), and should be a strict improvement (disables image filtering in some cases). 2) the bisect identifies a 5-6% regression, but the graphs show a 60%+ regression (e.g. https://chromeperf.appspot.com/report?sid=df06f0ac7568adf3ef1472691c2bc30a5dbca6008df5c4fbc63b6d68cf368b75) 3) in local testing the numbers are quite noisy (definitely more than 5%). Is there a way to re-run the blame bisect with a higher threshold?
,
Jul 31 2017
,
Aug 1 2017
,
Aug 2 2017
I see the bisect for issue 751154 is also zooming in on https://chromium-review.googlesource.com/c/575675. I cannot repro the regression locally, and, more importantly, the actual tests don't even hit the code affected by the CL. I've verified with an instrumented build that neither canvas_xxxxx_pixels_per_second nor text_xxxxx_pixels_per_second are drawing any images, and are not reaching SkBitmapProcInfo::init() at all. So something's off. Not sure why the bisect keeps picking on https://chromium-review.googlesource.com/c/575675, but I'm pretty sure it cannot be responsible for these regressions.
,
Aug 2 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972339980654200176
,
Aug 3 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: linux_perf_bisect Benchmark : thread_times.tough_scrolling_cases Metric : thread_raster_cpu_time_per_frame/text_30000_pixels_per_second Change : 20.37% | 0.905910612795 -> 1.05918930715 Suspected Commit Range 5 commits in range https://chromium.googlesource.com/chromium/src/+log/cebd3b079bbbe1716c0c4b5dc0ae27c03f5876a7..13b1999a87a545bcab93d66a2294758c3125a84b Revision Result N chromium@487410 0.905911 +- 0.300258 14 good chromium@487451 0.920002 +- 0.362076 21 good chromium@487474 0.901417 +- 0.124938 9 good chromium@487484 0.910623 +- 0.143609 14 good chromium@487485 --- --- build failure chromium@487486 --- --- build failure chromium@487487 --- --- build failure chromium@487488 --- --- build failure chromium@487489 0.961421 +- 0.378532 14 bad chromium@487494 0.973666 +- 0.217047 14 bad chromium@487577 1.04894 +- 0.287142 9 bad chromium@487743 1.00167 +- 0.153934 9 bad chromium@488076 1.03938 +- 0.271104 9 bad chromium@488741 1.05919 +- 0.189451 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=text.30000.pixels.per.second thread_times.tough_scrolling_cases More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8972339980654200176 For feedback, file a bug with component Speed>Bisection
,
Aug 3 2017
+mythria: any ideas how to help debug what could be going on here? See #13 and #17
,
Aug 4 2017
I am not quite sure, the bisects on different metrics point to the same cl. This makes me suspect that we are doing more work (for ex: scrolling more) because of some side effect of the change pointed by the bisect. If it somehow changes the amount we scroll, it could impact the amount of javascript we execute and hence regress V8 metrics. I will trigger another bisect with a slightly different range. If that doesn't succeed or doesn't point us to anything different then I think we could close it as won't fix.
,
Aug 4 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8972197505818636800
,
Aug 4 2017
I was only referring to regressions on V8 metrics when I said we can mark it as won't fix. I am not so sure about other regressions on thread_times.tough_scrolling_cases.
,
Aug 4 2017
The thread_times bisect hit build failures, but it looks like a different range of CLs were narrowed in on. I split them into bug 752457 . mythria: as the owner of the benchmark, feel free to WontFix this based on results from the bisect since this bug is now just about v8.runtimestats.browsing_desktop.
,
Aug 4 2017
sullivan@, mythria@ thanks for helping triage this. Re. V8 metrics, handwaving around mythria's theory: the net effect of the CL is lower rasterization cost in some cases (when images are drawn with an integral translation only, we skip bilerp filtering); so if the tests are somehow synchronized on rasterization times, it sounds plausible that they'd be seeing more JS callbacks when frames take less time to raster. FYI, https://chromium-review.googlesource.com/c/575675 was a fix for issue 744674 , and its effect can be seen on those graphs: https://chromeperf.appspot.com/group_report?bug_id=744674 Come to think of it, many of those improved metrics appear related to scroll update latency - so mythria's explanation definitely makes sense: reduced latency -> increased frequency -> more V8 work. Based on that, I now think the bisect is correct and we can close this as wontfix.
,
Aug 4 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Florin Malita Commit : 2a27a8ddae522a22a22e0aadf56399537b695df1 Date : Tue Jul 18 17:13:13 2017 Subject: Enable Skia's integral-translate bilerp optimization Bisect Details Configuration: linux_perf_bisect Benchmark : v8.runtimestats.browsing_desktop Metric : JavaScript:duration_avg/browse_social/browse_social_facebook_infinite_scroll Change : 5.18% | 2054.71571429 -> 2178.05816667 Revision Result N chromium@487500 2054.72 +- 229.549 14 good chromium@487504 2033.07 +- 111.48 9 good chromium@487506 2045.37 +- 88.4205 9 good chromium@487507 2108.64 +- 508.961 14 bad <-- chromium@487513 2161.28 +- 200.657 9 bad chromium@487525 2164.22 +- 177.437 9 bad chromium@487550 2213.77 +- 123.371 6 bad chromium@487600 2171.32 +- 184.056 9 bad chromium@487700 2178.06 +- 96.34 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.facebook.infinite.scroll v8.runtimestats.browsing_desktop More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8972197505818636800 For feedback, file a bug with component Speed>Bisection
,
Aug 4 2017
Great, the bisect came back with the same result, now expected. I think we're good here.
,
Aug 4 2017
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Jul 24 2017