Issue metadata
Sign in to add a comment
|
49.5% regression in smoothness.sync_scroll.key_mobile_sites_smooth at 508735:508849 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Oct 18 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965384791583210928
,
Oct 18 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : David Bokan Commit : b0d47f5f4d4b9717bfdd43ae6d40257235d0848f Date : Fri Oct 13 16:49:07 2017 Subject: Fix coordinate space issues in scroll.js Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : smoothness.sync_scroll.key_mobile_sites_smooth Metric : mean_input_event_latency/http___worldjournal.com_ Change : 48.15% | 45.2791666667 -> 67.081 Revision Result N chromium@508734 45.2792 +- 7.13884 6 good chromium@508763 44.747 +- 2.62932 6 good chromium@508778 45.0025 +- 2.04772 6 good chromium@508785 44.2677 +- 2.44812 6 good chromium@508789 44.9917 +- 3.2022 6 good chromium@508789,catapult@b0d47f5f4d 68.7233 +- 7.41183 6 bad <-- chromium@508789,catapult@dd5b39ac0c 66.92 +- 7.47891 6 bad chromium@508790 67.7732 +- 6.74238 6 bad chromium@508791 70.211 +- 8.31557 6 bad chromium@508792 68.3372 +- 6.23758 6 bad chromium@508849 67.081 +- 6.19212 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...worldjournal.com. smoothness.sync_scroll.key_mobile_sites_smooth More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965384791583210928 For feedback, file a bug with component Speed>Bisection
,
Oct 18 2017
bokan@ could you please take a look and check if the regression is expected or if there is a bug? Some explanation: This regression is caused by the same CL that caused crbug.com/775114 . The explanation for crbug.com/775114 was that chromium@506804 wrongly "improved" these metrics and chromium@508789 just fixed the wrong improvement and so metric values are going up to where they were before chromium@506804. However, I suspect that there could be a real regression because, unlike in crbug.com/775114 , the graphs in this bug, e.g. https://chromeperf.appspot.com/group_report?sid=b3e7ae3b6649d4312c920c4e30bf49c2f0bacbd6544d4a1ca99fa8823777b749, go up much more than pre-chromium@506804 levels.
,
Oct 18 2017
Thanks, that graph definitely looks fishy. I'll investigate.
,
Oct 20 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8965180205302535376
,
Oct 20 2017
Looks like there's a couple issues lumped in here. 1) There's a set of tests that have no matching change from when I landed the orignal "improving" CL. This is likely a separate regression. I've filed issue 775950 to track those separately. 2) There's another set where the graph recovers or improves but not back to the original level. Within this set, there's three cases: - Ref build also moved in the same direction, I think we can ignore these. e.g.: https://chromeperf.appspot.com/report?sid=dfd333902e6b7a169adb232688e5c6fabe7f86128c4f22ae58f99b8184b5ba4a&start_rev=504522&end_rev=510471 - It looks like there was a regression right before I landed my "improve" CL so the dashboard doesn't mark them as regressions, some clearer examples: https://chromeperf.appspot.com/report?sid=3aaca1dafd591aa9676f38f45fb39d2f5462628963baf8f568451a752fd8f26e&start_rev=504305&end_rev=510471 https://chromeperf.appspot.com/report?sid=9f49ab87539bbceae42f8e9d9b7bce025e9ea3b078242d52a1488191b892282d&start_rev=505468&end_rev=510418 https://chromeperf.appspot.com/report?sid=248f0d27bb62a59e65083fd78b79ebb9f7595da4f3d6bf58dd79ca5c3d5a1593&start_rev=505988&end_rev=510446 I've kicked off a bisect (referencing this bug): https://chromeperf.appspot.com/buildbucket_job_status/8965180205302535376 - Some don't have the bump before-hand but regressed much higher than expected (these all seem to be mean_input_event_latency): https://chromeperf.appspot.com/report?sid=da763bf9cc7186a7ccc03e3ee63afc07cd4dfd8b74598407420f3769230f26c2&start_rev=503982&end_rev=510461 https://chromeperf.appspot.com/report?sid=55a26229c75a2c92f370e896122a62479a2a592573c6c5416182e64c0be3900f&start_rev=504522&end_rev=510471 These warrant some more investigation.
,
Oct 20 2017
=== BISECT JOB RESULTS === Bisect failed for unknown reasons Please contact the team (see below) and report the error. Bisect Details Configuration: android_nexus6_perf_bisect Benchmark : thread_times.simple_mobile_sites Metric : thread_other_cpu_time_per_frame/http___www.ebay.co.uk_ To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8965180205302535376 For feedback, file a bug with component Speed>Bisection
,
Oct 23 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8964934140055181552
,
Oct 23 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8964934084976975216
,
Oct 23 2017
For the mean_input_event_latency cases mentioned above, I've reproduced locally and confirmed there's no real regression hiding in the range. The reason it jumps back up and appears worse than before is because the gestures being run previously did not account for zoom correctly. We were trying to scroll more than the content height which resulted in overscrolling which presumably lowered our overall latency. Now that we use the correct distance we don't have overscroll and so the graph is accurate (though higher). Still waiting on the results of the two bisects launched above (for the middle point of #2 from comment 7)
,
Oct 23 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Khushal Commit : 5c6f076df93c934c9955fe2ff20583a89a60b310 Date : Thu Oct 05 04:03:36 2017 Subject: cc: Eliminate required for activation veto. Bisect Details Configuration: android_webview_nexus6_aosp_perf_bisect Benchmark : thread_times.simple_mobile_sites Metric : thread_GPU_cpu_time_per_frame/http___www.ebay.co.uk_ Change : 15.31% | 1.69719102723 -> 1.95706744872 Revision Result N chromium@506436 1.69719 +- 0.0531902 6 good chromium@506556 1.70149 +- 0.170936 6 good chromium@506616 1.68303 +- 0.0697757 6 good chromium@506631 1.68835 +- 0.103963 6 good chromium@506639 1.68821 +- 0.0905627 6 good chromium@506641 1.71739 +- 0.158917 6 good chromium@506642 1.6936 +- 0.103959 6 good chromium@506643 1.9267 +- 0.112163 6 bad <-- chromium@506646 1.86113 +- 0.0882327 6 bad chromium@506675 1.95707 +- 0.248304 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8964934140055181552 For feedback, file a bug with component Speed>Bisection
,
Oct 23 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Khushal Commit : 5c6f076df93c934c9955fe2ff20583a89a60b310 Date : Thu Oct 05 04:03:36 2017 Subject: cc: Eliminate required for activation veto. Bisect Details Configuration: android_webview_arm64_aosp_perf_bisect Benchmark : thread_times.simple_mobile_sites Metric : thread_GPU_cpu_time_per_frame/http___www.ebay.co.uk_ Change : 11.18% | 1.67270166855 -> 1.85975851603 Revision Result N chromium@506590 1.6727 +- 0.168393 6 good chromium@506639 1.64061 +- 0.108456 6 good chromium@506641 1.65095 +- 0.14306 6 good chromium@506642 1.65967 +- 0.109965 6 good chromium@506643 1.84552 +- 0.18984 6 bad <-- chromium@506646 1.87341 +- 0.100557 6 bad chromium@506652 1.85054 +- 0.121723 6 bad chromium@506664 1.86286 +- 0.131174 6 bad chromium@506688 1.87983 +- 0.246049 6 bad chromium@506785 1.85976 +- 0.0598827 6 bad To Run This Test src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8964934084976975216 For feedback, file a bug with component Speed>Bisection |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Oct 18 2017