New issue
Advanced search Search tips

Issue 775950 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 773021
Owner:
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug-Regression



Sign in to add a comment

49.5% regression in smoothness.sync_scroll.key_mobile_sites_smooth at 508735:508849

Project Member Reported by chiniforooshan@chromium.org, Oct 18 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Oct 18 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=775950

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=b3e7ae3b6649d4312c920c4e30bf49c2f0bacbd6544d4a1ca99fa8823777b749


Bot(s) for this bug's original alert(s):

android-nexus5X
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Oct 18 2017

Mergedinto: 775114
Status: Duplicate (was: Untriaged)

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : David Bokan
  Commit : b0d47f5f4d4b9717bfdd43ae6d40257235d0848f
  Date   : Fri Oct 13 16:49:07 2017
  Subject: Fix coordinate space issues in scroll.js

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : smoothness.sync_scroll.key_mobile_sites_smooth
  Metric       : mean_input_event_latency/http___worldjournal.com_
  Change       : 48.15% | 45.2791666667 -> 67.081

Revision                                 Result                  N
chromium@508734                          45.2792 +- 7.13884      6      good
chromium@508763                          44.747 +- 2.62932       6      good
chromium@508778                          45.0025 +- 2.04772      6      good
chromium@508785                          44.2677 +- 2.44812      6      good
chromium@508789                          44.9917 +- 3.2022       6      good
chromium@508789,catapult@b0d47f5f4d      68.7233 +- 7.41183      6      bad       <--
chromium@508789,catapult@dd5b39ac0c      66.92 +- 7.47891        6      bad
chromium@508790                          67.7732 +- 6.74238      6      bad
chromium@508791                          70.211 +- 8.31557       6      bad
chromium@508792                          68.3372 +- 6.23758      6      bad
chromium@508849                          67.081 +- 6.19212       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...worldjournal.com. smoothness.sync_scroll.key_mobile_sites_smooth

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965384791583210928


For feedback, file a bug with component Speed>Bisection
Labels: OS-Android
Owner: bokan@chromium.org
Status: Assigned (was: Duplicate)
bokan@ could you please take a look and check if the regression is expected or if there is a bug? Some explanation:

This regression is caused by the same CL that caused  crbug.com/775114 . The explanation for  crbug.com/775114  was that chromium@506804 wrongly "improved" these metrics and chromium@508789 just fixed the wrong improvement and so metric values are going up to where they were before chromium@506804.

However, I suspect that there could be a real regression because, unlike in  crbug.com/775114 , the graphs in this bug, e.g. https://chromeperf.appspot.com/group_report?sid=b3e7ae3b6649d4312c920c4e30bf49c2f0bacbd6544d4a1ca99fa8823777b749, go up much more than pre-chromium@506804 levels.

Comment 5 by bokan@chromium.org, Oct 18 2017

Thanks, that graph definitely looks fishy. I'll investigate.

Comment 7 by bokan@chromium.org, Oct 20 2017

Looks like there's a couple issues lumped in here.

1) There's a set of tests that have no matching change from when I landed the orignal "improving" CL. This is likely a separate regression. I've filed  issue 775950  to track those separately.

2) There's another set where the graph recovers or improves but not back to the original level. Within this set, there's three cases:

 - Ref build also moved in the same direction, I think we can ignore these. e.g.:
    https://chromeperf.appspot.com/report?sid=dfd333902e6b7a169adb232688e5c6fabe7f86128c4f22ae58f99b8184b5ba4a&start_rev=504522&end_rev=510471

 - It looks like there was a regression right before I landed my "improve" CL so the dashboard doesn't mark them as regressions, some clearer examples:
    https://chromeperf.appspot.com/report?sid=3aaca1dafd591aa9676f38f45fb39d2f5462628963baf8f568451a752fd8f26e&start_rev=504305&end_rev=510471
    https://chromeperf.appspot.com/report?sid=9f49ab87539bbceae42f8e9d9b7bce025e9ea3b078242d52a1488191b892282d&start_rev=505468&end_rev=510418
    https://chromeperf.appspot.com/report?sid=248f0d27bb62a59e65083fd78b79ebb9f7595da4f3d6bf58dd79ca5c3d5a1593&start_rev=505988&end_rev=510446
   
   I've kicked off a bisect (referencing this bug): https://chromeperf.appspot.com/buildbucket_job_status/8965180205302535376

 - Some don't have the bump before-hand but regressed much higher than expected (these all seem to be mean_input_event_latency):
    https://chromeperf.appspot.com/report?sid=da763bf9cc7186a7ccc03e3ee63afc07cd4dfd8b74598407420f3769230f26c2&start_rev=503982&end_rev=510461
    https://chromeperf.appspot.com/report?sid=55a26229c75a2c92f370e896122a62479a2a592573c6c5416182e64c0be3900f&start_rev=504522&end_rev=510471
   
   These warrant some more investigation.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Oct 20 2017


=== BISECT JOB RESULTS ===
Bisect failed for unknown reasons

Please contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus6_perf_bisect
  Benchmark    : thread_times.simple_mobile_sites
  Metric       : thread_other_cpu_time_per_frame/http___www.ebay.co.uk_


To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965180205302535376


For feedback, file a bug with component Speed>Bisection

Comment 11 by bokan@chromium.org, Oct 23 2017

For the mean_input_event_latency cases mentioned above, I've reproduced locally and confirmed there's no real regression hiding in the range. The reason it jumps back up and appears worse than before is because the gestures being run previously did not account for zoom correctly. We were trying to scroll more than the content height which resulted in overscrolling which presumably lowered our overall latency. Now that we use the correct distance we don't have overscroll and so the graph is accurate (though higher).

Still waiting on the results of the two bisects launched above (for the middle point of #2 from comment 7)
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Oct 23 2017

Mergedinto: -775114 773021
Status: Duplicate (was: Assigned)

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Khushal
  Commit : 5c6f076df93c934c9955fe2ff20583a89a60b310
  Date   : Thu Oct 05 04:03:36 2017
  Subject: cc: Eliminate required for activation veto.

Bisect Details
  Configuration: android_webview_nexus6_aosp_perf_bisect
  Benchmark    : thread_times.simple_mobile_sites
  Metric       : thread_GPU_cpu_time_per_frame/http___www.ebay.co.uk_
  Change       : 15.31% | 1.69719102723 -> 1.95706744872

Revision             Result                    N
chromium@506436      1.69719 +- 0.0531902      6      good
chromium@506556      1.70149 +- 0.170936       6      good
chromium@506616      1.68303 +- 0.0697757      6      good
chromium@506631      1.68835 +- 0.103963       6      good
chromium@506639      1.68821 +- 0.0905627      6      good
chromium@506641      1.71739 +- 0.158917       6      good
chromium@506642      1.6936 +- 0.103959        6      good
chromium@506643      1.9267 +- 0.112163        6      bad       <--
chromium@506646      1.86113 +- 0.0882327      6      bad
chromium@506675      1.95707 +- 0.248304       6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964934140055181552


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Oct 23 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Khushal
  Commit : 5c6f076df93c934c9955fe2ff20583a89a60b310
  Date   : Thu Oct 05 04:03:36 2017
  Subject: cc: Eliminate required for activation veto.

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : thread_times.simple_mobile_sites
  Metric       : thread_GPU_cpu_time_per_frame/http___www.ebay.co.uk_
  Change       : 11.18% | 1.67270166855 -> 1.85975851603

Revision             Result                    N
chromium@506590      1.6727 +- 0.168393        6      good
chromium@506639      1.64061 +- 0.108456       6      good
chromium@506641      1.65095 +- 0.14306        6      good
chromium@506642      1.65967 +- 0.109965       6      good
chromium@506643      1.84552 +- 0.18984        6      bad       <--
chromium@506646      1.87341 +- 0.100557       6      bad
chromium@506652      1.85054 +- 0.121723       6      bad
chromium@506664      1.86286 +- 0.131174       6      bad
chromium@506688      1.87983 +- 0.246049       6      bad
chromium@506785      1.85976 +- 0.0598827      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...www.ebay.co.uk. thread_times.simple_mobile_sites

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8964934084976975216


For feedback, file a bug with component Speed>Bisection

Sign in to add a comment