New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 760174 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 760160
Owner: ----
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

12.6%-45.6% regression in blink_perf.dom at 496951:497059

Project Member Reported by pmeenan@chromium.org, Aug 29 2017

Issue description

See the link to graphs below.
 
Project Member

Comment 1 by 42576172...@developer.gserviceaccount.com, Aug 29 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=760174

(For debugging:) Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?sid=3d7df28433181547e6243d2473a2fbf02bd952655f64042dc6f645d171148747


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 29 2017

Cc: chaopeng@chromium.org
Owner: chaopeng@chromium.org
Status: Assigned (was: Untriaged)

=== Auto-CCing suspected CL author chaopeng@chromium.org ===

Hi chaopeng@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : chaopeng
  Commit : e8aa016b91432b7e599e19f75e60bc7b02e19f69
  Date   : Thu Aug 24 15:20:19 2017
  Subject: Add UX experimental feature flash all scrollbars for any scroll update

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : inner_html_with_selection/inner_html_with_selection
  Change       : 29.77% | 13.7981666667 -> 17.9056666667

Revision             Result                   N
chromium@496950      13.7982 +- 1.64144       6      good
chromium@497005      14.8283 +- 1.02699       6      good
chromium@497032      14.082 +- 1.38046        6      good
chromium@497046      15.1893 +- 0.602892      6      good
chromium@497053      15.406 +- 1.75563        6      good
chromium@497054      18.2091 +- 4.35296       9      bad       <--
chromium@497055      17.7705 +- 1.82023       6      bad
chromium@497056      18.1162 +- 2.77756       6      bad
chromium@497059      17.9057 +- 2.4112        6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969912156655506256


For feedback, file a bug with component Speed>Bisection
Cc: perezju@chromium.org
+perezju

I can not reproduce this performance change locally, also my patch is not related to selection and all behind flag. I don't think it related to my patch.

With my patch:

  "per_page_values": [
    {
      "std": 0.6020527385536018, 
      "name": "inner_html_with_selection", 
      "type": "list_of_scalar_values", 
      "important": true, 
      "values": [
        40.60500000000002, 
        41.47999999999979, 
        41.194999999999936, 
        40.17500000000018, 
        40.13999999999987
      ], 
      "units": "ms", 
      "page_id": 0
    }
  ]

Without my patch:

  "per_page_values": [
    {
      "std": 1.1684305713220686, 
      "name": "inner_html_with_selection", 
      "type": "list_of_scalar_values", 
      "important": true, 
      "values": [
        40.40499999999997, 
        43.625, 
        42.02000000000021, 
        41.49500000000012, 
        41.585000000000036
      ], 
      "units": "ms", 
      "page_id": 0
    }
  ]


Cc: -chaopeng@chromium.org
Owner: ----
Status: Untriaged (was: Assigned)
Sorry, the benchmark is a bit noisy and it looks like it just picked up the CL by accident.  Kicked off (hopefully) a cleaner bisect.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Aug 31 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : chaopeng
  Commit : e8aa016b91432b7e599e19f75e60bc7b02e19f69
  Date   : Thu Aug 24 15:20:19 2017
  Subject: Add UX experimental feature flash all scrollbars for any scroll update

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : inner_html_with_selection/inner_html_with_selection
  Change       : 36.42% | 13.4656666667 -> 18.3698333333

Revision             Result                   N
chromium@496950      13.4657 +- 0.957766      6      good
chromium@497005      15.8773 +- 2.34304       6      good
chromium@497032      14.6183 +- 1.60151       6      good
chromium@497046      15.7617 +- 1.13221       6      good
chromium@497053      15.7108 +- 2.42597       9      good
chromium@497054      18.9937 +- 3.45689       6      bad       <--
chromium@497055      18.082 +- 1.55704        6      bad
chromium@497056      17.9237 +- 2.4516        9      bad
chromium@497059      18.3698 +- 0.764697      6      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969811916367600128


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Aug 31 2017


=== BISECT JOB RESULTS ===
Perf regression found but unable to narrow commit range

Build failures prevented the bisect from narrowing the range further.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : remove_child_with_selection/remove_child_with_selection
  Change       : 11.54% | 46.2286666667 -> 51.5641666667

Suspected Commit Range
  2 commits in range
  https://chromium.googlesource.com/chromium/src/+log/3686eccdb7575bcf4d1630c3572625f3d64beb0f..32c9c46b63856450fb00bbc2bfa51e181da8e908


Revision             Result                   N
chromium@495876      46.2287 +- 1.55717       6        good
chromium@495962      48.8947 +- 0.432022      6        good
chromium@495973      49.6602 +- 0.679199      6        good
chromium@495979      50.3794 +- 2.19092       14       good
chromium@495980      ---                      ---      build failure
chromium@495981      51.6289 +- 3.72879       14       bad
chromium@495982      51.4701 +- 2.92767       9        bad
chromium@495984      52.2512 +- 2.33869       6        bad
chromium@496005      50.8135 +- 2.38753       6        bad
chromium@496047      51.5642 +- 3.7847        6        bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8969809199236276832


For feedback, file a bug with component Speed>Bisection
Cc: -perezju@chromium.org jbroman@chromium.org
+jbroman, owner of blink_perf.dom: These bisects (except #9, which appears to be in the wrong range) consistently report r497054 as the culprit, but as chaopeng says the feature is behind a flag and I can't figure out how it would affect the benchmark. any ideas?
That does seem quite bizarre. I've kicked off another couple bisects under the assumption that the actual regression is earlier, where the graphs first tick up.

It's annoying that some of these regressions come with a massive increase in variation, such that a run after the regression can spuriously show as "good". It might be nice if the bisect bots could do a few runs and then bisect on z-score=1 or something like that.
Cc: dtu@chromium.org simonhatch@chromium.org
Thanks!

+dtu, simonhatch for bisect suggestion in #13
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Oct 19 2017

Mergedinto: 760160
Status: Duplicate (was: Untriaged)

=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Michael Lippautz
  Commit : 146d898b6e27a3900dae50b497e24674d7d2adfb
  Date   : Mon Aug 21 09:24:08 2017
  Subject: [wrapper-tracing] NodeListsNodeData Align Oilpan and wrapper tracing

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : inner_html_with_selection/inner_html_with_selection
  Change       : 45.09% | 10.9668333333 -> 15.9113333333

Revision             Result                   N
chromium@495856      10.9668 +- 0.815618      6      good
chromium@495880      11.2608 +- 0.846219      6      good
chromium@495886      11.9138 +- 1.65413       6      good
chromium@495889      11.7737 +- 1.30782       6      good
chromium@495890      11.6597 +- 0.275839      6      good
chromium@495891      15.6857 +- 2.41175       6      bad       <--
chromium@495892      17.0717 +- 1.10067       6      bad
chromium@495904      16.09 +- 1.42803         6      bad
chromium@495952      16.1535 +- 2.01659       6      bad
chromium@496047      15.9113 +- 1.98468       6      bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965372959496925472


For feedback, file a bug with component Speed>Bisection
Project Member

Comment 16 by 42576172...@developer.gserviceaccount.com, Oct 19 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : Michael Lippautz
  Commit : 146d898b6e27a3900dae50b497e24674d7d2adfb
  Date   : Mon Aug 21 09:24:08 2017
  Subject: [wrapper-tracing] NodeListsNodeData Align Oilpan and wrapper tracing

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : blink_perf.dom
  Metric       : remove_child_with_selection/remove_child_with_selection
  Change       : 7.76% | 46.198 -> 49.7823333333

Revision             Result                   N
chromium@495856      46.198 +- 1.79744        6      good
chromium@495874      46.4253 +- 3.32489       6      good
chromium@495883      45.7097 +- 2.16327       6      good
chromium@495887      45.8978 +- 2.65584       9      good
chromium@495889      46.832 +- 6.26433        9      good
chromium@495890      46.5237 +- 3.00879       6      good
chromium@495891      50.2983 +- 0.630776      9      bad       <--
chromium@495925      49.9777 +- 1.80762       6      bad
chromium@495993      50.9255 +- 1.87514       6      bad
chromium@496130      50.6167 +- 3.47965       6      bad
chromium@496403      49.9247 +- 3.22245       6      bad
chromium@496950      49.7823 +- 1.87053       6      bad

Please refer to the following doc on diagnosing blink_perf regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.dom

More information on addressing performance regressions:
  http://g.co/ChromePerformanceRegressions

Debug information about this bisect:
  https://chromeperf.appspot.com/buildbucket_job_status/8965373099992050992


For feedback, file a bug with component Speed>Bisection

Sign in to add a comment