Issue metadata
Sign in to add a comment
|
10.7% regression in blink_perf.bindings at 514267:514368 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Nov 10 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8963319526810575872
,
Nov 10 2017
=== Auto-CCing suspected CL author vmpstr@chromium.org === Hi vmpstr@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Vladimir Levin Commit : 1eed797d6ce9dd71390ad3ef04efa9bb73043e6f Date : Tue Nov 07 00:24:03 2017 Subject: oop: Fail GrContext creation once and fallback to gpu raster. Bisect Details Configuration: winx64nvidia_perf_bisect Benchmark : blink_perf.bindings Metric : dom-attribute-on-prototoype/dom-attribute-on-prototoype Change : 6.49% | 178.943330619 -> 166.936285254 Revision Result N chromium@514266 178.943 +- 12.0254 9 good chromium@514292 177.267 +- 38.1229 9 good chromium@514305 183.709 +- 1.6571 6 good chromium@514311 181.826 +- 2.26767 6 good chromium@514314 181.36 +- 4.48302 6 good chromium@514315 160.301 +- 16.3126 6 bad <-- chromium@514316 166.922 +- 20.4987 6 bad chromium@514317 166.859 +- 27.3283 14 bad chromium@514368 166.936 +- 3.65828 9 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8963319526810575872 For feedback, file a bug with component Speed>Bisection
,
Nov 10 2017
,
Nov 10 2017
Other than an extra bool in gpu capabilities, this code shouldn't be affecting non-oop path (which isn't enabled right now). This seems a bit suspicious. I'll take a look.
,
Nov 10 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8963288876765176816
,
Nov 10 2017
Lets try another bisect, there's some noise in the numbers.
,
Nov 11 2017
=== BISECT JOB RESULTS === Perf regression found but unable to narrow commit range Build failures prevented the bisect from narrowing the range further. Bisect Details Configuration: winx64nvidia_perf_bisect Benchmark : blink_perf.bindings Metric : dom-attribute-on-prototoype/dom-attribute-on-prototoype Change : 5.88% | 177.844214861 -> 167.388294248 Suspected Commit Range 3 commits in range https://chromium.googlesource.com/chromium/src/+log/7ffe9a068f8dc718ce3d653198c999a0fb595eb5..b5624e0de59bf01c17073f12b3a9f0ce9f83c87d Revision Result N chromium@514200 177.844 +- 36.405 14 good chromium@514300 179.859 +- 3.33181 6 good chromium@514350 181.829 +- 4.73029 6 good chromium@514363 179.241 +- 10.0656 14 good chromium@514364 --- --- build failure chromium@514365 --- --- build failure chromium@514366 176.542 +- 23.5483 14 bad chromium@514369 165.319 +- 7.38177 6 bad chromium@514375 166.987 +- 5.57023 6 bad chromium@514400 167.388 +- 4.79586 9 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8963288876765176816 For feedback, file a bug with component Speed>Bisection
,
Nov 13 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8963052078037162704
,
Nov 13 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8963052062726227840
,
Nov 13 2017
=== Auto-CCing suspected CL author chrome-release-bot@chromium.org === Hi chrome-release-bot@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : chrome-release-bot Commit : 334fca1706da9943215c7b561b2411708959396e Date : Mon Nov 06 03:08:22 2017 Subject: Updating trunk VERSION from 3260.0 to 3261.0 Bisect Details Configuration: winx64ati_perf_bisect Benchmark : blink_perf.bindings Metric : dom-attribute-on-prototoype/dom-attribute-on-prototoype Change : 8.57% | 182.380739682 -> 166.442020844 Revision Result N chromium@514000 182.381 +- 3.60258 9 good chromium@514063 180.899 +- 16.6356 14 good chromium@514079 176.211 +- 10.5876 9 good chromium@514081 175.555 +- 8.49361 9 good chromium@514082 180.509 +- 4.09111 6 bad <-- chromium@514083 180.271 +- 9.8196 9 bad chromium@514087 181.395 +- 2.32801 6 bad chromium@514094 178.719 +- 35.3125 21 bad chromium@514125 170.701 +- 43.422 14 bad chromium@514250 170.532 +- 38.4732 14 bad chromium@514500 160.233 +- 14.7457 9 bad chromium@515000 166.442 +- 4.19625 6 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8963052062726227840 For feedback, file a bug with component Speed>Bisection
,
Nov 14 2017
=== BISECT JOB RESULTS === Bisect was unable to run to completion Error: INFRA_FAILURE The bisect was able to narrow the range, you can try running with: good_revision: 8e58ead89819ea23e43fb7b834594c2325885771 bad_revision : 653458c53bb34850fa3c2fdd46832eb7dbd979e6 If failures persist contact the team (see below) and report the error. Bisect Details Configuration: winx64nvidia_perf_bisect Benchmark : blink_perf.bindings Metric : dom-attribute-on-prototoype/dom-attribute-on-prototoype Revision Result N chromium@514000 182.839 +- 4.17481 14 good chromium@514009 182.635 +- 3.86996 14 good chromium@514012 --- --- build failure chromium@514015 --- --- build failure chromium@514016 --- --- build failure chromium@514017 179.756 +- 31.3555 21 bad chromium@514031 181.033 +- 8.20833 14 bad chromium@514062 178.686 +- 13.3212 9 bad chromium@514125 172.283 +- 38.0751 14 bad chromium@514250 170.367 +- 28.5243 9 bad chromium@514500 160.779 +- 12.8031 6 bad chromium@515000 165.771 +- 3.57043 6 bad Please refer to the following doc on diagnosing blink_perf regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark_harnesses/blink_perf.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.bindings More information on addressing performance regressions: http://g.co/ChromePerformanceRegressions Debug information about this bisect: https://chromeperf.appspot.com/buildbucket_job_status/8963052078037162704 For feedback, file a bug with component Speed>Bisection
,
Jan 12 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/12e7a86f040000
,
Jan 12 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/12e651b7040000
,
Jan 12 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/12ac16df040000
,
Jan 12 2018
Re-kicked bisects on pinpoint so we can get a better idea of what's going on, removing vmpstr as owner since other bisects blamed other CLs.
,
Jan 13 2018
馃搷 Found significant differences after each of 9 commits. https://pinpoint-dot-chromeperf.appspot.com/job/12ac16df040000 Add documentation and TODOs to Keyframe code By smcgruer@chromium.org 路 Mon Nov 06 22:58:14 2017 chromium @ 7136a8a60136ed8724e19053748e34871ff25170 馃懟 Add rules for preferred/verified apps By djacobo@chromium.org 路 Mon Nov 06 23:00:27 2017 chromium @ a0b4070adbd9dedb58085cb9cc343b816f7bdd38 Stop unnecessary creation of ListAttributeTargetObserver By adithyas@chromium.org 路 Mon Nov 06 23:03:43 2017 chromium @ dc78612b571a8ca5729f1125f39ffa0d50632061 chromeos: Delete experimental spurious-power-button flags. By derat@chromium.org 路 Mon Nov 06 23:15:14 2017 chromium @ e5137e3aa92595669c787165b834e5ef4d7a63a3 Check return value from SerializeChanges in SnapshotAccessibilityTree By dmazzoni@chromium.org 路 Mon Nov 06 23:15:45 2017 chromium @ de17d566c402e75179dd38750885d4e7f329124f Fixed factory errors for iOS WebViews. By martis@chromium.org 路 Mon Nov 06 23:16:23 2017 chromium @ d76dd17f6417febfff41789fcec4b7fb542dfece Do not notify ui::LayerDelegate during a threaded animation. By fdoray@chromium.org 路 Mon Nov 06 23:18:03 2017 chromium @ 4b50f620cd644ef5e588d739b39e94b17594503d wpt: querySelector with missing right token By ericwilligers@chromium.org 路 Mon Nov 06 23:18:13 2017 chromium @ e696bfacfee0f956e9d832dab288b795fde7047b oop: Fail GrContext creation once and fallback to gpu raster. By vmpstr@chromium.org 路 Tue Nov 07 00:24:03 2017 chromium @ 1eed797d6ce9dd71390ad3ef04efa9bb73043e6f Understanding performance regressions: http://g.co/ChromePerformanceRegressions
,
Jan 13 2018
,
Jan 14 2018
馃樋 Pinpoint job stopped with an error. https://pinpoint-dot-chromeperf.appspot.com/job/12e7a86f040000
,
Jan 14 2018
馃樋 Pinpoint job stopped with an error. https://pinpoint-dot-chromeperf.appspot.com/job/12e651b7040000
,
Mar 19 2018
I think this benchmark might be noisy? It's unlikely that oop patch I have caused this, since it should be disabled and it doesn't really add any new functionality otherwise. Pinpoint in https://pinpoint-dot-chromeperf.appspot.com/job/12ac16df040000 does seem to show a drop in num of runs for my patch, but there are other drops and error bars seem high :( -> khushalsagar for second opinion
,
Mar 19 2018
The code affected by the oop change is all behind a flag. Out of the changes listed in #17, Stop unnecessary creation of ListAttributeTargetObserver looks like the most probable suspect to me. adithyas@, could you verify that this regression is not from this change?
,
Mar 19 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/16c66ef9440000
,
Mar 19 2018
馃搷 Couldn't reproduce a difference. https://pinpoint-dot-chromeperf.appspot.com/job/16c66ef9440000
,
Mar 21 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/17327319440000
,
Mar 22 2018
馃搷 Couldn't reproduce a difference. https://pinpoint-dot-chromeperf.appspot.com/job/17327319440000
,
Mar 22 2018
I don't think my change caused this. Looking at the source of the test in question (https://crrev.com/55373bc3fb3503bde81d39adb399b506ecadcc89/third_party/WebKit/PerformanceTests/Bindings/dom-attribute-on-prototoype.html), there are no input elements created, so the code changed isn't even executed by the test. Interestingly, looking at the perf report for this bug (https://chromeperf.appspot.com/group_report?bug_id=783694), there are 3 different regression ranges for the 3 different graphs. The intersection of the 3 is the range 514363 - 514368. My guess is there was a V8 change in 514368 (https://crrev.com/6845895ebe71817773eda92f12b00a47c0b356bc) that caused this, but I'm not sure which of the commits here is the culprit (https://chromium.googlesource.com/v8/v8/+log/42e0a23f..31bbfa86). Perhaps someone from the V8 team would have a better idea?
,
Mar 28 2018
馃搷 Pinpoint job started. https://pinpoint-dot-chromeperf.appspot.com/job/11eebdfb440000
,
Mar 31 2018
馃搷 Found significant differences after each of 5 commits. https://pinpoint-dot-chromeperf.appspot.com/job/11eebdfb440000 oop: Fail GrContext creation once and fallback to gpu raster. by vmpstr@chromium.org https://chromium.googlesource.com/chromium/src/+/1eed797d6ce9dd71390ad3ef04efa9bb73043e6f Fix hung streams if a track change never reaches have_enough. by dalecurtis@chromium.org https://chromium.googlesource.com/chromium/src/+/3defa5495fc19bb5115286876dc94e0887874cd2 [ServiceWorker] Eliminate unused function of sw registration status response by xiaofeng.zhang@intel.com https://chromium.googlesource.com/chromium/src/+/923a6d873166c8c4633e96301159462bebaac5b3 [ic] Migrate API getters to data handlers by verwaest@chromium.org https://chromium.googlesource.com/v8/v8/+/460652c9786c686094b71f9d64e3e6025252c11a Drop custom accessor deoptimization support by verwaest@chromium.org https://chromium.googlesource.com/v8/v8/+/c82cd31325c174c92456e6d8a6f9f8a970d805d9 Understanding performance regressions: http://g.co/ChromePerformanceRegressions
,
Apr 9 2018
Even though this microbenchmark may regress, the work is important for real-world performance (it actually gets faster in real-world use-cases) and security. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Nov 10 2017