Issue metadata
Sign in to add a comment
|
3.2%-6.3% regression in speedometer at 418870:418924 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Sep 16 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9001369782443401104
,
Sep 16 2016
=== Auto-CCing suspected CL author agrieve@chromium.org === Hi agrieve@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Move repack_pseudo_locales //chrome -> //chrome/test Author : agrieve Commit description: According to https://www.chromium.org/developers/testing/fake-bidi, it is no longer usable via --locale_pak, and so is used only by the (disabled) test bidi_checker_web_ui_test.cc. BUG=645716 Review-Url: https://codereview.chromium.org/2339513002 Cr-Commit-Position: refs/heads/master@{#418889} Commit : cddda4b3eebaacab696a28aa18f89f92aceda3a9 Date : Thu Sep 15 17:10:08 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418877 757.978 2.14984 5 good chromium@418885 779.493 6.41567 5 good chromium@418887 782.193 11.0235 5 good chromium@418888 786.719 7.41706 5 good chromium@418889 812.385 5.6273 5 bad <-- chromium@418893 801.287 1.02028 5 bad Bisect job ran on: win_x64_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: jQuery-TodoMVC/jQuery-TodoMVC Relative Change: 5.71% Score: 99.5 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1472 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9001369782443401104 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5824175288090624 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 16 2016
Change was a test-only refactoring. Couldn't have affected the build afaict.
,
Sep 16 2016
,
Sep 21 2016
I just did a manual comparison run between Chrome 55 Canary and Chrome 54 Beta on Speedometer. Scores are runs per minute (higher is better): 55 canary: 117.1 117.6 118.4 54 beta: 127.5 126.9 127.1 There was a regression happening on ToT => reopening the bug.
,
Sep 21 2016
Kicking off a few more bisects with clobbered builds. Let's see.
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000908305634125424
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000908280678514464
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000908258686546432
,
Sep 21 2016
=== Auto-CCing suspected CL author hiroshige@chromium.org === Hi hiroshige@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Make internals.isPreloaded() to remain the same before/after clearPreloads() Author : hiroshige Commit description: Previously, internals.isPreloaded() depends on |ResourceFetcher::m_preloads|, which is cleared by ResourceFetcher::clearPreloads(). This caused internals.isPreloaded() to turn false after around document's load event. This CL makes ResourceFetcher to keep a set of preloaded URLs (separately from |m_preloads|) if a blink::Internals object is created, and use the set to calculate internals.isPreloaded(). BUG= 643621 Review-Url: https://codereview.chromium.org/2332333003 Cr-Commit-Position: refs/heads/master@{#418821} Commit : 7978814bba57818814b7a10aa3366e16c6d33784 Date : Thu Sep 15 10:07:33 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418769 1258.83 21.6203 12 good chromium@418798 1260.22 9.67084 5 good chromium@418813 1239.73 14.0796 8 good chromium@418820 1248.71 15.1815 18 good chromium@418821 1262.04 16.2887 18 bad <-- chromium@418822 1274.9 20.4072 12 bad chromium@418824 1274.35 8.08934 8 bad chromium@418827 1280.07 14.2913 12 bad chromium@418884 1287.63 11.7437 12 bad chromium@418999 1316.1 26.0461 8 bad Bisect job ran on: mac_10_11_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: React-TodoMVC/React-TodoMVC Relative Change: 3.81% Score: 99.8 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/916 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000908280678514464 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5051174389022720 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000885430788857840
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000885414096035616
,
Sep 21 2016
> C#11 My change is for layout test and shouldn't affect the behavior unless blink::Internals is created. Also, the bisect result doesn't look significant: chromium@418798 1260.22 9.67084 5 good <-- chromium@418821 1262.04 16.2887 18 bad <-- Re-running bisects.
,
Sep 21 2016
=== Auto-CCing suspected CL author bsalomon@google.com === Hi bsalomon@google.com, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Remove "f" suffix from literal in LCD DF text shader. Author : bsalomon Commit description: Also improve DF GP names GOLD_TRYBOT_URL= https://gold.skia.org/search?issue=2340603003 Review-Url: https://codereview.chromium.org/2340603003 Commit : 32d1e95ca593adfaa12bc48d4ac0d5501a4b2046 Date : Thu Sep 15 14:29:52 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418177 783.383 0.916814 5 good chromium@418581 757.488 5.87542 5 good chromium@418797 759.949 5.62115 5 good chromium@418848 760.591 3.70479 5 good chromium@418873 774.28 5.68569 5 good chromium@418880 762.264 2.00222 5 good chromium@418883 770.927 2.7974 5 good chromium@418885 768.522 7.09481 8 good chromium@418885,skia@1dfec74d6b 773.079 8.84836 8 good chromium@418885,skia@32d1e95ca5 797.875 7.67219 5 bad <-- chromium@418885,skia@e839e78443 787.775 2.31396 8 bad chromium@418886 789.942 5.47406 8 bad chromium@418898 799.241 4.61361 5 bad chromium@418999 807.777 3.91784 5 bad Bisect job ran on: win_x64_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: jQuery-TodoMVC/jQuery-TodoMVC Relative Change: 3.11% Score: 99.5 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1477 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000908305634125424 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6465025814822912 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 21 2016
=== Auto-CCing suspected CL author vabr@chromium.org === Hi vabr@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : c/browser, c/common, components O-P: Change ScopedTempDir::path() to GetPath() Author : vabr Commit description: path() is being deprecated, GetPath() has better checking against wrong use. For more context, see https://codereview.chromium.org/2275553005/. BUG= 640599 Review-Url: https://codereview.chromium.org/2317123002 Cr-Commit-Position: refs/heads/master@{#418804} Commit : 8023d87a36c3eb4aacbf3e9ea427ecf9b572880f Date : Thu Sep 15 08:14:14 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418777 233.491 2.80836 41 good chromium@418791 234.161 2.12538 41 good chromium@418798 233.261 2.32761 62 good chromium@418802 234.22 2.01883 62 good chromium@418803 234.9 2.47903 41 good chromium@418804 236.284 2.30972 41 bad <-- chromium@418805 235.588 2.31297 41 bad chromium@418833 236.881 2.27075 18 bad chromium@418888 237.61 3.17779 12 bad chromium@418999 238.604 2.97788 12 bad Bisect job ran on: win_8_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC Relative Change: 1.40% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2202 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000908258686546432 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6360714506141696 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 22 2016
=== Auto-CCing suspected CL author v8-autoroll@chromium.org === Hi v8-autoroll@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Update V8 to version 5.5.208. Author : v8-autoroll Commit description: Summary of changes available at: https://chromium.googlesource.com/v8/v8/+log/6f1ef12e..fa07ac00 Please follow these instructions for assigning/CC'ing issues: https://github.com/v8/v8/wiki/Triaging%20issues Please close rolling in case of a roll revert: https://v8-roll.appspot.com/ This only works with a Google account. CQ_INCLUDE_TRYBOTS=master.tryserver.blink:linux_precise_blink_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel TBR=hablich@chromium.org,machenbach@chromium.org,littledan@chromium.org,vogelheim@chromium.org Review-Url: https://codereview.chromium.org/2347653002 Cr-Commit-Position: refs/heads/master@{#418891} Commit : c08e4b2781c2f42ef7b55604aa107d46e3abb265 Date : Thu Sep 15 17:15:09 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418869 1281.09 15.2686 12 good chromium@418881 1284.84 12.6789 27 good chromium@418887 1289.03 19.154 27 good chromium@418890 1293.23 19.3298 27 good chromium@418890,v8@b848716c98 1309.08 12.3845 41 good chromium@418890,v8@0e938d4ac0 1310.31 15.2546 41 good chromium@418890,v8@fa07ac0055 1296.89 11.0483 5 good chromium@418891 1320.02 16.5563 27 bad <-- chromium@418892 1308.64 16.9924 8 bad chromium@418893 1305.66 15.3097 27 bad Bisect job ran on: mac_10_11_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: React-TodoMVC/React-TodoMVC Relative Change: 2.65% Score: 98.0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/917 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000885430788857840 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6097221995462656 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 22 2016
=== Auto-CCing suspected CL author avi@chromium.org === Hi avi@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Remove QUIC's Interval::difference(vector<Interval*>*). Author : avi Commit description: It's unused. BUG= 555865 R=bnc@chromium.org Review-Url: https://codereview.chromium.org/2339223003 Cr-Commit-Position: refs/heads/master@{#418890} Commit : 04d50002836c9cc18d131e450d1db50d5cf67cd5 Date : Thu Sep 15 17:12:55 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418884 1279.78 17.2301 27 good chromium@418888 1289.92 15.2446 140 good chromium@418889 1289.23 17.3783 140 good chromium@418890 1295.29 15.8478 93 bad <-- chromium@418892 1299.53 12.0775 12 bad chromium@418899 1293.21 13.1079 18 bad chromium@418913 1291.99 15.8509 27 bad chromium@418942 1302.22 18.3979 27 bad chromium@418999 1300.45 12.0546 18 bad Bisect job ran on: mac_10_11_perf_bisect Bug ID: 647635 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer Test Metric: React-TodoMVC/React-TodoMVC Relative Change: 0.63% Score: 98.0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_11_perf_bisect/builds/918 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000885414096035616 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6696679707770880 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 22 2016
This can't be that CL. That CL removed dead code, literally called nowhere. Filed bug 649352 about the AutoBisect spamming.
,
Sep 22 2016
FWIW the Skia CL that was cited in #15 also is extremely unlikely to be at fault. It changed a literal from "1.0f" to "1.0" in a GLSL shader.
,
Sep 23 2016
I is strange that those alerts cannot be reproduced. I rechecked 55 vs 54 again with --enable-benchmarking. The performance is now approx. the same. My assumption is that there was some hiccup with that flag recently and Ignition was running? Because when I run the benchmarks with Ignition explicitly switched on, I get a similar tankage.
,
Sep 27 2016
One of the issues is that the metrics being bisected on are quite noisy. You can tell that there was a regression when you look at the average between runs before and after the revisions range, but not necessarily by picking individual revisions on either side of it. VanillaJS-TodoMVC in particular isn't a useful metric to bisect on in this case.
,
Sep 27 2016
(Seems like I have nothing to contribute here, feel free to re-add me if you disagree.)
,
Oct 6 2016
Closing this ... seems to be flaky bots. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by nikolaos@chromium.org
, Sep 16 2016