Issue metadata
Sign in to add a comment
|
14.8%-18% regression in blink_perf.layout at 433544:434478 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Dec 1 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994448634165693440
,
Dec 1 2016
=== PERF REGRESSION === === Auto-CCing suspected CL author nednguyen@google.com === Hi nednguyen@google.com, the bisect results pointed to your CL, please take a look at the results. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Revert of [Telemetry] Disable system_health.memory_desktop on win10 (patchset #1 id:1 of https://codereview.chromium.org/2520423005/ ) Author : nednguyen Commit description: Reason for revert: System health benchmark should be disabled at story level, not the whole benchmark level (if possible). Original issue's description: > [Telemetry] Disable system_health.memory_desktop on win10 > > It crashes on Windows Zen Book, and Windows High DPI: Both win10 configurations. > > BUG= 667941 > CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.perf:linux_perf_cq;master.tryserver.chromium.perf:mac_retina_perf_cq;master.tryserver.chromium.perf:winx64_10_perf_cq > > Committed: https://crrev.com/a3e6a48f576f8130788f731c8d8e6c7525118edf > Cr-Commit-Position: refs/heads/master@{#434132} TBR=eakuefner@chromium.org,fmeawad@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 667941 Review-Url: https://codereview.chromium.org/2522273002 Cr-Commit-Position: refs/heads/master@{#434200} Commit : c1fa4049b9bf140f14ae31b2bd6bdac5b81df4ad Date : Wed Nov 23 18:15:51 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@433559 188.591 20.8672 25 good chromium@433933 188.261 18.4484 25 good chromium@434120 188.537 14.4354 25 good chromium@434167 194.556 34.2936 25 good chromium@434191 199.981 18.536 25 good chromium@434197 197.479 35.2223 25 good chromium@434199 202.627 22.9335 25 good chromium@434200 210.176 17.8501 25 bad <-- chromium@434203 209.4 33.0845 25 bad chromium@434214 205.973 26.2939 25 bad chromium@434307 197.949 19.7967 25 bad Bisect job ran on: linux_perf_bisect Bug ID: 670413 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.shadow_dom Test Metric: ChangingSelect/ChangingSelect Relative Change: 4.96% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6870 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994448634165693440 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5794597152751616 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 1 2016
+simonhatch, dtu: see #3, bisect is not correct. I am not sure blink_perf.shadow_dom is bisectable, though. I'll bisect blink_perf.layout.
,
Dec 1 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994439891535893920
,
Dec 1 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8994439870743314640
,
Dec 2 2016
=== PERF REGRESSION === === Auto-CCing suspected CL author chromeos-commit-bot@chromium.org === Hi chromeos-commit-bot@chromium.org, the bisect results pointed to your CL, please take a look at the results. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Automated Commit: Committing new LKGM version 9011.0.0 for chromeos. Author : chromeos-commit-bot Commit description: Cr-Commit-Position: refs/heads/master@{#433814} Commit : 1e66e96c50f124c0615cc29c0152dcd45ec9c2a5 Date : Tue Nov 22 08:10:37 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@433543 1823.6 38.5743 25 good chromium@433740 1810.16 155.647 25 good chromium@433789 1785.26 109.428 25 good chromium@433802 1814.62 107.753 25 good chromium@433808 1816.42 120.244 25 good chromium@433811 1796.47 102.837 25 good chromium@433813 1803.3 134.274 25 good chromium@433814 1751.53 116.322 25 bad <-- chromium@433838 1757.39 133.052 25 bad chromium@433936 1780.58 99.6244 25 bad chromium@434329 1761.62 109.298 25 bad Bisect job ran on: linux_perf_bisect Bug ID: 670413 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.layout Test Metric: flexbox-row-wrap/flexbox-row-wrap Relative Change: 3.40% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6872 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994439870743314640 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=4566148727701504 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 2 2016
=== PERF REGRESSION === === Auto-CCing suspected CL author bashi@chromium.org === Hi bashi@chromium.org, the bisect results pointed to your CL, please take a look at the results. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : memory coordinator: Set variation parameters if available Author : bashi Commit description: In V0 implementation we have several parameters to determine the global state. Override these parameters when variations are available. BUG= 617492 Review-Url: https://codereview.chromium.org/2518653002 Cr-Commit-Position: refs/heads/master@{#433987} Commit : b935609873e0305e0666b89c4cff2c2a89901bf9 Date : Tue Nov 22 21:39:05 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@433543 3585.84 284.937 25 good chromium@433936 3639.97 320.653 25 good chromium@433986 3718.3 117.454 25 good chromium@433987 3605.03 326.494 25 bad <-- chromium@433988 3664.15 230.062 25 bad chromium@433990 3628.75 144.384 25 bad chromium@433993 3606.66 147.739 25 bad chromium@433999 3636.73 187.576 25 bad chromium@434011 3624.01 449.316 25 bad chromium@434035 3550.68 183.991 25 bad chromium@434133 3496.6 521.125 25 bad chromium@434329 3383.74 420.878 25 bad Bisect job ran on: linux_perf_bisect Bug ID: 670413 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.layout Test Metric: flexbox-column-nowrap/flexbox-column-nowrap Relative Change: 5.64% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6871 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994439891535893920 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5814275350724608 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 2 2016
I don't think my CL caused the regression. The CL changed the code which run only when --enabled-feature=MemoryCoordinator.
,
Dec 5 2016
re: #c4 From what I can see, this is what happens in bisect 3: The reference value are established at roughly 188 (GOOD) and 197 (BAD) chromium@433559 188.591 20.8672 25 good ... other commits chromium@434307 197.949 19.7967 25 bad Couple revisions are tested with values very close to GOOD: chromium@433559 188.591 20.8672 25 good <-- lkgr chromium@433933 188.261 18.4484 25 good <--- STEP chromium@434120 188.537 14.4354 25 good <--- STEP ... other commits chromium@434307 197.949 19.7967 25 bad <-- fkbr Then we get a set of values on chromium@434214 that gives a mean far outside either the good or bad set. Compare Samples gives a REJECT to both, and in revision_state.py:_check_revision_good() when this happens, we declare it either good or bad depending on the distance of the mean of the set to the means of the last known good and first known bad (lkgr/fkbr). chromium@433559 188.591 20.8672 25 good chromium@433933 188.261 18.4484 25 good chromium@434120 188.537 14.4354 25 good <-- lkgr ... other commits chromium@434214 205.973 26.2939 25 bad <--- STEP chromium@434307 197.949 19.7967 25 bad <-- fkbr Similar thing happens on chromium@434167, Compare Samples gives REJECT on both, and GOOD was closer this time vs the BAD revision established in previous iteration. chromium@433559 188.591 20.8672 25 good chromium@433933 188.261 18.4484 25 good chromium@434120 188.537 14.4354 25 good <-- lkgr chromium@434167 194.556 34.2936 25 good <--- STEP ... other commits chromium@434214 205.973 26.2939 25 bad <-- fkbr chromium@434307 197.949 19.7967 25 bad Same thing happens on chromium@434191: chromium@433559 188.591 20.8672 25 good chromium@433933 188.261 18.4484 25 good chromium@434120 188.537 14.4354 25 good chromium@434167 194.556 34.2936 25 good <-- lkgr chromium@434191 199.981 18.536 25 good <--- STEP ... other commits chromium@434214 205.973 26.2939 25 bad <-- fkbr chromium@434307 197.949 19.7967 25 bad Maybe Dave has some ideas on how to handle situations like this?
,
Dec 5 2016
,
Dec 5 2016
Handling of multiple regressions perhaps?
,
Dec 13 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8993377034242768432
,
Dec 14 2016
=== PERF REGRESSION === === Auto-CCing suspected CL author thakis@chromium.org === Hi thakis@chromium.org, the bisect results pointed to your CL, please take a look at the results. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Disable Precompile-platform with clang-cl. Author : thakis Commit description: It currently asserts while trying to build that PCH file. BUG= 667891 , 495697 NOTRY=true Review-Url: https://codereview.chromium.org/2520833005 Cr-Commit-Position: refs/heads/master@{#434028} Commit : 5b7b04e7fa59d53b8b7917d4c0cf4081abdfb0e3 Date : Tue Nov 22 23:57:45 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@433543 1788.94 125.398 30 good chromium@433936 1795.0 170.952 30 good chromium@433986 1817.44 116.149 30 good chromium@434011 1823.07 107.919 30 good chromium@434023 1842.29 69.0779 30 good chromium@434026 1844.08 132.499 30 good chromium@434027 1845.87 245.589 30 good chromium@434028 1781.49 167.705 30 bad <-- chromium@434029 1767.58 241.003 30 bad chromium@434035 1750.39 258.749 30 bad chromium@434133 1755.26 207.858 30 bad chromium@434329 1756.67 176.699 30 bad Bisect job ran on: linux_perf_bisect Bug ID: 670413 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.layout Test Metric: flexbox-row-nowrap/flexbox-row-nowrap Relative Change: 1.80% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6938 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993377034242768432 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5772778819551232 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Dec 14 2016
That's a win-only change affecting only clang, which in win isn't used anywhere yet. Your bisecter seems to be broken.
,
Jul 27 2017
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by lanwei@chromium.org
, Dec 1 2016