Issue metadata
Sign in to add a comment
|
12.5% regression in system_health.memory_desktop at 410354:410387 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Aug 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004700642382146240
,
Aug 10 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@410353 16384.0 0.0 12 good chromium@410387 16384.0 0.0 8 bad Bisect job ran on: win_perf_bisect Bug ID: 636420 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: browse_social-memory:chrome:all_processes:reported_by_chrome:v8:allocated_by_malloc:effective_size_avg/browse_social_twitter Relative Change: 0.00% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6829 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004700642382146240 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5857330214731776 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 11 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004634535981878912
,
Aug 11 2016
,
Aug 11 2016
=== Auto-CCing suspected CL author erikchen@chromium.org === Hi erikchen@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Revert of Reland #1: "base: Implement GetCurrentThreadPriority." (patchset #2 id:20001 of https://codereview.chromium.org/2215513002/ ) Author : erikchen Commit description: Reason for revert: Still causes startup regressions: https://bugs.chromium.org/p/chromium/issues/detail?id=635464#c4 Original issue's description: > Reland #1: "base: Implement GetCurrentThreadPriority." > > The Chrome thread priority is saved in the thread dictionary during > SetCurrentThreadPriority(). This seemed a better approach than using > thread_policy_get(), since there are 4 Chrome priorities which won't cleanly map > to thread flavors. > > BUG=601270 > > Committed: https://crrev.com/23d10a36f2e815edd610098a65e5890f279d2183 > Cr-Commit-Position: refs/heads/master@{#410163} TBR=mark@chromium.org,gab@chromium.org # Not skipping CQ checks because original CL landed more than 1 days ago. BUG=601270 Review-Url: https://codereview.chromium.org/2221083002 Cr-Commit-Position: refs/heads/master@{#410390} Commit : 23c669b5b4cf228dab6da2cc47293b4650a7b923 Date : Mon Aug 08 17:55:57 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@410386 12640256 0.0 5 good chromium@410389 12640256 0.0 5 good chromium@410390 13699232 0.0 5 bad <-- chromium@410391 13699232 0.0 5 bad chromium@410395 13699232 0.0 4 bad chromium@410404 13699232 0.0 5 bad Bisect job ran on: mac_10_10_perf_bisect Bug ID: 636420 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_tools_stackoverflow Relative Change: 8.38% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2295 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004634535981878912 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6348294368788480 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 11 2016
Something is really broken with the measurement. Someone from the system health team needs to look into it.
,
Aug 11 2016
Petr, can you take a look? We are also seeing some 16384 values here like in bug 636755 .
,
Aug 12 2016
erikchen: What exactly do you mean when you say "something is really broken with the measurement"? The metric on the screenshot in #7 is memory:chrome:all_processes:reported_by_chrome:v8:ALLOCATED_BY_MALLOC:effective_size_avg whereas the bisected metric in #6 is memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg (screenshot attached) The data on the DASHBOARD (https://chromeperf.appspot.com/group_report?bug_id=636420) matches the values in the traces (whose revisions according to their metadata match the range on the dashboard): r410386: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-08_12-51-00-54722.html r410404: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-08_15-19-09-68428.html Similarly, the BISECT values in #6 match the values in the traces (https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2295): r410389: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-14-54-12068.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-30-16-7639.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_08-45-53-37204.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-01-18-72715.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-16-50-52424.html r410390: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-34-11-23055.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_09-49-32-99357.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-04-57-62088.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-20-19-59964.html https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/trace-file-id_32-2016-08-11_10-36-48-3395.html Anyway, the difference between the dashboard traces and bisect traces seems to be exactly the same. Given this and the low standard deviation of the results in #6, I think it's safe to conclude that the regression was caused by 410390. erikchen: Please investigate the issue and decide on further action. sullivan: I think this might be just a coincidence. 16,384 B = 16 KiB (exactly), which is why it might be quite a common value (because allocations usually happen in chunks that are multiples of page size, which is typically 4 KiB).
,
Aug 12 2016
I landed a CL, and shortly thereafter, reverted it. There should be no net change in performance. The graphs that you linked appear very noisy, as evidenced by both the graph I attached, and the graph you attached. I really have no idea what regression bisect is trying to point out. I also don't see how the revert of a CL (whose landing caused no change in the graphs) could be related to this regression(?).
,
Aug 12 2016
#10: OK, this is a fair point and I agree with the noise argument. Thanks for looking into the issue!
,
Aug 12 2016
,
Aug 30 2016
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by petrcermak@chromium.org
, Aug 10 2016