Issue metadata
Sign in to add a comment
|
1.1%-8.3% regression in system_health.memory_desktop at 478034:478245 |
||||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jun 13 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8976884634330785968
,
Jun 13 2017
=== Auto-CCing suspected CL author dtseng@chromium.org === Hi dtseng@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : dtseng Commit : feef72d0d19c8336b0199b6b7c8ad10b0251f675 Date : Fri Jun 09 05:46:48 2017 Subject: Rich editable text implementation using spannables Bisect Details Configuration: win_x64_perf_bisect Benchmark : system_health.memory_desktop Metric : memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_games/load_games_lazors Change : 8.31% | 6307840.0 -> 6832128.0 Revision Result N chromium@478159 6307840 +- 0.0 6 good chromium@478198 6307840 +- 0.0 6 good chromium@478208 6366094 +- 494303 9 good chromium@478210 6307840 +- 0.0 6 good chromium@478211 6832128 +- 0.0 6 bad <-- chromium@478213 6832128 +- 0.0 6 bad chromium@478218 6832128 +- 0.0 6 bad chromium@478237 6832128 +- 0.0 6 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8976884634330785968 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4609482564304896 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Jun 16 2017
,
Jun 20 2017
,
Jun 23 2017
Highly unlikely it was that cl. The cl adds to a feature that's still disabled by default.
,
Jun 23 2017
Agreed, see extended analysis here: https://bugs.chromium.org/p/chromium/issues/detail?id=736506
,
Jun 26 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8975726769989631376
,
Jun 26 2017
I am not sure I fully agree the rationale of this WontFix (although I agree that the CL blamed in #3 is not a culprit). This bug has, among the other things, 131 alerts on malloc jumping up. The jump on that metric is sharp and stable and doesn't seem to be noise-related (see screenshot attached). I did down manually*, I can see that the jump is visible even in malloc:allocated_objects_size and comes from the browser process. You closed this saying in #7 "See analysis in ..." but Issue 736506 is about Discussion to stop alerting on effective_size for v8. I see very little correlation between v8 and browser's malloc. Without running any bisect, my human neural network says that those 131 alerts are an actual regression and are due to cd0c21dfb32b6235c5c4bed740dac090b4e25bae . This is why I suspect that: if you expand one of those charts, you see that there was another up&down movement on the same metric (see screenshot). I manually inspected those ranges and eye balled the 3 ranges in the /--\___/ pattern. Here's what I see: - The first jump up happened in 475281 - 475313: "Enable PrefService by default" is in that range - The ump down happened in 475439 - 475489: "Revert "Enable PrefService by default" is in that range - The alert is in 478067 - 478158: "Enable PrefService by default". is in that range (it's a reland) At this point, don't you find there is an extremely strong correlation between PrefService being landed/reverted/relanded and the shape of that metric? I kicked another bisect for you https://chromeperf.appspot.com/buildbucket_job_status/8975726769989631376 , let's see if that matches my predictions. Reopening this bug because I genuinely think that there is a real regression hiding behind this WontFix which was just accidentally mixed with another alert-due-to-noise. (I think, yet again, we are getting hit by the fact that the batches of CLs are too large and too many things overlap) * https://chromeperf.appspot.com/report?sid=4ba5a3c70d47accb184e384462678d7c195887cb7b138085a97ce3bbef774593 +sullivan,+benhenry FYI for the problem "batching too many CLs mixes together too many different causes and can cause blaming/wontfixing wrong CLs"
,
Jun 26 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : Sam McNally Commit : cd0c21dfb32b6235c5c4bed740dac090b4e25bae Date : Fri Jun 09 01:31:36 2017 Subject: Enable PrefService by default. Bisect Details Configuration: winx64_10_perf_bisect Benchmark : system_health.memory_desktop Metric : memory:chrome:browser_process:reported_by_chrome:malloc:allocated_objects_size_avg/load_media/load_media_youtube Change : 0.52% | 74393382.5556 -> 74730555.2222 Revision Result N chromium@478066 74393383 +- 814233 9 good chromium@478112 74385290 +- 812966 9 good chromium@478135 74401913 +- 820508 9 good chromium@478147 74370573 +- 747972 9 good chromium@478153 74349996 +- 771495 9 good chromium@478156 74367008 +- 776470 9 good chromium@478157 74679586 +- 794245 9 bad <-- chromium@478158 74730555 +- 1024585 9 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.youtube system_health.memory_desktop Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8975726769989631376 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5803844165959680 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Jun 26 2017
Ta-daa! I love those well behaved bisects, especially when they match analytical predictions |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by erikc...@chromium.org
, Jun 13 2017