Issue metadata
Sign in to add a comment
|
1.4MB regression in system_health.memory_mobile at 447218:449231 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Feb 8 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988183214898376144
,
Feb 9 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/blank_about/blank_about_blank Revision Result N chromium@447645 102598440 +- 3474955 21 good chromium@447696 103060459 +- 6568885 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988183214898376144 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6160516088594432 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 10 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988006078149185792
,
Feb 11 2017
=== Auto-CCing suspected CL author kjellander@chromium.org === Hi kjellander@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : kjellander Commit : 529ccd92e2a916c7e5650a7222ca09b4406b97da Date : Thu Feb 02 12:09:53 2017 Subject: Update WebRTC video quality test to use statically linked zxing Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:proportional_resident_size_avg/blank_about/blank_about_blank Change : 0.39% | 101965169.143 -> 101570344.0 Revision Result N chromium@447537 101965169 +- 1237633 14 good chromium@447641 102662440 +- 689548 6 good chromium@447693 103426173 +- 199502 6 good chromium@447719 104937597 +- 5759925 6 good chromium@447732 105217320 +- 5598907 6 good chromium@447739 104230013 +- 267363 6 good chromium@447742 103161469 +- 2134174 6 good chromium@447744 102504573 +- 339739 6 good chromium@447745 101570344 +- 527488 9 bad <-- Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988006078149185792 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5294170148700160 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Feb 12 2017
+hjd, primiano, perezju: The bisect in #5 shows a pattern really similar to https://bugs.chromium.org/p/chromium/issues/detail?id=688448#c19 Removing kjellander because it's extremely unlikely his CL is the culprit, even if bisects continue to pinpoint it for the same reason as in the other bug.
,
Feb 13 2017
Right, my CL only affects WebRTC video quality tests running in https://build.chromium.org/p/chromium.webrtc/waterfall and https://build.chromium.org/p/chromium.webrtc.fyi/waterfall - and they only run for the desktop platform.
,
Feb 13 2017
Yep it looks like the same bug although this time it's hiding a real regression, (although it kind of looks like the all_processes:reported_by_os:system_memory:proportional_resident_size_avg is still going up in the graph? [1] in which case maybe it's not real and a bug.) We can track the bisect bug at crbug.com/690042 and the regression here. One option to get this bisect to work right now would be to temporarily change the bot to restart the phone between runs. [1]: https://chromeperf.appspot.com/report?sid=b3f7d52a78eac8098f599c2dd7342e92a9d5c04347b75b03ad51db1bd0201026&start_rev=422373&end_rev=449906
,
Feb 15 2017
There seems to be indeed some 1.4MB regression in overall PSS but, as usual, it's complicated ... I'm widening the range from 447218 (when metrics were stable last time) to 449231 (when metrics appear to be stable again ... at least for a while). Breaking up by processes we see: https://chromeperf.appspot.com/report?sid=d27271b96ab7cc8fce7063dafec83e4085525977f926b3c3cf8e7dde186a69c5&rev=447696 On the browser process: - 447578:447645 +0.8MB - 447931:447964 -0.8MB - 448048:448118 +1.0MB - 448406:448453 -0.5MB - Total: +0.5MB On the renderer processes (ignoring up-downs that cancel each other): - 447746:447776 +0.3MB - 448223:448238 +0.2MB - 448603:448630 +0.2MB - Total: +0.7MB On the gpu process: - 448406:448453 +0.2MB - Total: +0.2MB Grand Total: ~ +1.4MB So many small events adding up to a larger regression :( Not sure what would be the best way to proceed?
,
Feb 15 2017
Should we try to track down those two ranges in Browser that are large? It doesn't look like a 1.4MB regression in overall is a dealbreaker with Android, but this is not great.
,
Feb 16 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8987510755936544624
,
Feb 16 2017
Started a bisect on the second large bump in the browser process. Let's see if we find something there.
,
Feb 16 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:browser_process:reported_by_os:proportional_resident_size_avg/blank_about/blank_about_blank Change : 0.23% | 94271949.7143 -> 94489135.2381 Revision Result N chromium@448047 94271950 +- 2160809 21 good chromium@448118 94489135 +- 2809591 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/None Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6096717805518848 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Feb 16 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:browser_process:reported_by_os:proportional_resident_size_avg/blank_about/blank_about_blank Revision Result N chromium@448047 94292674 +- 1929812 21 good chromium@448118 94490306 +- 2830665 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8987510755936544624 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6684908946194432 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 11 2017
,
Aug 16 2017
This is 6 months old and the bisects didn't reproduce, closing. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by briander...@chromium.org
, Feb 8 2017