New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 727300 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jan 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug-Regression



Sign in to add a comment

2.2% regression in system_health.memory_mobile at 474923:475155

Project Member Reported by toyoshim@chromium.org, May 29 2017

Issue description

Probably between 474953:474984 as clankium bots alert.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=727300

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDght755goM


Bot(s) for this bug's original alert(s):

android-nexus7v2
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, May 29 2017

Cc: sigbjo...@opera.com
Owner: sigbjo...@opera.com

=== Auto-CCing suspected CL author sigbjornf@opera.com ===

Hi sigbjornf@opera.com, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : sigbjornf
  Commit : ec3056b0028de68485f6e5c4b6d57f225d334605
  Date   : Fri May 26 08:24:26 2017
  Subject: Enable heap compaction on all 'container' arenas.

Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/load_search/load_search_taobao
  Change       : 2.19% | 23024025.3333 -> 23527492.0

Revision             Result                  N
chromium@474922      23024025 +- 184209      6      good
chromium@474952      23015663 +- 177259      6      good
chromium@474960      22989551 +- 258667      6      good
chromium@474964      23124889 +- 224467      6      good
chromium@474965      23063279 +- 133029      6      good
chromium@474966      23618116 +- 224403      6      bad       <--
chromium@474967      23663855 +- 370710      6      bad
chromium@474981      23635183 +- 196867      6      bad
chromium@475039      23532100 +- 128699      6      bad
chromium@475155      23527492 +- 269247      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978259422223850080

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5504203272749056


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 4 by 42576172...@developer.gserviceaccount.com, May 30 2017

 Issue 727473  has been merged into this issue.
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, May 30 2017

Issue 727469 has been merged into this issue.
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jun 12 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/load_tools/load_tools_docs

Revision             Result                  N
chromium@474857      30613743 +- 453682      21      good
chromium@474935      30602918 +- 484203      21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.docs system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976996684156246480

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6184301579730944


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, Jun 12 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : sigbjornf
  Commit : ec3056b0028de68485f6e5c4b6d57f225d334605
  Date   : Fri May 26 08:24:26 2017
  Subject: Enable heap compaction on all 'container' arenas.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/load_media/load_media_facebook_photos
  Change       : 2.43% | 38106180.0 -> 39032046.6667

Revision             Result                  N
chromium@474900      38106180 +- 280698      6      good
chromium@474950      38563908 +- 383749      6      good
chromium@474963      38366447 +- 304129      6      good
chromium@474965      38540185 +- 391280      6      good
chromium@474966      39029487 +- 342688      6      bad       <--
chromium@474969      39016345 +- 266927      6      bad
chromium@474975      38988697 +- 183765      6      bad
chromium@474999      39032047 +- 303818      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.facebook.photos system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976996655580359280

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6423547498463232


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: haraken@chromium.org keishi@chromium.org
Labels: OS-Android
This is actually good news, I'd argue -- the memset(..,0, ...) followed by RemoveFromHeap() of the unused pages (=> madvise(..., MADV_FREE) of the heap page) after compaction is not causing PSS on Android to be updated & reduced right away. i.e., a measurement problem.

Using MADV_REMOVE (like discardable memory does) on Android is a possibility, if we want to make this more accurate. I think that ought to do it (and it would make the explicit memset()-clearing redundant.)
Sounds good to using MADV_REMOVE.

Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Jun 13 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Error: INFRA_FAILURE

The bisect was able to narrow the range, you can try running with:
  good_revision: c5989b12b1bf78c6a04303e86c498a4ae9460a81
  bad_revision : 09743e506fb47d8ecc66a8bc72c83fb5635c2e21

If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/load_tools/load_tools_docs

Revision             Result                  N
chromium@474857      30345967 +- 196999      6      good
chromium@474896      30747375 +- 301179      6      bad
chromium@474935      30726212 +- 369157      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.docs system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976966592744726240

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6184301579730944


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
ftr, https://codereview.chromium.org/2942233002/ experimented with MADV_REMOVE (and some variations), but I don't see concludeable positive differences on the nexus5 and nexus7 perf results. 

( Results for system_health.memory_mobile are noisy, cf. dummy CL https://codereview.chromium.org/2951673002/ )
Status: Assigned (was: Untriaged)
Explictly assigning. A CL you landed tripped one of the speed metrics we measure in the lab. If this is the first time this has happened to one of your CLs, or if it's been a while, please read: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/addressing_performance_regressions.md

We're looking for one of the following:
1. Justification via explanation
2. Plan to revert or fix
3. Angry rage throwing of equipment at my head

Just be aware that I'm trained in trumpet playing and First Aid and am not afraid to use it.

Note: This was a bulk edit message and not very personal.
Project Member

Comment 16 by sheriffbot@chromium.org, Aug 1 2017

Labels: Hotlist-Recharge-BouncingOwner
Owner: ----
Status: Untriaged (was: Assigned)
The assigned owner "sigbjornf@opera.com" is not able to receive e-mails, please re-triage.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Cc: primiano@chromium.org perezju@chromium.org hjd@chromium.org
Owner: haraken@chromium.org
Status: Assigned (was: Untriaged)
Assigning to haraken, who reviewed the CL. This is a pretty widespread memory regression, ~500kib across dozens of pages/multiple devices. the CL author doesn't get bugmail, so we didn't catch that it wasn't being acted on for a few months. Any idea what to do at this point?
Owner: keishi@chromium.org
Hmm. This looks like a real regression.

keishi@: Would you mind taking a look at this?

This looks strange because the CL enabled the heap compaction, so it should decrease the memory usage. I don't see any reason it will increase the memory usage...

keishi: any update here?
Status: WontFix (was: Assigned)
This seems to have mostly recovered.

Sign in to add a comment