New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 712370 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

1%-62.4% regression in system_health.memory_mobile at 464071:464717

Project Member Reported by benhenry@google.com, Apr 17 2017

Issue description

Graphs to come. Clearly a regression as none of the ref builds are also regressing.
 

Comment 1 by benhenry@google.com, Apr 17 2017

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=712370

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwtfk8AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwpmWrAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggs_83QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwqndugoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwqTJuwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwp7GtgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggrfMsAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggoGPtQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwr6huwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwsr52AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwuOXoAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggrPK8woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgguDO6QkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggr-E9QsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsvIugoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwpWztgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggv3D7wsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwvbzwQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggoGPtQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggteptwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggoDbsQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqHu3wgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwt6XvQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqvnswgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggv3k-wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsXJqQgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwp7_sQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqf1qwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwvjh_AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggpWFjAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqvn0wsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggv_M5ggM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwuiCxgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwsCq6woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggorhtQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwtHG4woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggq-30QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwrHkjgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggu2M0AgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggvK5vgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwqzB_goM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggtHL5QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_LnlkwgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwujm5wkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsv1oQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggtH68AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsu_qQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwpqhtAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggurdiwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggo6xzgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwtzY0QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggorh1QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqmYoQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggrLo1AoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggvK5vgoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_IPM6woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwsD0sgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggv_v8AkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwpiehQkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggq71hwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggvHvmAoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwujm5woM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggqPVyAsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwvbzgQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwpz-tgsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwtzY0QgM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_KupgAkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgwoqmmwoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_L_CoQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggp-DlwkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsXJiQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgws6P9wsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggoOciQoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggv3kuwsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_J_i2QoM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggsu_yQsM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDg_J2owQoM


Bot(s) for this bug's original alert(s):

android-nexus5
android-nexus5X
android-nexus7v2
android-webview-nexus5X
android-webview-nexus6
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 18 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/background_news/background_news_nytimes

Revision             Result                  N
chromium@464072      15206107 +- 596547      14      good
chromium@464717      24540891 +- 690020      14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982028569512560416

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5843295562891264


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Apr 19 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus7_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/background_news/background_news_nytimes

Revision             Result                   N
chromium@464072      15173047 +- 533978       14      good
chromium@464717      24517193 +- 1007343      14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981937217974047088

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5843295562891264


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Apr 21 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/background_news/background_news_nytimes

Revision             Result                   N
chromium@464070      25131886 +- 85806.8      14      good
chromium@464717      35715950 +- 126304       14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=background.news.nytimes system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981755083062997520

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5843295562891264


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Apr 30 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/browse_social/browse_social_twitter

Revision             Result                   N
chromium@464070      58458112 +- 92681.9      14      good
chromium@464717      77042834 +- 9656828      14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.twitter system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981030183400309392

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5843295562891264


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: simonhatch@chromium.org
+simonhatch these are some large-ish regressions, any idea why bisects are failing?
Cc: dtu@chromium.org
Looks like it's hitting the 24 hour limit running the test. Gathering the reference values runs fine and repro's a large regression, but it can't find an mid point cl that's working before hitting 24 hr limit, all failing.

In cases like this where the test may have been broken in the range, the bisect can't do much until we get functionality like optionally patching in a fix, which is a feature Pinpoint will support.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, May 25 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/browse_social/browse_social_twitter

Revision             Result                    N
chromium@464070      64892298 +- 35115667      13      good
chromium@464717      78917339 +- 9265001       14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.twitter system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978781241617157664

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5593517293830144


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, May 26 2017


=== BISECT JOB RESULTS ===
Bisect was unable to run to completion

Please try rerunning the bisect.


If failures persist contact the team (see below) and report the error.


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/browse_social/browse_social_twitter

Revision             Result                    N
chromium@464070      60238994 +- 23289822      14      good
chromium@464717      75713615 +- 20844506      13      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.social.twitter system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8978661511443513328

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5593517293830144


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Jun 12 2017

Cc: jbau...@chromium.org
Owner: jbau...@chromium.org

=== Auto-CCing suspected CL author jbauman@chromium.org ===

Hi jbauman@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : jbauman
  Commit : 7c2bb41a8275988ed20cbf8f5dbaba08a0f9f222
  Date   : Thu Apr 13 23:18:51 2017
  Subject: Do additional blit and Present on first draw to overlay.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_news/load_news_flipboard
  Change       : 9.27% | 13467648.0 -> 14716757.3333

Revision             Result                  N
chromium@464500      13467648 +- 152297      6      good
chromium@464600      13733888 +- 194463      6      good
chromium@464601      16273920 +- 404841      6      bad       <--
chromium@464602      15610197 +- 285876      6      bad
chromium@464604      15144277 +- 395117      6      bad
chromium@464607      15721131 +- 498474      6      bad
chromium@464613      15412395 +- 493220      6      bad
chromium@464625      15026347 +- 172132      6      bad
chromium@464650      14932309 +- 184756      6      bad
chromium@464700      14716757 +- 159681      6      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.news.flipboard system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976998606313197824

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4988094131994624


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Owner: benhenry@chromium.org
My change is windows-only.
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Jun 16 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_media/load_media_imgur


Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.imgur system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976796432837521728

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5105445791858688


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 22 by 42576172...@developer.gserviceaccount.com, Jun 18 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_media/load_media_imgur


Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.media.imgur system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8976615531838372256

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5105445791858688


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 23 by sheriffbot@chromium.org, Jul 19 2017

Labels: Hotlist-Google
Status: WontFix (was: Untriaged)
This is shipped and too old to be actionable.

Sign in to add a comment