New issue
Advanced search Search tips

Issue 670227 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Dec 2016
Cc:
perezju@chromium.org
simonhatch@chromium.org
sullivan@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug-Regression

Blocked on: View detail
issue 664765
issue 672524
issue 672183


Sign in to add a comment

1.8MiB regression in gpu_memory at 434480:434535

Project Member Reported by perezju@chromium.org, Dec 1 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Dec 1 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434494  7349213  14682.4  27  good
chromium@434529  7348889  11275.4  27  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.taobao system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_search/load_search_taobao
Relative Change: None

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/931
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994484483136968112


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5811565494796288

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 4 by perezju@chromium.org, Dec 5 2016

Summary: 1.8MiB regression in gpu_memory at 434480:434535 (was: 1.2%-3.8% regression in system_health.memory_mobile at 434480:434535)
I'm ignoring two small v8 regressions (<200KiB)

Remain two gpu_memory regressions:
- One on about:blank, showing an improvement --> regression pattern,
- One on browse:news:washingtonpost

I'll try to bisect on the later.
Project Member

Comment 6 by 42576172...@developer.gserviceaccount.com, Dec 5 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@434479  76592014  5133984  27  good
chromium@434529  76639801  6201147  27  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.washingtonpost system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/browse_news/browse_news_washingtonpost
Relative Change: 0.06%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4404
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994109572113770176


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5456485247614976

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Dec 5 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@434479  5193570  3727898  26  good
chromium@434529  5195245  3715526  27  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.washingtonpost system_health.memory_mobile
Test Metric: memory:chrome:browser_process:reported_by_os:gpu_memory:gl:proportional_resident_size_avg/browse_news/browse_news_washingtonpost
Relative Change: 0.03%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4406
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8994097790628486656


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5297369761972224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 10 by perezju@chromium.org, Dec 6 2016

Cc: sullivan@chromium.org
Another clear regression in #7 that the bisect failed to reproduce.

Comment 11 by sullivan@chromium.org, Dec 7 2016

Cc: simonhatch@chromium.org
+simonhatch, any ideas on #10?

perezju: where do I look in the memory dump in the trace to see if there is a further breakdown of that metric?

Comment 12 by perezju@chromium.org, Dec 7 2016

Labels: -Pri-2 Pri-3
TL;DR: Probably no need to chase a regression here.

Ah, good idea Annie. I started looking at breaking down by process, e.g. as seen here:
https://chromeperf.appspot.com/report?sid=513855b91ab384fd50704c4ae6fb424eed47adc6ae48bfbb43dfeeb131fff337&start_rev=428799&end_rev=436818

- On gpu process we see two big drops at 429292:429420 (23MiB) and 433321:433440 (11MiB).
- On browser process we see a comparatively smaller drop at 429449:429542 (1.8MiB) follow by a regression at 434480:434529 of the same size.

The combined metric appears to show two big drops followed by a small regression (that triggered this alert). But looking closely, the corresponding small improvement is also there at 429449:429542.

This *also* explains the improvement -> regression pattern we clearly see in about:blank.

So, chances are, this is the result of a land/revert pattern and not a real regression.

I'll kick off some more bisects anyway to confirm this.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Dec 7 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@434479  57478561  3755068  27  good
chromium@434529  57474162  3739234  27  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank
Relative Change: 0.01%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4414
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993942471719171376


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5773307704508416

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 16 by simonhatch@chromium.org, Dec 7 2016

Blockedon: 672183

Comment 17 by simonhatch@chromium.org, Dec 7 2016 

re: #11

There's been a few of these, haven't got a clear idea why they're not reproducing yet. I've tried kicking off some bisects without story-filter to see if that's having any effect on the results.
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Dec 8 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Infra Failure: Step('device_status (3)') returned 1
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@434479  55971840  0.0      8  good
chromium@434504  57741312  157683   3  bad
chromium@434529  57488998  1696235  5  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank
Relative Change: 2.71%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4420
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993908658969569792


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5795723138826240

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Dec 8 2016 


===== BISECT JOB RESULTS =====
Status: failed


=== Bisection aborted ===
The bisect was aborted because Bisect cannot identify a culprit: Bisect failed to reproduce the regression with enough confidence.
Please contact the the team (see below) if you believe this is in error.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@434479  57486905  3767297  27  good
chromium@434529  57485540  3765021  27  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank
Relative Change: 0.00%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4425
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993850275043404688


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5813185804763136

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 22 by simonhatch@chromium.org, Dec 8 2016 

Looks like the bisect in #c18 did indeed reproduce the regression when running without story filter (vs the bisect in #c20 which ran with story filter and didn't reproduce). I'll rekick this.
Project Member

Comment 23 by 42576172...@developer.gserviceaccount.com, Dec 9 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@434479  55971840  0.0      5  good
chromium@434504  57765888  186986   5  bad
chromium@434529  57855181  58617.2  5  bad

Bisect job ran on: android_nexus5_perf_bisect
Bug ID: 670227

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests system_health.memory_mobile
Test Metric: memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank
Relative Change: 3.36%

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4427
Job details: https://chromeperf.appspot.com/buildbucket_job_status/8993829033903548656


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6421271032627200

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 24 by simonhatch@chromium.org, Dec 9 2016

Blockedon: 664765

Comment 25 by simonhatch@chromium.org, Dec 12 2016

Blockedon: 672524
#c18 crbug.com/672524

Comment 26 by perezju@chromium.org, Dec 15 2016

Status: WontFix (was: Untriaged)
Closing to clean up our backlog in regression bugs. As per #12, there is probably no unaccounted regression for us to chase here.
Project Member

Comment 30 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:browser_process:reported_by_os:gpu_memory:gl:proportional_resident_size_avg/browse_news/browse_news_washingtonpost

Revision             Result                  N
chromium@434479      5193570 +- 3727898      26      good
chromium@434529      5195245 +- 3715526      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.washingtonpost system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982624184578307840

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5297369761972224


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 33 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/browse_news/browse_news_washingtonpost

Revision             Result                   N
chromium@434479      76592014 +- 5133984      27      good
chromium@434529      76639801 +- 6201147      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=browse.news.washingtonpost system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982615955294281344

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5456485247614976


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 34 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank

Revision             Result                   N
chromium@434479      57478561 +- 3755068      27      good
chromium@434529      57474162 +- 3739234      27      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=blank.about.blank system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982604006363144416

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5773307704508416


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 35 by 42576172...@developer.gserviceaccount.com, Apr 11 2017 


=== BISECT JOB RESULTS ===
Bisect failed unexpectedly

Bisect was aborted with the following:
  Infra Failure: Step('device_status (3)') returned 1


Bisect Details
  Configuration: android_nexus5_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:gpu_memory:proportional_resident_size_avg/blank_about/blank_about_blank

Revision             Result                   N
chromium@434479      55971840 +- 0.0          8      good
chromium@434504      57741312 +- 157683       3      bad
chromium@434529      57488998 +- 1696235      5      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8982580671035479648

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5795723138826240


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
►

Sign in to add a comment