New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 631120 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: Aug 2016
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 636336



Sign in to add a comment

30.2% regression in system_health.memory_mobile at 406503:406511

Project Member Reported by petrcermak@chromium.org, Jul 25 2016

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=631120

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICgqoiduwgM


Bot(s) for this bug's original alert(s):

android-nexus5X
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jul 25 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406502  14545237  2092927  12  good
chromium@406511  13357568  1037915  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 10.03%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/367
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006145051510091920


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5754027401281536

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406502  14225067  1779319  12  good
chromium@406530  13304832  2343833  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 5.68%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/448
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005154771229391376


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5876228171497472

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406436  14417237  1676482  12  good
chromium@406521  13766656  1915912  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 1.70%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/462
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004899986646200752


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5899186348752896

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/469
Failure reason: the build has failed.
Additional errors:
The revision range could not be expanded, or the commit positions could not be resolved into commit hashes.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/473
Failure reason: the build has failed due to infrastructure failure.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/472
Failure reason: the build has failed due to infrastructure failure.

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/472
Failure reason: the build has failed due to infrastructure failure.

Blockedon: 636336
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Aug 10 2016

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/473
Failure reason: the build has failed due to infrastructure failure.

Note that the failure here was from yesterday; the bisects today are getting further: https://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/489
Filed https://github.com/catapult-project/catapult/issues/2646 on the update in #18 being late and duplicated with #12.
Project Member

Comment 22 by 42576172...@developer.gserviceaccount.com, Aug 10 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N  Good?
chromium@406502  13731328  1831919  8  good
chromium@406511  12450816  1583697  8  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 7.39%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/491
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004793804045289840


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6396525844889600

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 24 by 42576172...@developer.gserviceaccount.com, Aug 11 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406436  13802155  1871388  12  good
chromium@406525  14045184  1630162  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 1.35%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/501
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004636017301836704


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5794851325476864

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
I'm starting to think that the "regression" is just noise. Unless the next bisect succeeds, I'm most probably going to mark this as such.
Project Member

Comment 28 by 42576172...@developer.gserviceaccount.com, Aug 16 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406392  14518955  2353667  12  good
chromium@406521  14092288  1886660  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 3.17%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/518
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004240073395243168


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5781568384139264

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 30 by 42576172...@developer.gserviceaccount.com, Aug 17 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean      Std Dev  N   Good?
chromium@406502  13399654  2325460  10  good
chromium@406511  13926400  2052121  8   bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 631120

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile
Test Metric: browse_news-memory:chrome:all_processes:reported_by_os:system_memory:java_heap:private_dirty_size_avg/browse_news_washingtonpost
Relative Change: 13.77%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/489
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004705224747321872


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5897142950952960

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Assigned)
The metric is too noisy to bisect.
Labels: SystemHealth-Sheriff
Labels: -Performance-Sheriff

Sign in to add a comment