New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 640536 link

Starred by 1 user
Status: WontFix
Owner:
█ nikolaos@chromium.org
Last visit > 30 days ago
Closed: Sep 2016
Cc:
hablich@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 640915


Sign in to add a comment

4.9% regression in speedometer at 413899:413927

Project Member Reported by nikolaos@chromium.org, Aug 24 2016 
See the link to graphs below.

Comment 1 by nikolaos@chromium.org, Aug 24 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=640536

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg3vnjrgoM


Bot(s) for this bug's original alert(s):

android-nexus9
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413898  4770.21  58.9912  18  good
chromium@413927  4780.6   71.449   18  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 1.56%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2058
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003454766230945600


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5782997299625984

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413847  4636.27  16.4853  5  good
chromium@413849  4723.02  79.6669  5  good
chromium@413850  4857.15  35.5201  5  bad
chromium@413852  4826.54  94.4307  5  bad
chromium@413857  4856.69  44.1692  5  bad
chromium@413867  4772.54  29.7983  5  bad
chromium@413887  4822.81  52.7702  5  bad
chromium@413927  4774.19  61.9055  5  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 2.97%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2059
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003429046121083280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5830084905140224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413847  4636.27  16.4853  5  good
chromium@413849  4723.02  79.6669  5  good
chromium@413850  4857.15  35.5201  5  bad
chromium@413852  4826.54  94.4307  5  bad
chromium@413857  4856.69  44.1692  5  bad
chromium@413867  4772.54  29.7983  5  bad
chromium@413887  4822.81  52.7702  5  bad
chromium@413927  4774.19  61.9055  5  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 2.97%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2059
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003429046121083280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5830084905140224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Aug 25 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413898  4820.18  125.358  18  good
chromium@413927  4759.28  58.8354  18  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 0.79%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2064
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003429046121083280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5830084905140224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 10 by sullivan@chromium.org, Aug 25 2016

Blockedon: 641133

Comment 11 by sullivan@chromium.org, Aug 25 2016

Blockedon: -641133 640915
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Aug 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413898  4752.21  90.5578  18  good
chromium@413927  4768.43  81.6813  18  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 2.86%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2071
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002971994755473152


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6450068473774080

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Aug 30 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413847  4600.91  29.6239  5  good
chromium@413849  4664.86  54.6686  5  good
chromium@413850  4856.68  26.4374  5  bad
chromium@413852  4791.61  48.393   5  bad
chromium@413857  4711.48  40.7104  5  bad
chromium@413867  4765.64  79.6872  5  bad
chromium@413887  4780.82  24.3996  5  bad
chromium@413927  4751.48  63.4341  5  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 3.27%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2075
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002949210031628784


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5798557905321984

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Aug 30 2016 


===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413847  4660.54  68.3359  12  good
chromium@413849  4689.89  50.5853  8   good
chromium@413850  4780.24  74.7754  12  bad
chromium@413852  4849.92  72.8927  12  bad
chromium@413857  4737.31  39.112   8   bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: 1.91%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2076
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002919728217933232


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5303966114512896

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 19 by 42576172...@developer.gserviceaccount.com, Sep 26 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2128
Failure reason: the build has failed.
Additional errors:
Either of the initial "good" or "bad" revisions failed to be tested or built.
Project Member

Comment 21 by 42576172...@developer.gserviceaccount.com, Sep 27 2016 

Bisect failed: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2133
Failure reason: the build has failed.
Additional errors:
Either of the initial "good" or "bad" revisions failed to be tested or built.
Project Member

Comment 23 by 42576172...@developer.gserviceaccount.com, Sep 27 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413860  4772.49  55.1135  5  bad

Bisect job ran on: android_nexus9_perf_bisect
Bug ID: 640536

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: React-TodoMVC/React-TodoMVC
Relative Change: None
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/2134
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000379173113430384


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5842581261385728

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 24 by hablich@chromium.org, Sep 28 2016

Cc: hablich@chromium.org
Status: WontFix (was: Assigned)
Flake
►

Sign in to add a comment