New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 640531 link

Starred by 1 user
Status: Duplicate
Owner:
█ nikolaos@chromium.org
Last visit > 30 days ago
Closed: Aug 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 640915


Sign in to add a comment

3.1%-5.8% regression in speedometer at 413819:413863

Project Member Reported by nikolaos@chromium.org, Aug 24 2016 
See the link to graphs below.
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413845  173.346  2.84114  8  good
chromium@413848  172.211  3.18992  8  good
chromium@413849  172.45   3.44023  8  good
chromium@413850  178.164  2.48123  8  bad
chromium@413851  179.89   2.75635  8  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC
Relative Change: 4.06%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2328
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003454941821627536


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5798422406234112

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N  Good?
chromium@413845  173.346  2.84114  8  good
chromium@413848  172.211  3.18992  8  good
chromium@413849  172.45   3.44023  8  good
chromium@413850  178.164  2.48123  8  bad
chromium@413851  179.89   2.75635  8  bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC
Relative Change: 4.06%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2328
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003454941821627536


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5798422406234112

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413831  152.938  16.8705  12  good
chromium@413846  146.847  2.64776  5   good
chromium@413848  147.974  2.13993  8   good
chromium@413849  149.404  4.93342  8   good
chromium@413850  156.032  3.89581  8   bad
chromium@413853  155.06   2.68909  5   bad
chromium@413860  154.253  3.0364   8   bad

Bisect job ran on: linux_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC
Relative Change: 4.36%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6664
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003448202138259280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5842668890882048

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Aug 24 2016 


===== BISECT JOB RESULTS =====
Status: failed


===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413831  152.938  16.8705  12  good
chromium@413846  146.847  2.64776  5   good
chromium@413848  147.974  2.13993  8   good
chromium@413849  149.404  4.93342  8   good
chromium@413850  156.032  3.89581  8   bad
chromium@413853  155.06   2.68909  5   bad
chromium@413860  154.253  3.0364   8   bad

Bisect job ran on: linux_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC
Relative Change: 4.36%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6664
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003448202138259280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5842668890882048

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Aug 25 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413845  1407.96  28.8261  18  good
chromium@413851  1407.6   20.54    18  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: BackboneJS-TodoMVC/BackboneJS-TodoMVC
Relative Change: 1.71%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/599
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003454941821627536


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5798422406234112

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Aug 25 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413845  1414.75  24.0751  18  good
chromium@413851  1409.09  24.0373  18  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: BackboneJS-TodoMVC/BackboneJS-TodoMVC
Relative Change: 0.03%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/600
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9003448202138259280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5842668890882048

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 14 by sullivan@chromium.org, Aug 25 2016

Blockedon: 641133

Comment 15 by sullivan@chromium.org, Aug 25 2016

Blockedon: -641133 640915
Whoops, missed that these bisects have status failed. This is actually caused by bug 640915.

Comment 17 by robert...@chromium.org, Aug 29 2016 

Launched new bisect now that the blocking bug is fixed: https://chromeperf.appspot.com/buildbucket_job_status/9002972087716718736
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Aug 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@413818  1419.69  26.4339  18  good
chromium@413851  1427.08  30.7152  18  bad

Bisect job ran on: android_nexus5X_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: BackboneJS-TodoMVC/BackboneJS-TodoMVC
Relative Change: 1.47%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5X_perf_bisect/builds/616
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002972087716718736


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5278283283824640

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Aug 30 2016

Mergedinto: 641656
Status: Duplicate (was: Assigned)

===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : [heap] Tracer: Handle incremental marking scopes
Author  : mlippautz
Commit description:
  
Before this patch all tracing scopes in incremental marking would be reset
during a gc tracer start/stop cycle. This patch handles scopes the same way it
does other incremental marking metrics.

Also:
- Align finalization metric with regular marking metric.
- Smaller cleanups

BUG= chromium:639818 

Review-Url: https://codereview.chromium.org/2273673002
Cr-Commit-Position: refs/heads/master@{#38834}
Commit  : 7695642e2c7d893d300920ac0084905288f72859
Date    : Tue Aug 23 15:13:50 2016


===== TESTED REVISIONS =====
Revision                       Mean     Std Dev  N   Good?
chromium@413845                171.158  3.31851  12  good
chromium@413848                171.202  3.20534  8   good
chromium@413849                170.412  3.97407  8   good
chromium@413849,v8@b305c7dfcb  171.233  3.6839   8   good
chromium@413849,v8@d99b2db1a8  172.193  4.36324  8   good
chromium@413849,v8@7695642e2c  181.258  3.68296  5   bad    <--
chromium@413850                178.121  2.08334  8   bad
chromium@413854                178.861  4.33453  8   bad
chromium@413863                175.081  2.45096  8   bad

Bisect job ran on: mac_10_10_perf_bisect
Bug ID: 640531

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests speedometer
Test Metric: VanillaJS-TodoMVC/VanillaJS-TodoMVC
Relative Change: 2.13%
Score: 99.8

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_10_10_perf_bisect/builds/2341
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002949254773924016


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5866916067934208

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
►

Sign in to add a comment