New issue
Advanced search Search tips

Issue 631432 link

Starred by 2 users
Status: WontFix
Owner:
tdres...@chromium.org
Closed: Aug 2016
Cc:
tdres...@chromium.org
sullivan@chromium.org
primiano@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

77.2% regression in smoothness.sync_scroll.key_mobile_sites_smooth at 407020:407072

Project Member Reported by primiano@chromium.org, Jul 26 2016 
See the link to graphs below.

Comment 1 by primiano@chromium.org, Jul 26 2016 

All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=631432

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg2p6hqAoM


Bot(s) for this bug's original alert(s):

android-nexus6
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Jul 26 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  10.083   2.89005  12  good
chromium@407072  10.8207  2.1182   12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 0.76%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2345
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006079394014931072


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4895243321737216

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jul 27 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  8.36356  2.10333  9   good
chromium@407072  11.9116  9.10235  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 11.29%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2358
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005983402570573696


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5804077351436288

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Jul 28 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  10.1628  2.38968  12  good
chromium@407072  13.9597  9.89422  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 79.67%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2361
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005904546749568240


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5907101000400896

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 9 by primiano@chromium.org, Jul 29 2016

Owner: tdres...@chromium.org
Assigning to benchmark owner, I'm going to leave for holidays in ~days and couldn't get anything out of this during my shift :/

Comment 11 by tdres...@chromium.org, Jul 29 2016 

Appears to be page and device specific:
https://chromeperf.appspot.com/report?sid=97bb6878b7c7368138aa018c8374ed8cea5fa71f43148a305f349d4deeceeb19

I'm guess this won't repro, but launched a stddev bisect, which might have better luck catching the problem than the mean bisect?

https://chromeperf.appspot.com/buildbucket_job_status/9005802801908609008

Comment 15 by primiano@chromium.org, Jul 29 2016

Cc: primiano@chromium.org
 Issue 631430  has been merged into this issue.

Comment 16 by sullivan@chromium.org, Jul 29 2016

Cc: tdres...@chromium.org sullivan@chromium.org
 Issue 631431  has been merged into this issue.
Project Member

Comment 17 by 42576172...@developer.gserviceaccount.com, Jul 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  17.1992  15.6514  12  good
chromium@407072  12.5662  9.37476  11  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 12.14%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2375
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005805697382978688


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5239120708763648

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, Jul 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  16.6006  17.842   9   good
chromium@407072  9.09167  1.89715  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 68.74%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2376
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005802801908609008


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5862686877810688

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 20 by 42576172...@developer.gserviceaccount.com, Jul 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  10.3208  1.84798  11  good
chromium@407072  9.88913  2.22891  8   bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 1.91%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2377
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005798570683926992


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5890149435572224

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 21 by 42576172...@developer.gserviceaccount.com, Jul 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  16.0867  14.5679  9   good
chromium@407072  10.7116  2.33409  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 38.92%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2378
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005797088278763920


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5875170468691968

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 22 by 42576172...@developer.gserviceaccount.com, Jul 29 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  9.25567  2.3133   12  good
chromium@407072  11.0347  1.14095  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 22.76%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2379
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005796902972076960


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5901685180858368

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 23 by 42576172...@developer.gserviceaccount.com, Jul 30 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  12.1075  9.71208  11  good
chromium@407072  14.1248  12.8652  12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 70.48%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2383
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005784207259126176


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5775616213778432

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 25 by 42576172...@developer.gserviceaccount.com, Aug 2 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  12.668   7.02691  12  good
chromium@407072  10.4617  2.3322   12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 10.04%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2404
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005421214324220960


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6457447170965504

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 27 by tdres...@chromium.org, Aug 4 2016 

Last bisect requested to bypass no repro check.
I'm pretty sure I did this last time too, but it keeps claiming no repro...
Project Member

Comment 28 by 42576172...@developer.gserviceaccount.com, Aug 4 2016 


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@407019  9.7285   2.09746  12  good
chromium@407072  8.63125  1.9228   12  bad

Bisect job ran on: android_nexus6_perf_bisect
Bug ID: 631432

Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.sync_scroll.key_mobile_sites_smooth
Test Metric: mean_input_event_latency/http___www.baidu.com_s?wd_barack+obama_rsv_bp_0_rsv_spt_3_rsv_sug3_9_rsv_sug_0_rsv_sug4_3824_rsv_sug1_3_inputT_4920
Relative Change: 15.63%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus6_perf_bisect/builds/2417
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005236029494605696


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5898826141925376

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

Comment 29 by tdres...@chromium.org, Aug 5 2016

Status: WontFix (was: Assigned)
Oh, this is failing despite the "bypass no repro" check because I'm seeing a change in the wrong direction...

This isn't reproducing, marking WontFix.
►

Sign in to add a comment