New issue
Advanced search Search tips

Issue 626191 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Jul 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

46.3% regression in smoothness.key_desktop_move_cases at 403793:403824

Project Member Reported by qyears...@chromium.org, Jul 7 2016

Issue description

This regression occurs only on the Maps page, and comes soon after an improvement of comparable magnitude.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=626191

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg0sj1rwkM


Bot(s) for this bug's original alert(s):

chromium-rel-win7-single

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@403792  21.9222  0.760239  18  good
chromium@403824  21.5802  0.423486  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 626191

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.key_desktop_move_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 1.66%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6663
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007837278807669312


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5907542467674112

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Trying another bisect. The regression isn't the same magnitude as the recovery, so I don't think it's something as simple as a revert.
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, Jul 15 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@403792  21.4301  0.402636  18  good
chromium@403824  21.4098  0.455954  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 626191

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.key_desktop_move_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 1.28%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6704
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007038744181312928


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5864356688953344

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 8 by 42576172...@developer.gserviceaccount.com, Jul 19 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@403792  21.5027  0.62462   18  good
chromium@403824  21.6439  0.716045  18  bad

Bisect job ran on: win_perf_bisect
Bug ID: 626191

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.key_desktop_move_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 3.60%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6715
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006693031638710160


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5284925134077952

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Bisect failed to reproduce again -- but the regression looks fairly clear on the graph... tried bisecting on two other windows bisect bots (win_8... and win_x64...)
Project Member

Comment 12 by 42576172...@developer.gserviceaccount.com, Jul 19 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@403792  20.6016  0.295457  18  good
chromium@403824  20.4873  0.243486  18  bad

Bisect job ran on: win_x64_perf_bisect
Bug ID: 626191

Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.key_desktop_move_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 0.07%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_x64_perf_bisect/builds/1334
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006688929223494464


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5873005075365888

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Status: WontFix (was: Assigned)
Consistently failing to reproduce regression.
Project Member

Comment 14 by 42576172...@developer.gserviceaccount.com, Jul 19 2016

Cc: junov@chromium.org
Owner: junov@chromium.org

=== Auto-CCing suspected CL author junov@chromium.org ===

Hi junov@chromium.org, the bisect results pointed to your CL below as possibly
causing a regression. Please have a look at this info and see whether
your CL be related.


===== BISECT JOB RESULTS =====
Status: completed


===== SUSPECTED CL(s) =====
Subject : Change ASSERT to CHECK in DrawingBuffer::reset for investigation
Author  : junov
Commit description:
  
Changing this assert to a CHECK which will re-introduce crashes
related to bug 542857, but will result in crash reports that
will allow us to understand how the DrawingBuffer got into
a bad state.

CL to be reverted before next branch point.

BUG=542857
TBR=kbr@chromium.org

Review-Url: https://codereview.chromium.org/2126603004
Cr-Commit-Position: refs/heads/master@{#403824}
Commit  : 9cd13f726995393db68768a714049ea7650b9f86
Date    : Tue Jul 05 21:30:13 2016


===== TESTED REVISIONS =====
Revision         Mean     Std Dev    N   Good?
chromium@403792  17.3051  0.0947674  12  good
chromium@403809  17.2768  0.109523   93  good
chromium@403817  17.2866  0.119431   89  good
chromium@403821  17.2787  0.115896   61  good
chromium@403823  17.2304  0.133296   17  good
chromium@403824  17.3374  0.141915   93  bad    <--

Bisect job ran on: win_8_perf_bisect
Bug ID: 626191

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.key_desktop_move_cases
Test Metric: mean_frame_time/mean_frame_time
Relative Change: 0.97%
Score: 99.0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_8_perf_bisect/builds/2044
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006688948293089280


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5214987497242624

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Cc: -junov@chromium.org
Owner: qyears...@chromium.org
That bisect result looks like a false positive.

Sign in to add a comment