New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 609788 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 609482
Owner:
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

5.1% regression in smoothness.gpu_rasterization.tough_path_rendering_cases at 391403:391429

Project Member Reported by alexclarke@chromium.org, May 6 2016

Issue description

Might be a duplicate of  http://crbug.com/609482   if so we can close this.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=609788

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgICg1K60rgoM


Bot(s) for this bug's original alert(s):

chromium-rel-mac-hdd

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@391402  28.0565  0.301643  18  good
chromium@391429  28.1789  0.947959  18  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 609788

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases
Test Metric: frame_times/frame_times
Relative Change: 0.93%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/533
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013418010517788064


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5853581308067840

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!

===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev  N   Good?
chromium@391402  30.295   6.06366  18  good
chromium@391429  29.3711  3.6421   17  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 609788

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases
Test Metric: frame_times/frame_times
Relative Change: 1.95%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/534
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013418001462583184


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5885720212799488

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 5 by 42576172...@developer.gserviceaccount.com, May 12 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@391400  28.6941  0.845563  18  good
chromium@391430  28.6098  0.460685  18  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 609788

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases
Test Metric: frame_times/frame_times
Relative Change: 1.09%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/547
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9012831859814153696


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5780690679889920

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Owner: noel@chromium.org
Noel could this be due to https://codereview.chromium.org/1937433002?

Comment 7 by noel@chromium.org, May 26 2016

Cc: noel@chromium.org senorblanco@chromium.org alexclarke@chromium.org
Owner: ----
Status: Available (was: Assigned)
There are two test pages in this page set: some <canvas> GUI benchmark and an ie-edge benchmark called "ChalkBoard".  senorblanco@ added this page set.

Neither use <canvas>.toDataURL based on my read of theie benchmark code.  My change was to <canvas>.toDatatURL ... for WEBP images :)

> Might be a duplicate of  http://crbug.com/609482   if so we can close this.

Maybe not - that came after the ranges reported in this bug.  From #3, the tightest range we have is chromium@391402 .. chromium@391429

   chromium@391402  28.0565  0.301643  18  good
   chromium@391429  28.1789  0.947959  18  bad

Comment 8 by noel@chromium.org, May 26 2016

Seems mac-only iirc,  There are a few mac changes in that range.  I started a bisect on 391402 ... 391428.

Comment 9 by noel@chromium.org, May 26 2016

Also, https://chromeperf.appspot.com/group_report?bug_id=609788 we can look at the graph for the two test pages individually:

  - <canvas> path test (GUI benchmark) did not change.  
  - SVG path test (ie-edge Chalkboard) did change.

Seems to be a regression in SVG path rendering, and so not my change for sure.
Project Member

Comment 10 by 42576172...@developer.gserviceaccount.com, May 27 2016


===== BISECT JOB RESULTS =====
Status: completed


=== Bisection aborted ===
The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression.
Please contact the the team (see below) if you believe this is in error.

=== Warnings ===
The following warnings were raised by the bisect job:

 * Bisect failed to reproduce the regression with enough confidence.

===== TESTED REVISIONS =====
Revision         Mean     Std Dev   N   Good?
chromium@391402  28.0739  0.37438   18  good
chromium@391428  28.0393  0.828691  18  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 609788

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases
Test Metric: frame_times/frame_times
Relative Change: 2.07%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/562
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9011560729528062512


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5884224607879168

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 11 by sheriffbot@chromium.org, Jun 1 2016

Labels: -M-52 M-53 MovedFrom-52
Moving this nonessential bug to the next milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -performance-sheriff Performance-Sheriff
Looks like the metric has partially recovered.  I'll try once more with the bisect.

===== BISECT JOB RESULTS =====
Status: completed


===== TESTED REVISIONS =====
Revision                         Mean     Std Dev   N  Good?
chromium@391300                  28.3418  0.171507  5  good
chromium@391400                  28.2194  0.382445  5  good
chromium@391425                  27.9336  0.469302  5  good
chromium@391432                  28.0229  0.419488  5  good
chromium@391435                  28.2307  0.396476  5  good
chromium@391435,skia@00d44e014c  28.284   0.266932  5  good
chromium@391436                  29.86    0.714385  5  bad
chromium@391437                  29.7713  0.504735  5  bad
chromium@391438                  29.4544  0.434397  5  bad
chromium@391450                  29.4652  0.727625  5  bad
chromium@391500                  29.6022  0.85032   5  bad

Bisect job ran on: mac_hdd_perf_bisect
Bug ID: 609788

Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases
Test Metric: frame_times/frame_times
Relative Change: 4.45%
Score: 0

Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/629
Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007708145332295808


Not what you expected? We'll investigate and get back to you!
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5840771027042304

| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Tests>AutoBisect.  Thank you!
Project Member

Comment 14 by sheriffbot@chromium.org, Jul 11 2016

Labels: -M-53 MovedFrom-53
This issue has been moved once and is lower than Pri-1. Removing the milestone.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Owner: ericrk@chromium.org
Let me assign ericrk@chromium.org to check if this bug is reasonable to be open.

> chromium@391436                  29.86    0.714385  5  bad
This change was a skia roll, https://codereview.chromium.org/1950843002 for two skia changes, https://chromium.googlesource.com/skia.git/+log/02125d10d5d0..0736f3386820

And one of them was made by ericrk@chromium.org. That's why.
Status: Assigned (was: Available)
Perf sheriff checking in: Any update on this regression ericrk@?
Mergedinto: 609482
Status: Duplicate (was: Assigned)
From looking at the most recent bisect, it looks like this is a duplicate of 609482 (as the initial comment mentioned). I know we found tighter regression ranges earlier, but I think those may have been noise or bisect errors.

This bug lines up with the previous one, in that there was an improvement in these metrics at  https://codereview.chromium.org/1912063002, which introduces a bug - this improvement is removed by the "regression" cited in this issue, which fixes the bug that caused the performance improvement (but led to incorrect rendering).

Sign in to add a comment