Issue metadata
Sign in to add a comment
|
5.1% regression in smoothness.gpu_rasterization.tough_path_rendering_cases at 391403:391429 |
||||||||||||||||||||||||
Issue descriptionMight be a duplicate of http://crbug.com/609482 if so we can close this.
,
May 6 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@391402 28.0565 0.301643 18 good chromium@391429 28.1789 0.947959 18 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 609788 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases Test Metric: frame_times/frame_times Relative Change: 0.93% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/533 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013418010517788064 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5853581308067840 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 7 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@391402 30.295 6.06366 18 good chromium@391429 29.3711 3.6421 17 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 609788 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases Test Metric: frame_times/frame_times Relative Change: 1.95% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/534 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9013418001462583184 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5885720212799488 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 12 2016
Re-kicked bisect by expanding revision range: https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/547
,
May 12 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@391400 28.6941 0.845563 18 good chromium@391430 28.6098 0.460685 18 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 609788 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases Test Metric: frame_times/frame_times Relative Change: 1.09% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/547 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9012831859814153696 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5780690679889920 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
May 26 2016
,
May 26 2016
There are two test pages in this page set: some <canvas> GUI benchmark and an ie-edge benchmark called "ChalkBoard". senorblanco@ added this page set. Neither use <canvas>.toDataURL based on my read of theie benchmark code. My change was to <canvas>.toDatatURL ... for WEBP images :) > Might be a duplicate of http://crbug.com/609482 if so we can close this. Maybe not - that came after the ranges reported in this bug. From #3, the tightest range we have is chromium@391402 .. chromium@391429 chromium@391402 28.0565 0.301643 18 good chromium@391429 28.1789 0.947959 18 bad
,
May 26 2016
Seems mac-only iirc, There are a few mac changes in that range. I started a bisect on 391402 ... 391428.
,
May 26 2016
Also, https://chromeperf.appspot.com/group_report?bug_id=609788 we can look at the graph for the two test pages individually: - <canvas> path test (GUI benchmark) did not change. - SVG path test (ie-edge Chalkboard) did change. Seems to be a regression in SVG path rendering, and so not my change for sure.
,
May 27 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@391402 28.0739 0.37438 18 good chromium@391428 28.0393 0.828691 18 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 609788 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases Test Metric: frame_times/frame_times Relative Change: 2.07% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/562 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9011560729528062512 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5884224607879168 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jun 1 2016
Moving this nonessential bug to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 8 2016
Looks like the metric has partially recovered. I'll try once more with the bisect.
,
Jul 8 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@391300 28.3418 0.171507 5 good chromium@391400 28.2194 0.382445 5 good chromium@391425 27.9336 0.469302 5 good chromium@391432 28.0229 0.419488 5 good chromium@391435 28.2307 0.396476 5 good chromium@391435,skia@00d44e014c 28.284 0.266932 5 good chromium@391436 29.86 0.714385 5 bad chromium@391437 29.7713 0.504735 5 bad chromium@391438 29.4544 0.434397 5 bad chromium@391450 29.4652 0.727625 5 bad chromium@391500 29.6022 0.85032 5 bad Bisect job ran on: mac_hdd_perf_bisect Bug ID: 609788 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests smoothness.gpu_rasterization.tough_path_rendering_cases Test Metric: frame_times/frame_times Relative Change: 4.45% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/mac_hdd_perf_bisect/builds/629 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007708145332295808 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5840771027042304 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 11 2016
This issue has been moved once and is lower than Pri-1. Removing the milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Aug 2 2016
Let me assign ericrk@chromium.org to check if this bug is reasonable to be open. > chromium@391436 29.86 0.714385 5 bad This change was a skia roll, https://codereview.chromium.org/1950843002 for two skia changes, https://chromium.googlesource.com/skia.git/+log/02125d10d5d0..0736f3386820 And one of them was made by ericrk@chromium.org. That's why.
,
Aug 2 2016
,
Aug 8 2016
Perf sheriff checking in: Any update on this regression ericrk@?
,
Aug 8 2016
From looking at the most recent bisect, it looks like this is a duplicate of 609482 (as the initial comment mentioned). I know we found tighter regression ranges earlier, but I think those may have been noise or bisect errors. This bug lines up with the previous one, in that there was an improvement in these metrics at https://codereview.chromium.org/1912063002, which introduces a bug - this improvement is removed by the "regression" cited in this issue, which fixes the bug that caused the performance improvement (but led to incorrect rendering). |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by alexclarke@chromium.org
, May 6 2016