Issue metadata
Sign in to add a comment
|
23.3% regression in blink_perf.canvas at 416452:416453 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Sep 9 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9002041730709104096
,
Sep 9 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@416451 110.322 1.28027 18 good chromium@416453 110.345 0.994471 18 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 645313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.canvas Test Metric: putImageData/putImageData Relative Change: 0.90% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4104 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9002041730709104096 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5816418812035072 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 6 2016
Perf fixit: removing perf sheriff as owner of bugs to clarify that the rotation is responsible for triage.
,
Nov 17 2016
Why would we bisect this? It's an improvement!
,
Nov 18 2016
Re-opening. Looks like a valid regression. The y-axis is runs/s, so higher is better.
,
Nov 18 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8995620069876942016
,
Nov 19 2016
=== Auto-CCing suspected CL author perezju@chromium.org === Hi perezju@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : [Telemetry] Switch ShouldTearDownStateAfterEachStoryRun to True as default Author : perezju Commit description: To reduce complexity of benchmarks and imcrease stability of measurements benchmarks will now tear down the state (i.e. restart the browser) between each page/story of a test. Perf sherfiffs warning: This CL is expected to affect measurements of many benchmarks. BUG= chromium:640990 Review-Url: https://codereview.chromium.org/2304893002 Commit : 4f812c9035e6cf9bad6774bb6ff71c1822167e0a Date : Sat Sep 03 10:45:26 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@416263 134.479 32.1516 25 good chromium@416384 135.305 17.4451 25 good chromium@416445 143.976 17.344 25 good chromium@416449 139.348 19.75 25 good chromium@416451 143.14 19.0927 25 good chromium@416452 146.929 14.4159 25 good chromium@416452,catapult@4f812c9035 110.431 6.88509 25 bad <-- chromium@416453 111.028 7.5283 25 bad chromium@416460 110.697 8.47733 25 bad chromium@416475 110.972 7.2581 25 bad chromium@416505 110.156 7.96942 25 bad chromium@416747 110.575 8.25142 25 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 645313 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.canvas Test Metric: putImageData/putImageData Relative Change: 17.77% Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/4357 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8995620069876942016 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5809242664075264 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Nov 21 2016
Not a real regression, but the test being re-baselined. The benchmark now restarts the browser between individual stories, and so we're measuring in a more "cold" scenario. +Ned FYI, +junov benchmark owner. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by briander...@chromium.org
, Sep 9 2016