Issue metadata
Sign in to add a comment
|
11.8% regression in blink_perf.canvas at 418872:418934 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Sep 21 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9000920989755016320
,
Sep 21 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418871 319.079 10.4331 18 good chromium@418934 320.763 9.10368 18 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 648902 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.canvas Test Metric: putImageData/putImageData Relative Change: 6.48% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1568 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000920989755016320 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6664456984068096 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 22 2016
kicked another bisect with a wider range
,
Sep 27 2016
CC: perfbot sheriff since bisect wasn't reported at all. Copy and paste results manually from https://chromeperf.appspot.com/group_report?bug_id=648902 2016-09-21 11:27:54 ===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418871 319.079 10.4331 18 good chromium@418934 320.763 9.10368 18 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 648902 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.canvas Test Metric: putImageData/putImageData Relative Change: 6.48% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1568 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9000920989755016320 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6664456984068096 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Sep 27 2016
Oops, the result of #6 wasn't for the second bisect of #5. But, according to the link of #5, Status of the bisect job 9000474346997120192: COMPLETED, SUCCESS Master: master.tryserver.chromium.perf Builder: winx64ati_perf_bisect Test: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.canvas Metric: putImageData/putImageData Good revision: 418798 Bad revision: 419073 Job created on (UTC): Mon Sep 26 07:43:22 2016 Job completed on(UTC): Mon Sep 26 09:49:15 2016 Buildbot link: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1571 And builtbot result says "The metric values for the initial \"good\" and \"bad\" revisions do not represent a clear regression." https://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1571/steps/Post%20bisect%20results/logs/stdio The "Post bisect results" step for the buildbot was green, but wasn't reported actually. sullivan@, may I assign you to this?
,
Sep 27 2016
Ive filed https://github.com/catapult-project/catapult/issues/2870 about the bisect update issue.
,
Oct 11 2016
Perf sheriff ping
,
Oct 19 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8998329423724513088
,
Oct 19 2016
Trying another bisect job to check whether https://github.com/catapult-project/catapult/issues/2870 is a consistent problem or a transient problem. Meanwhile, this looks it's it's probably not a real regression: - one bot - no ref build - graph is noisy - bisect didn't reproduce
,
Oct 20 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@418871 325.302 10.2803 18 good chromium@418934 328.191 8.52536 18 bad Bisect job ran on: winx64ati_perf_bisect Bug ID: 648902 Test Command: src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests blink_perf.canvas Test Metric: putImageData/putImageData Relative Change: 1.53% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/winx64ati_perf_bisect/builds/1621 Job details: https://chromeperf.appspot.com/buildbucket_job_status/8998329423724513088 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5797281956102144 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Oct 26 2016
I think we can close this, the graph recovered afterwards and it's so noisy we'd never bisect it.
,
Nov 14 2016
|
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by toyoshim@chromium.org
, Sep 21 2016