Issue metadata
Sign in to add a comment
|
16.9% regression in blink_perf.parser at 399670:399678 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jun 18 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@399669 76.5498 4.76902 12 good chromium@399678 78.5204 1.18311 8 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 621155 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-last/query-selector-last Relative Change: 4.83% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3016 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9009581989899447696 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5903820173869056 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 3 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@399669 81.5881 3.73976 12 good chromium@399678 81.1516 4.3244 8 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 621155 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-last/query-selector-last Relative Change: 0.91% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3044 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9008138087808825808 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5805701813764096 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 4 2016
Moving this nonessential bug to the next milestone. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Jul 11 2016
,
Jul 22 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9006406766868175088
,
Jul 22 2016
Trying a wider bisect.
,
Jul 23 2016
=== Auto-CCing suspected CL author tzik@chromium.org === Hi tzik@chromium.org, the bisect results pointed to your CL below as possibly causing a regression. Please have a look at this info and see whether your CL be related. ===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Introduce base::IsWeakReceiver for base::Bind to support external weak pointers Author : tzik Commit description: base::IsWeakReceiver is an injection point for external libraries to configure a receiver to behave as a weak pointer. Namely, WTF::WeakPtr, WTF::WeakPersistent and WTF::CrossThreadWeakPersistent in Blink will be specialize base::IsWeakReceiver, and the callback will be cancelled on null instances of them. Review-Url: https://codereview.chromium.org/2048023004 Cr-Commit-Position: refs/heads/master@{#399689} Commit : 1ae80b2655df4f23b87776625de2ae8eb0e03df2 Date : Tue Jun 14 13:21:04 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@399669 79.3234 1.53616 5 good chromium@399683 79.41 1.30433 8 good chromium@399687 77.5605 5.12939 8 good chromium@399688 75.4576 5.25906 5 good chromium@399689 66.9232 1.5013 8 bad <-- chromium@399690 65.3447 2.45234 8 bad chromium@399697 66.5615 0.677081 5 bad chromium@399725 66.8714 0.699173 5 bad chromium@399780 65.5333 1.28732 5 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 621155 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-last/query-selector-last Relative Change: 17.38% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3090 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9006406766868175088 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5781528278204416 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 23 2016
My change was likely not affect the performance of these test. The deviation of the metric was high at that time, and became low around 399766-399829, so something happened around that range.
,
Jul 25 2016
Sorry about that, tzik! Agreed with your comment, the test seemed to be bimodal between the high and low values and now always reports the low value. Assigning to blink_perf.parser owner yukishiino, since the test was previously bimodal it's hard to bisect correctly. This only happens on one device type (nexus 7). Anything that can be done here, or should we WontFix?
,
Aug 1 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005516282862648640
,
Aug 1 2016
Lets try another bisect, but it's possible this will end ups as wontfix.
,
Aug 2 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@399600 67.2844 1.03508 12 good chromium@399700 66.6886 0.902652 8 bad Bisect job ran on: android_nexus7_perf_bisect Bug ID: 621155 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests blink_perf.parser Test Metric: query-selector-last/query-selector-last Relative Change: 0.74% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus7_perf_bisect/builds/3130 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005516282862648640 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5830792651997184 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 3 2016
Because it already recovered (and regressed again), close this specific issue as WontFix. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by majidvp@chromium.org
, Jun 17 2016