Issue metadata
Sign in to add a comment
|
1.5%-3.6% regression in page_cycler.intl_ko_th_vi at 383945:383980 |
||||||||||||||||||||
Issue descriptionFairly small regression but worth looking into none the less.
,
Mar 31 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@383956 84113.3888892486.937241 18 good chromium@383959 83216.7777781676.937494 18 bad Bisect job ran on: linux_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests page_cycler.typical_25 Test Metric: vm_private_dirty_final_renderer/vm_private_dirty_final_renderer Relative Change: 0.10% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6365 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016658697829544896 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=599513 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Mar 31 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@383956 41911.416667842.796693 12 good chromium@383976 41761.5 989.503135 12 bad Bisect job ran on: android_nexus9_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --also-run-disabled-tests page_cycler.intl_ar_fa_he Test Metric: vm_private_dirty_final_browser/http___ynet.co.il_ Relative Change: 0.91% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/1699 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016658491253014352 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=599513 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 7 2016
just to update, previous bisect jobs are aborted and alerts are not yet recovered. majidvp@ could you please check the issue
,
Apr 15 2016
Kick of another bisect in https://chromeperf.appspot.com/buildbucket_job_status/9015279966077456944
,
Apr 22 2016
Last one failed at device_status_check step, kick off another one in https://chromeperf.appspot.com/buildbucket_job_status/9014645727377478496
,
Apr 29 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@383948 46204.9 417.266 8 good chromium@383969 48134.5 3050.24 8 bad Bisect job ran on: android_nexus9_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.typical_25 Test Metric: vm_private_dirty_final_browser/http___colorado.edu Relative Change: 6.85% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/1764 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9014085114989083072 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5902295498227712 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Apr 29 2016
Navigated to "https://chromeperf.appspot.com/group_report?bug_id=599513" and observed that alerts on few Graphs have been recovered and few are not. Could some one please look into the issue and update. Thanks in Advance/
,
May 6 2016
===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@383948 46204.9 417.266 8 good chromium@383969 48134.5 3050.24 8 bad Dave, Prasad: can we bisect on std dev?
,
May 13 2016
Some of the graphs had recovered, and there are few more which didn't recover. #9: We do an option to kick-off bisect on std_dev, but Dave or Roberto might comment whether this is handled in statistical analysis of bisect results.
,
May 13 2016
It is not handled, but dtu had mentioned before that the distributions should be different so bisect should catch it (IIUC). I filed a bad bisect report, we can chat more about it in today's meeting about bisects.
,
Jun 17 2016
gentle ping..
,
Jun 17 2016
Based on #9 and #10, I kicked of two stddev bisects: - https://chromeperf.appspot.com/buildbucket_job_status/9009573606910063744 - https://chromeperf.appspot.com/buildbucket_job_status/9009573779822456752
,
Jun 17 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@383947 84354.3 4052.26 17 good chromium@383976 85905.7 3064.12 18 bad Bisect job ran on: linux_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.typical_25 Test Metric: vm_private_dirty_final_renderer/http___www.gamestop.com_ps3 Relative Change: 5.15% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bisect/builds/6539 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9009573779822456752 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5257068697616384 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jun 18 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@383948 39031.7 80.8595 6 good chromium@383969 39053.5 101.915 8 bad Bisect job ran on: android_nexus5_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.intl_ko_th_vi Test Metric: vm_private_dirty_final_renderer/vm_private_dirty_final_renderer Relative Change: 0.07% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus5_perf_bisect/builds/3752 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9009573929909601344 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6438700892815360 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jun 18 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@383948 46512.5 915.384 8 good chromium@383969 45949.4 589.152 8 bad Bisect job ran on: android_nexus9_perf_bisect Bug ID: 599513 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests page_cycler.typical_25 Test Metric: vm_private_dirty_final_browser/http___colorado.edu Relative Change: 0.90% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_nexus9_perf_bisect/builds/1831 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9009573606910063744 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5887046984400896 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jun 24 2016
Kicked off a bisect on ChromiumPerf/linux-release/page_cycler.typical_25 / vm_private_dirty_final_renderer / http___www.gamestop.com_ps3 which shows a very clear regression.
,
Jul 1 2016
3 months old at this point. I'm calling it. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by majidvp@chromium.org
, Mar 31 2016