Issue metadata
Sign in to add a comment
|
44.1% regression in webrtc.datachannel at 378150:378150 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Mar 17 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@378149 214113.11111115836.16571618 good chromium@378150 207622.22222217470.18333718 bad Bisect job ran on: win_perf_bisect Bug ID: 595719 Test Command: python src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests webrtc.datachannel Test Metric: vm_working_set_final_size_total/vm_working_set_final_size_total Relative Change: 4.92% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6412 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017933062939000288 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 18 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@378130 216280.44444416974.11703718 good chromium@378150 222142.0 24691.19849918 bad Bisect job ran on: win_perf_bisect Bug ID: 595719 Test Command: python src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests webrtc.datachannel Test Metric: vm_working_set_final_size_total/vm_working_set_final_size_total Relative Change: 13.83% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6415 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017847658119501232 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 18 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@378149 211252.44444418137.94903218 good chromium@378150 213552.66666714334.31097318 bad Bisect job ran on: win_perf_bisect Bug ID: 595719 Test Command: python src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests webrtc.datachannel Test Metric: vm_working_set_final_size_total/vm_working_set_final_size_total Relative Change: 4.05% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6416 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017834214202336688 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 21 2016
Adding test owner. phoglund@, any idea what's going on there? There has been a _lot_ of movement in this metric recently.
,
Mar 21 2016
I don't know. It looks like the metric holds together pretty well, but I can't find any reasonable culprits when it goes up nor reverts when it goes down. Could be memory metrics are too noisy for this particular test.
,
Mar 21 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@378149 230508.444444101415.00811918 good chromium@378150 219446.44444427576.30065718 bad Bisect job ran on: win_perf_bisect Bug ID: 595719 Test Command: python src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests webrtc.datachannel Test Metric: vm_working_set_final_size_total/vm_working_set_final_size_total Relative Change: 2.28% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6427 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9017572150307129200 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with label Cr-Tests-AutoBisect. Thank you!
,
Mar 24 2016
,
Mar 29 2016
Another bisect here: https://chromeperf.appspot.com/buildbucket_job_status/9016826369490048864
,
Mar 29 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Value Std. Dev. Num Values Good? chromium@378145 230408.0 13033.55632918 good chromium@378160 234723.77777878442.68596 18 bad Bisect job ran on: win_perf_bisect Bug ID: 595719 Test Command: python src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --also-run-disabled-tests webrtc.datachannel Test Metric: vm_working_set_final_size_total/vm_working_set_final_size_total Relative Change: 22.28% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6440 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9016826369490048864 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=595719 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Mar 30 2016
,
Apr 8 2016
Re #6, sounds like we should definitely not be monitoring. Annie, can you make that change?
,
Apr 11 2016
Re #12: no, it appears to be a waste of time unfortunately.
,
Apr 11 2016
Thanks, Ryan, looks like every test that reports old style memory metrics is monitored; I filed bug 602283 to track. Closing this one. |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by tdres...@chromium.org
, Mar 17 2016