Issue metadata
Sign in to add a comment
|
1.1%-14.1% regression in system_health.memory_desktop at 407855:407904 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Aug 1 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005529203281191376
,
Aug 1 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 17590955 682978 12 good chromium@407904 17809408 370728 8 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_media-memory:chrome:all_processes:reported_by_chrome:v8:effective_size_avg/load_media_youtube Relative Change: 3.63% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6776 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005529203281191376 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5831573530738688 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 5 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9005161852206611520
,
Aug 6 2016
===== BISECT JOB RESULTS ===== Status: completed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 25096.0 0.0 5 good chromium@407879 25096.0 0.0 5 good chromium@407882 25096.0 0.0 5 good chromium@407883 25096.0 0.0 5 good chromium@407883,catapult@db2aa902c9 25096.0 0.0 5 good chromium@407884 25484.0 0.0 5 bad chromium@407885 25484.0 0.0 5 bad chromium@407889 25484.0 0.0 5 bad chromium@407904 25484.0 0.0 5 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:font_caches:effective_size_avg/load_search_baidu Relative Change: 1.55% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6802 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9005161852206611520 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5789532209807360 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 8 2016
Don't know what happened here, but the bisect essentially blamed on https://codereview.chromium.org/2181243003, which is pretty much impossible because it changes the Catapult dashboard which has nothing to do with Chrome's executable. I'll try another bisect.
,
Aug 8 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004902103339183904
,
Aug 9 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 58644.0 0.0 5 good chromium@407879 58644.0 0.0 5 good chromium@407882 58644.0 0.0 5 good chromium@407883 58644.0 0.0 5 good chromium@407884 59032.0 0.0 5 bad chromium@407889 59032.0 0.0 5 bad chromium@407904 59032.0 0.0 5 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:font_caches:effective_size_avg/load_tools_weather Relative Change: 0.66% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6811 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004902103339183904 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5829291393155072 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 9 2016
I think the bisect is actually pointing at https://codereview.chromium.org/2181093002 --- can the fake flash version cause performance test differences?
,
Aug 9 2016
+waffles, aiolos see #9
,
Aug 9 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004841982796883104
,
Aug 9 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 58644.0 0.0 5 good chromium@407879 58644.0 0.0 5 good chromium@407882 58644.0 0.0 5 good chromium@407883 58644.0 0.0 5 good chromium@407884 59032.0 0.0 5 bad chromium@407889 59032.0 0.0 5 bad chromium@407904 59032.0 0.0 5 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_tools-memory:chrome:all_processes:reported_by_chrome:font_caches:effective_size_avg/load_tools_weather Relative Change: 0.66% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6811 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004902103339183904 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5829291393155072 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 9 2016
Yes, this is likely https://codereview.chromium.org/2181093002, since before that change, it's possible that the perf test wasn't loading Flash at all (because the version was so low (old) that it was forbidden). See thoughts at https://bugs.chromium.org/p/chromium/issues/detail?id=623804#c18; the guidance I received is that we should be testing with Flash anyways (if Flash is indeed present on the page) as that is closest to the real user experience. Along those lines, maybe close as wontfix?
,
Aug 9 2016
#13: Yes, I agree that we should fix this small regression (388 bytes) as WontFix.
,
Aug 10 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004749872782236464
,
Aug 10 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 25096.0 0.0 5 good chromium@407879 25096.0 0.0 5 good chromium@407882 25096.0 0.0 5 good chromium@407883 25096.0 0.0 5 good chromium@407884 25484.0 0.0 5 bad chromium@407889 25484.0 0.0 5 bad chromium@407904 25484.0 0.0 5 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:font_caches:effective_size_avg/load_search_baidu Relative Change: 1.55% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6827 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004902103339183904 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5829291393155072 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 17 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9004068764670243968
,
Aug 17 2016
===== BISECT JOB RESULTS ===== Status: failed ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@407854 25096.0 0.0 5 good chromium@407879 25096.0 0.0 5 good chromium@407882 25096.0 0.0 5 good chromium@407883 25096.0 0.0 5 good chromium@407884 25484.0 0.0 5 bad chromium@407889 25484.0 0.0 5 bad chromium@407904 25484.0 0.0 5 bad Bisect job ran on: win_perf_bisect Bug ID: 633192 Test Command: src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_desktop Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:font_caches:effective_size_avg/load_search_baidu Relative Change: 1.55% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/6827 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9004902103339183904 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5829291393155072 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 30 2016
,
Aug 30 2016
|
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by petrcermak@chromium.org
, Aug 1 2016