Issue metadata
Sign in to add a comment
|
5% regression in system_health.memory_mobile at 405269:405324 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Jul 14 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9007143277190477136
,
Jul 14 2016
===== BISECT JOB RESULTS ===== Status: completed === Bisection aborted === The bisect was aborted because The metric values for the initial "good" and "bad" revisions do not represent a clear regression. Please contact the the team (see below) if you believe this is in error. === Warnings === The following warnings were raised by the bisect job: * Bisect failed to reproduce the regression with enough confidence. ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@405268 77854017 826309 12 good chromium@405324 78802188 987497 8 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 628288 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:malloc:effective_size_avg/load_search_yahoo Relative Change: 1.35% Score: 0 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/794 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007143277190477136 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=5263032544919552 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 15 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9007062707693250816
,
Jul 15 2016
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/9007060871539525584
,
Jul 15 2016
,
Jul 16 2016
===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add benchmark that imitates news reading. Author : ulan Commit description: Each story in the benchmark does the following: 1. Load the main page. 2. Open and scroll the first news item. 3. Go back to the main page and scroll it. 4. Open and scroll the second news item. 5. Go back to the main page and scroll it. 6. etc. This workflow is interesting because it is close to real world and generates a lot of garbage by creating and discarding pages. A couple of websites crash with OOM on svelte device, so by tracking and optimizing for this benchmark we can improve user experience on low-memory devices. *** NOTE TO SHERRIF *** Regressions are expected in system_health.memory_mobile and system_health.memory_desktop for the following stories: - load:news:cnn - load:social:facebook - load:news:qq - load:news:reddit - load:social:twitter If there are failing tests in form of benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.* please do not revert this CL but instead disabling failing test by adding them into the _DISABLE_TESTS list in tools/perf/benchmarks/system_health_smoke_test.py BUG=589726 CQ_INCLUDE_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2118293002 Cr-Commit-Position: refs/heads/master@{#405212} Commit : 5b1847df818e5be4592e9a6ac4832350bb663023 Date : Wed Jul 13 17:39:28 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@405036 75884179 1013516 5 good chromium@405180 75251653 731270 5 good chromium@405198 75313872 224645 5 good chromium@405207 75781681 754439 5 good chromium@405210 74768313 835734 8 good chromium@405211 76395360 1349606 8 good chromium@405212 78017583 577303 5 bad <-- chromium@405216 77821660 677681 5 bad chromium@405252 78347973 1181772 5 bad chromium@405324 79063634 1045016 5 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 628288 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:malloc:effective_size_avg/load_search_yahoo Relative Change: 4.19% Score: 99.9 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/799 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007062707693250816 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=6172128357908480 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Jul 16 2016
===== BISECT JOB RESULTS ===== Status: completed ===== SUSPECTED CL(s) ===== Subject : Add benchmark that imitates news reading. Author : ulan Commit description: Each story in the benchmark does the following: 1. Load the main page. 2. Open and scroll the first news item. 3. Go back to the main page and scroll it. 4. Open and scroll the second news item. 5. Go back to the main page and scroll it. 6. etc. This workflow is interesting because it is close to real world and generates a lot of garbage by creating and discarding pages. A couple of websites crash with OOM on svelte device, so by tracking and optimizing for this benchmark we can improve user experience on low-memory devices. *** NOTE TO SHERRIF *** Regressions are expected in system_health.memory_mobile and system_health.memory_desktop for the following stories: - load:news:cnn - load:social:facebook - load:news:qq - load:news:reddit - load:social:twitter If there are failing tests in form of benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.* please do not revert this CL but instead disabling failing test by adding them into the _DISABLE_TESTS list in tools/perf/benchmarks/system_health_smoke_test.py BUG=589726 CQ_INCLUDE_TRYBOTS=tryserver.chromium.perf:android_s5_perf_cq;tryserver.chromium.perf:mac_retina_perf_cq Review-Url: https://codereview.chromium.org/2118293002 Cr-Commit-Position: refs/heads/master@{#405212} Commit : 5b1847df818e5be4592e9a6ac4832350bb663023 Date : Wed Jul 13 17:39:28 2016 ===== TESTED REVISIONS ===== Revision Mean Std Dev N Good? chromium@405124 75870475 595255 8 good chromium@405174 75543774 980324 5 good chromium@405199 75241075 884617 8 good chromium@405206 75687989 619337 8 good chromium@405209 75129571 741762 8 good chromium@405211 75282098 991955 8 good chromium@405212 77968494 1035722 5 bad <-- chromium@405224 77669875 683371 8 bad chromium@405324 79230702 839488 5 bad Bisect job ran on: android_s5_perf_bisect Bug ID: 628288 Test Command: src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --also-run-disabled-tests system_health.memory_mobile Test Metric: load_search-memory:chrome:all_processes:reported_by_chrome:malloc:effective_size_avg/load_search_yahoo Relative Change: 3.94% Score: 99.8 Buildbot stdio: http://build.chromium.org/p/tryserver.chromium.perf/builders/android_s5_perf_bisect/builds/800 Job details: https://chromeperf.appspot.com/buildbucket_job_status/9007060871539525584 Not what you expected? We'll investigate and get back to you! https://chromeperf.appspot.com/bad_bisect?try_job_id=4641051755675648 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Aug 30 2016
,
Aug 30 2016
|
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by petrcermak@chromium.org
, Jul 14 2016