Issue metadata
Sign in to add a comment
|
1.4%-16.1% regression in system_health.memory_mobile at 447578:447645 |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Feb 3 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988655071917839584
,
Feb 3 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_search/load_search_amazon Revision Result N chromium@447577 9376719 +- 3275937 21 good chromium@447645 9675191 +- 3842124 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.amazon system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988655071917839584 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5172304981524480 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 3 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988639296319866736
,
Feb 3 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988639290587133040
,
Feb 4 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:native_heap:proportional_resident_size_avg/load_news/load_news_hackernews Revision Result N chromium@447577 35187496 +- 699256 21 good chromium@447645 35200662 +- 939307 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.news.hackernews system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988639290587133040 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6111291871068160 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 4 2017
=== Auto-CCing suspected CL author asvitkine@chromium.org === Hi asvitkine@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : asvitkine Commit : a86e8e115054ad167458d0379980db5473cb20de Date : Wed Feb 01 19:33:59 2017 Subject: Don't set cpu architecture field on iOS in UMA logs. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_social/load_social_facebook Change : 1.31% | 8138678.85714 -> 8228205.71429 Revision Result N chromium@447577 8138679 +- 1698665 14 good chromium@447578 9652565 +- 448626 6 bad <-- chromium@447579 9496576 +- 83542.3 6 bad chromium@447580 9513643 +- 507215 6 bad chromium@447582 9734144 +- 288074 6 bad chromium@447586 9846443 +- 42828.8 6 bad chromium@447594 9645397 +- 151564 6 bad chromium@447612 9542315 +- 119327 6 bad chromium@447645 8228206 +- 2254966 14 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.social.facebook system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988639296319866736 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5031634266488832 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 4 2017
Bisect is wrong. My change was iOS-specific and could not have affected Android.
,
Feb 6 2017
Specifically, here's my change:
metrics::SystemProfileProto::Hardware* hardware =
system_profile->mutable_hardware();
+#if !defined(OS_IOS)
+ // On iOS, OperatingSystemArchitecture() returns values like iPad4,4 which is
+ // not the actual CPU architecture. Don't set it until the API is fixed. See
+ // crbug.com/370104 for details.
hardware->set_cpu_architecture(base::SysInfo::OperatingSystemArchitecture());
+#endif
hardware->set_system_ram_mb(base::SysInfo::AmountOfPhysicalMemoryMB());
So the above should not have resulted in any difference in compiled code on Android.
Re-assigning back to reporter.
,
Feb 6 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988383767213701856
,
Feb 6 2017
Thanks asvitkine! Sorry for the incorrect ping from Bisect. I've kicked off a couple more to see if we can get a correct bisect.
,
Feb 6 2017
=== Auto-CCing suspected CL author baxley@chromium.org === Hi baxley@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : baxley Commit : 3432563dba5d8feb7819262688a0741a66c6bf35 Date : Wed Feb 01 21:20:41 2017 Subject: Update web shell app to not load dev.chromium.org on launch. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_tools/load_tools_stackoverflow Change : 16.57% | 9255936.0 -> 7722666.66667 Revision Result N chromium@447577 9255936 +- 2518157 6 good chromium@447594 7925760 +- 194053 6 good chromium@447603 8212139 +- 271384 6 good chromium@447607 8515584 +- 220819 6 good chromium@447609 8769536 +- 307729 6 good chromium@447610 9131008 +- 190805 6 good chromium@447611 7748949 +- 66404.9 6 bad <-- chromium@447645 7722667 +- 72025.4 6 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.tools.stackoverflow system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988383767213701856 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5792117953134592 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 6 2017
My change was also iOS specific (part of web shell app and tests, not even iOS Chrome code), so I don't think this could affect Android. Assigning back to jasontiller@ - let me know if you have any questions
,
Feb 6 2017
Yikes! Sorry about that, y'all. CCing in speed infra (+sullivan) and the benchmark owner (+perezju) who might have some more insight into what's going on here.
,
Feb 6 2017
The bisects both look like they got bimodal results.
,
Feb 7 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8988327389205641216
,
Feb 7 2017
Kicked off a bisect on "memory:chrome:gpu_process:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_news/load_news_bbc" where the regression looks cleaner from: https://chromeperf.appspot.com/report?sid=79a4b6e4980e7300d110f1f024dc7902b52b4dfa9d2b7d6563f68276f14aac93&rev=447645
,
Feb 7 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : asvitkine Commit : a86e8e115054ad167458d0379980db5473cb20de Date : Wed Feb 01 19:33:59 2017 Subject: Don't set cpu architecture field on iOS in UMA logs. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:gpu_process:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_news/load_news_bbc Change : 3.14% | 1369429.33333 -> 1412437.33333 Revision Result N chromium@447577 1369429 +- 27272.5 6 good chromium@447578 1601024 +- 32855.9 6 bad <-- chromium@447579 1570304 +- 17058.1 6 bad chromium@447580 1557504 +- 52693.7 6 bad chromium@447582 1518421 +- 31342.3 6 bad chromium@447586 1492992 +- 66347.0 6 bad chromium@447594 1473536 +- 16190.9 6 bad chromium@447611 1447083 +- 16914.1 6 bad chromium@447645 1412437 +- 56546.1 6 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.news.bbc system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8988327389205641216 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5793065530294272 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Tests>AutoBisect. Thank you!
,
Feb 7 2017
+hjd, +primiano: hmm.. those results in #18 suspiciously look like the metric monotonically increasing in time. Hector, could you have a look?
,
Feb 7 2017
,
Feb 8 2017
So far: * We're not restarting the phone between runs in the bisect (unlike on the dashboard). * The metric is indeed monotonically increasing in time in the bisect. * Hard to say if this repos locally as the metric is quite noisy. * reported_by_chrome's java_heap metric is rock solid (down to the byte) (both locally and in the bisect). * However there is a weird movement of memory from allocated_objects to <unspecified> in java_heap as time goes on both locally and on the bisect.
,
Feb 8 2017
Seems like a false regression, all graphs have recovered: https://chromeperf.appspot.com/report?sid=79a4b6e4980e7300d110f1f024dc7902b52b4dfa9d2b7d6563f68276f14aac93&rev=447645 https://chromeperf.appspot.com/group_report?bug_id=688448 However we discovered two new bugs: crbug.com/690040 java_heap:allocated_objects_size_avg is super noisy crbug.com/690042 bisect 'leaks' java_heap:allocated_objects_size_avg over many runs
,
Mar 11 2017
,
Apr 11 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8982651911306650768 |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by jasontiller@chromium.org
, Feb 3 2017