Issue metadata
Sign in to add a comment
|
Confirm whether Webview perf bots remove system Chrome before running |
||||||||||||||||||||
Issue descriptionSee the link to graphs below.
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981888412687730688
,
Apr 19 2017
=== Auto-CCing suspected CL author cjgrant@chromium.org === Hi cjgrant@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : cjgrant Commit : b64525a610068d10330a05bc076791ba22f60c5e Date : Thu Apr 13 15:29:52 2017 Subject: VR: Destroy scene and scene manager on the thread they were created. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_games/load_games_lazors Change : 3.55% | 8940763.42857 -> 9131154.28571 Revision Result N chromium@464411 8940763 +- 913810 14 good chromium@464413 9112917 +- 88677.1 6 good chromium@464414 9247061 +- 415140 6 good chromium@464415 11444565 +- 206405 6 bad <-- chromium@464418 11112789 +- 137861 6 bad chromium@464425 10724693 +- 268802 6 bad chromium@464439 10266965 +- 440817 6 bad chromium@464467 9698304 +- 332735 6 bad chromium@464526 9131154 +- 1307319 14 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981888412687730688 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6430651667972096 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 19 2017
The CL flagged by this bisect can't possibly be the culprit. The CL affects only Chrome in VR mode, which isn't running in these tests. Also, we moved some destroy calls to fix a crash, and the memory now being freed correctly is tiny. I'll close this for now, but feel free to ping if there are concerns.
,
Apr 19 2017
Please don't close the perf bug if you think your CL is not the culprit; you can unassign instead. I'm going to kick off another bisect on the ebay link. Simon, Juan: It's a little strange how the numbers seem to go up in the middle and down at the start and end of the bisect in #3. Any ideas?
,
Apr 19 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981876377043971120
,
Apr 21 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: android_webview_arm64_aosp_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_search/load_search_ebay Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.ebay system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981876377043971120 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 21 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981697328342919664
,
Apr 23 2017
=== BISECT JOB RESULTS === NO Perf regression found, tests failed to produce values Bisect Details Configuration: android_webview_arm64_aosp_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_search/load_search_ebay Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.ebay system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981697328342919664 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 24 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8981424907172570800
,
Apr 24 2017
=== BISECT JOB RESULTS === Perf regression found with culprit Suspected Commit Author : cjgrant Commit : b64525a610068d10330a05bc076791ba22f60c5e Date : Thu Apr 13 15:29:52 2017 Subject: VR: Destroy scene and scene manager on the thread they were created. Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_games/load_games_lazors Change : 3.63% | 8964973.71429 -> 9138761.14286 Revision Result N chromium@464411 8964974 +- 992263 14 good chromium@464413 9156949 +- 192614 6 good chromium@464414 9240917 +- 426404 6 good chromium@464415 11424427 +- 236478 6 bad <-- chromium@464418 11148971 +- 224882 6 bad chromium@464425 10743808 +- 269337 6 bad chromium@464439 10236587 +- 421146 6 bad chromium@464467 9721515 +- 244334 6 bad chromium@464526 9138761 +- 1427608 14 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8981424907172570800 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Apr 24 2017
Hmm wonder if there was another regression that got reverted in that range. That's 2 bisects that gave similar hump-in-the-middle results.
,
May 15 2017
Sorry, the "regression" on the webview bot is from a time when those bots were (incorrectly) reporting metrics for both memory:chrome:* and memory:webview:*. The later are metrics from the test loading pages; the former were probably coming from some stray chrome process running on the background. [I've marked this regresison as ingored.] This was fixed on the tester bots (completely removing chrome from devices), +mikecase to confirm whether the fix was also applied to bisect bots (namely: android_webview_arm64_aosp_perf_bisect). The other regression is tricky, see: https://chromeperf.appspot.com/report?sid=2fb18691377dd0a1c671d735bfb875cbed9e332ff93224a34962d5d1e9699c05&start_rev=459677&end_rev=465732 - There is a clear large jump on java_heap in the range 464527-464652 (B), tracked at issue 712370 - This alert however was triggered on the range 464412-464526 (A), exactly one build earlier. - Browser and renderer processes do not show any regression in range (B); but gpu process does show some small bump in (A). Trying to kick-off a bisect there now. To reiterate, I'm bisecting: https://chromeperf.appspot.com/report?sid=8a073f87ead9edf74c0bce4ea33bac3c700b1adf8b6485c600ce10527c8ee066&start_rev=459677&end_rev=465732 On the range: 464412-464526
,
May 15 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8979528653512909312
,
May 15 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:gpu_process:reported_by_chrome:java_heap:effective_size_avg/load_games/load_games_lazors Revision Result N chromium@464411 20375546 +- 0.0 21 good chromium@464526 20375546 +- 0.0 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8979528653512909312 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5807681576632320 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
May 15 2017
I'll take bisect in #15 as "there is no real regression" here, and follow up on issue 712370 on the larger regression. So the only thing remaining here is about Webview bisects possibly running some Chrome stray processes.
,
May 15 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8979522184440444128
,
May 15 2017
=== BISECT JOB RESULTS === NO Perf regression found Bisect Details Configuration: android_nexus5X_perf_bisect Benchmark : system_health.memory_mobile Metric : memory:chrome:gpu_process:reported_by_chrome:java_heap:effective_size_avg/load_games/load_games_lazors Revision Result N chromium@464411 20375546 +- 0.0 21 good chromium@464526 20375546 +- 0.0 21 bad Please refer to the following doc on diagnosing memory regressions: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md To Run This Test src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8979522184440444128 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5807681576632320 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
May 15 2017
re Comment 15: The WebView perf bisect bots are uninstalling the system Chrome package (and since they have AOSP builds, I don't think they ever had system Chrome) You can see here... https://uberchromegw.corp.google.com/i/tryserver.chromium.perf/builders/android_webview_arm64_aosp_perf_bisect/builds/1188 They run provision_devices with following arg... --remove-system-packages com.android.chrome com.google.android.webview com.android.webview org.chromium.webview_shell Is that all of the info you need?
,
May 16 2017
Yep, sounds good. Thanks for double-checking! |
|||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||
Comment 1 by alexclarke@chromium.org
, Apr 19 2017