New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 713092 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Last visit > 30 days ago
Closed: May 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Confirm whether Webview perf bots remove system Chrome before running

Project Member Reported by alexclarke@chromium.org, Apr 19 2017

Issue description

See the link to graphs below.
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=713092

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDggr-nqgkM,agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgguW74QoM


Bot(s) for this bug's original alert(s):

android-nexus5X
android-webview-nexus5X
Project Member

Comment 3 by 42576172...@developer.gserviceaccount.com, Apr 19 2017

Cc: cjgrant@chromium.org
Owner: cjgrant@chromium.org

=== Auto-CCing suspected CL author cjgrant@chromium.org ===

Hi cjgrant@chromium.org, the bisect results pointed to your CL, please take a look at the
results.


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : cjgrant
  Commit : b64525a610068d10330a05bc076791ba22f60c5e
  Date   : Thu Apr 13 15:29:52 2017
  Subject: VR: Destroy scene and scene manager on the thread they were created.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_games/load_games_lazors
  Change       : 3.55% | 8940763.42857 -> 9131154.28571

Revision             Result                  N
chromium@464411      8940763 +- 913810       14      good
chromium@464413      9112917 +- 88677.1      6       good
chromium@464414      9247061 +- 415140       6       good
chromium@464415      11444565 +- 206405      6       bad       <--
chromium@464418      11112789 +- 137861      6       bad
chromium@464425      10724693 +- 268802      6       bad
chromium@464439      10266965 +- 440817      6       bad
chromium@464467      9698304 +- 332735       6       bad
chromium@464526      9131154 +- 1307319      14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981888412687730688

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6430651667972096


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Status: WontFix (was: Untriaged)
The CL flagged by this bisect can't possibly be the culprit.  The CL affects only Chrome in VR mode, which isn't running in these tests.  Also, we moved some destroy calls to fix a crash, and the memory now being freed correctly is tiny.

I'll close this for now, but feel free to ping if there are concerns.
Cc: perezju@chromium.org simonhatch@chromium.org
Owner: ----
Status: Available (was: WontFix)
Please don't close the perf bug if you think your CL is not the culprit; you can unassign instead. I'm going to kick off another bisect on the ebay link.

Simon, Juan: It's a little strange how the numbers seem to go up in the middle and down at the start and end of the bisect in #3. Any ideas?
Project Member

Comment 7 by 42576172...@developer.gserviceaccount.com, Apr 21 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_search/load_search_ebay


Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.ebay system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981876377043971120

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 9 by 42576172...@developer.gserviceaccount.com, Apr 23 2017


=== BISECT JOB RESULTS ===
NO Perf regression found, tests failed to produce values

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_chrome:java_heap:effective_size_avg/load_search/load_search_ebay


Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.search.ebay system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981697328342919664

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 11 by 42576172...@developer.gserviceaccount.com, Apr 24 2017


=== BISECT JOB RESULTS ===
Perf regression found with culprit

Suspected Commit
  Author : cjgrant
  Commit : b64525a610068d10330a05bc076791ba22f60c5e
  Date   : Thu Apr 13 15:29:52 2017
  Subject: VR: Destroy scene and scene manager on the thread they were created.

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:all_processes:reported_by_os:system_memory:java_heap:proportional_resident_size_avg/load_games/load_games_lazors
  Change       : 3.63% | 8964973.71429 -> 9138761.14286

Revision             Result                  N
chromium@464411      8964974 +- 992263       14      good
chromium@464413      9156949 +- 192614       6       good
chromium@464414      9240917 +- 426404       6       good
chromium@464415      11424427 +- 236478      6       bad       <--
chromium@464418      11148971 +- 224882      6       bad
chromium@464425      10743808 +- 269337      6       bad
chromium@464439      10236587 +- 421146      6       bad
chromium@464467      9721515 +- 244334       6       bad
chromium@464526      9138761 +- 1427608      14      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981424907172570800

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=4973702661275648


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Hmm wonder if there was another regression that got reverted in that range. That's 2 bisects that gave similar hump-in-the-middle results.
Cc: mikec...@chromium.org
Sorry, the "regression" on the webview bot is from a time when those bots were (incorrectly) reporting metrics for both memory:chrome:* and memory:webview:*. The later are metrics from the test loading pages; the former were probably coming from some stray chrome process running on the background. [I've marked this regresison as ingored.]

This was fixed on the tester bots (completely removing chrome from devices), +mikecase to confirm whether the fix was also applied to bisect bots (namely: android_webview_arm64_aosp_perf_bisect).

The other regression is tricky, see:
https://chromeperf.appspot.com/report?sid=2fb18691377dd0a1c671d735bfb875cbed9e332ff93224a34962d5d1e9699c05&start_rev=459677&end_rev=465732
- There is a clear large jump on java_heap in the range 464527-464652 (B), tracked at  issue 712370 
- This alert however was triggered on the range 464412-464526 (A), exactly one build earlier.
- Browser and renderer processes do not show any regression in range (B); but gpu process does show some small bump in (A).

Trying to kick-off a bisect there now. To reiterate, I'm bisecting:
https://chromeperf.appspot.com/report?sid=8a073f87ead9edf74c0bce4ea33bac3c700b1adf8b6485c600ce10527c8ee066&start_rev=459677&end_rev=465732

On the range: 464412-464526
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, May 15 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:gpu_process:reported_by_chrome:java_heap:effective_size_avg/load_games/load_games_lazors

Revision             Result               N
chromium@464411      20375546 +- 0.0      21      good
chromium@464526      20375546 +- 0.0      21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8979528653512909312

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5807681576632320


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Owner: mikec...@chromium.org
Status: Assigned (was: Available)
Summary: Confirm whether Webview perf bots remove system Chrome before running (was: 2%-5.2% regression in system_health.memory_mobile at 464412:464641)
I'll take bisect in #15 as "there is no real regression" here, and follow up on  issue 712370  on the larger regression.

So the only thing remaining here is about Webview bisects possibly running some Chrome stray processes.
Project Member

Comment 18 by 42576172...@developer.gserviceaccount.com, May 15 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_nexus5X_perf_bisect
  Benchmark    : system_health.memory_mobile
  Metric       : memory:chrome:gpu_process:reported_by_chrome:java_heap:effective_size_avg/load_games/load_games_lazors

Revision             Result               N
chromium@464411      20375546 +- 0.0      21      good
chromium@464526      20375546 +- 0.0      21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-chromium --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=load.games.lazors system_health.memory_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8979522184440444128

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5807681576632320


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
re Comment 15: The WebView perf bisect bots are uninstalling the system Chrome package (and since they have AOSP builds, I don't think they ever had system Chrome)

You can see here...
https://uberchromegw.corp.google.com/i/tryserver.chromium.perf/builders/android_webview_arm64_aosp_perf_bisect/builds/1188

They run provision_devices with following arg...
--remove-system-packages com.android.chrome com.google.android.webview com.android.webview org.chromium.webview_shell

Is that all of the info you need?
Status: WontFix (was: Assigned)
Yep, sounds good. Thanks for double-checking!

Sign in to add a comment