New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 714351 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 712677



Sign in to add a comment

Benchmark runtime regressions due to WebView N5X not loading

Project Member Reported by nedngu...@google.com, Apr 22 2017

Issue description

The benchmark's cycle time blows up from 33 minutes to 209 minutes because every single test fails due to TimedOut while loading :-(

(https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus5X_WebView_Perf__2_%2F4966%2F%2B%2Frecipes%2Fsteps%2Fmemory.top_10_mobile%2F0%2Fstdout)
 
All graphs for this bug:
  https://chromeperf.appspot.com/group_report?bug_id=714351

Original alerts at time of bug-filing:
  https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgovKvnwoM


Bot(s) for this bug's original alert(s):

android-webview-nexus5X
Components: -Speed>Telemetry Speed>Benchmarks>Waterfall
Cycle time is blowing up in many webview benchmarks 
Cc: mikec...@chromium.org
All of the WebView devices on the waterfall need to be flashed to AOSP builds of Android. They were flashed to non-AOSP builds a few weeks back. This went unnoticed until recently due to another issue.

The non-AOSP builds of Android try to load webview from com.google.android.webview package. This is not ever installed since the open-source webview builds use com.android.webview. AOSP builds however, load webview from com.android.webview.
Blockedon: 712677
Note that the bump in cicle time is also showing up in non webview bots:

https://chromeperf.appspot.com/report?sid=abd7a90dedd5971b57513fe318fb1b7a43fdad2a7b00e2ee7197eb21c9c3da27&rev=465047
Ok, yes, most of the bumps on

https://chromeperf.appspot.com/group_report?bug_id=714351

are due to the new EM stories (on r465012).

I'll try to split off regressions because of that CL and others (like the webview bot) which have different causes.
Summary: Benchmark runtime regressions due to WebView N5X not loading (was: 527.4% regression in memory.top_10_mobile at 465212:465353)
Ok, managed to split off all regressions, gladly nothing new. We had

- Regressions on WebView N5X due to webview not loading, this issue.
- Regressions due to new EM stories,  issue 714635 , WontFix.
- An earlier 18 min regression when battors were added to bot,  issue 714643 .
- An even earlier but smaller (1-3 min) regression that was already filed at issue 709603.
Project Member

Comment 13 by 42576172...@developer.gserviceaccount.com, Apr 25 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : memory.top_10_mobile
  Metric       : benchmark_duration/benchmark_duration

Revision             Result                   N
chromium@465211      5.59962 +- 0.232659      21      good
chromium@465353      5.63866 +- 0.42248       21      bad

Please refer to the following doc on diagnosing memory regressions:
  https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/memory_benchmarks.md

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests memory.top_10_mobile

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981469263580522416

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5283757289897984


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 15 by 42576172...@developer.gserviceaccount.com, Apr 28 2017


=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: android_webview_arm64_aosp_perf_bisect
  Benchmark    : thread_times.key_idle_power_cases
  Metric       : tasks_per_second_total_all/set-timeout.html

Revision             Result                   N
chromium@465211      5.59962 +- 0.232659      21      good
chromium@465353      5.63866 +- 0.42248       21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=android-webview --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=set.timeout.html thread_times.key_idle_power_cases

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8981277852127314784

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=5283757289897984


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Project Member

Comment 16 by sheriffbot@chromium.org, Jul 19 2017

Labels: Hotlist-Google
Status: Archived (was: Assigned)
Archiving some more old bugs on which probably I won't be working.

Sign in to add a comment