New issue
Advanced search Search tips

Issue 763974 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Dec 10
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

Add CQ coverage for telemetry_perf_unittests on Android WebView

Project Member Reported by xunji...@chromium.org, Sep 11 2017

Issue description

We have CQ coverage for telemetry_perf_unittests for Android, Mac, Linux and Windows. Can we add CQ coverage for telemetry_perf_unittests for Android Webview?

Context: I landed a flag change in Telemetry (https://chromiumcodereview.appspot.com/3003143002/). The CL broke WebView perf tests ( Issue 753948 ). It was not caught in catapult's CQ or Chromium CQ.



 
Cc: torne@chromium.org bokan@chromium.org
Cc: -bokan@chromium.org boliu@chromium.org
I tried doing " tools/perf/run_tests telemetry_perf_unittests --browser=android-webview"

But I got:

Running tests with 1 Android device(s).
0 tests passed in 66.0s, 0 skipped, 0 failures.


Looks like telemetry_perf_unittests target is somehow skipped/disabled on WebView?
Helen: for android-webview to work, you also need to build the embedder app & specify it with "--webview-embedder-apk"
Blocking: -753948
I tried:
tools/perf/run_tests telemetry_perf_webview_tests --browser=android-webview --webview-embedder-apk=out/Release/apks/SystemWebViewShell.apk

and
tools/perf/run_tests telemetry_perf_unittests --browser=android-webview --webview-embedder-apk=out/net_android/apks/SystemWebViewShell.apk

Both gave me:
Running tests with 1 Android device(s).
0 tests passed in 4.9s, 0 skipped, 0 failures.


tools/perf/run_benchmark works for me, so I am no longer blocked on this.
The command that I used is: 
tools/perf/run_benchmark -v --browser=android-webview --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...m.facebook.com.barackobama power.typical_10_mobile --webview-embedder-apk=out/Release/apks/SystemWebViewShell.apk


Hey, I was having a look at this. Which bot is running telemetry_perf_unittests on Android?

I couldn't find them here e.g.:
https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/
(that bot is android_n5x_swarming_rel's counterpart, so they still run on the CQ)
I think we should try to reenable t_p_u on LARNG. Is that config swarmed yet?
it's been swarmed for a while (and was swarmed when you disabled it)
Re #5: Ok, the actual command should be:

$ tools/perf/run_tests --browser android-webview --device android

It's working now on a test device on my desk (although some tests appear to be failing).
Components: Infra>Client>Chrome
Moving Infra>Client>Android -> Infra>Client>Chrome+OS=Android
Components: -Infra>Client>Android
Status: Available (was: Untriaged)
Some bot names have changed, so posting the new ones for reference:
linux_android_rel_ng's mirror: https://ci.chromium.org/buildbot/chromium.android/KitKat%20Phone%20Tester%20%28rel%29/
android_n5x_swarming_rel's: mirror: https://ci.chromium.org/buildbot/chromium.android/Marshmallow%20Phone%20Tester%20%28rel%29/

Both on the CQ. Both running telemetry_perf_unittests.

What else needs to be done here? Note that we're not running any webview tests on linux_android_rel_ng, so I doubt we'd need to add a webview version of t_p_u on there as well.
Project Member

Comment 15 by sheriffbot@chromium.org, Dec 7

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Status: Archived (was: Untriaged)
No response to #14... closing this.
Status: Untriaged (was: Archived)
To #14: I think what needs to be done here is to find a builder that we can add t_p_u, or some simple smoke tests against webview. It would have helped caught  issue 906678 
Components: Tests>Telemetry
Status: WontFix (was: Untriaged)
Not every breakage deserves CQ coverage, and we've stepped back from the effort to add webview coverage to the cq ( bug 589168 )

Given how flaky and demanding telemetry_perf_unittests is, and given how little spare android capacity we have, this seems like a wontfix.

Sign in to add a comment