Add CQ coverage for telemetry_perf_unittests on Android WebView |
|||||||||||
Issue descriptionWe have CQ coverage for telemetry_perf_unittests for Android, Mac, Linux and Windows. Can we add CQ coverage for telemetry_perf_unittests for Android Webview? Context: I landed a flag change in Telemetry (https://chromiumcodereview.appspot.com/3003143002/). The CL broke WebView perf tests ( Issue 753948 ). It was not caught in catapult's CQ or Chromium CQ.
,
Sep 11 2017
,
Sep 14 2017
I tried doing " tools/perf/run_tests telemetry_perf_unittests --browser=android-webview" But I got: Running tests with 1 Android device(s). 0 tests passed in 66.0s, 0 skipped, 0 failures. Looks like telemetry_perf_unittests target is somehow skipped/disabled on WebView?
,
Sep 14 2017
Helen: for android-webview to work, you also need to build the embedder app & specify it with "--webview-embedder-apk"
,
Sep 15 2017
I tried: tools/perf/run_tests telemetry_perf_webview_tests --browser=android-webview --webview-embedder-apk=out/Release/apks/SystemWebViewShell.apk and tools/perf/run_tests telemetry_perf_unittests --browser=android-webview --webview-embedder-apk=out/net_android/apks/SystemWebViewShell.apk Both gave me: Running tests with 1 Android device(s). 0 tests passed in 4.9s, 0 skipped, 0 failures. tools/perf/run_benchmark works for me, so I am no longer blocked on this. The command that I used is: tools/perf/run_benchmark -v --browser=android-webview --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...m.facebook.com.barackobama power.typical_10_mobile --webview-embedder-apk=out/Release/apks/SystemWebViewShell.apk
,
Sep 26 2017
Hey, I was having a look at this. Which bot is running telemetry_perf_unittests on Android? I couldn't find them here e.g.: https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/
,
Sep 26 2017
https://build.chromium.org/p/chromium.android/builders/Android%20N5X%20Swarm%20Builder Ned dropped them off of Android Tests & LARNG a few months back in https://chromium.googlesource.com/chromium/src/+/b51378065a1d1ede626abeab2122e95191f49db4
,
Sep 26 2017
(that bot is android_n5x_swarming_rel's counterpart, so they still run on the CQ)
,
Sep 26 2017
I think we should try to reenable t_p_u on LARNG. Is that config swarmed yet?
,
Sep 26 2017
it's been swarmed for a while (and was swarmed when you disabled it)
,
Sep 26 2017
Re #5: Ok, the actual command should be: $ tools/perf/run_tests --browser android-webview --device android It's working now on a test device on my desk (although some tests appear to be failing).
,
Dec 4 2017
Moving Infra>Client>Android -> Infra>Client>Chrome+OS=Android
,
Dec 4 2017
,
Dec 7 2017
Some bot names have changed, so posting the new ones for reference: linux_android_rel_ng's mirror: https://ci.chromium.org/buildbot/chromium.android/KitKat%20Phone%20Tester%20%28rel%29/ android_n5x_swarming_rel's: mirror: https://ci.chromium.org/buildbot/chromium.android/Marshmallow%20Phone%20Tester%20%28rel%29/ Both on the CQ. Both running telemetry_perf_unittests. What else needs to be done here? Note that we're not running any webview tests on linux_android_rel_ng, so I doubt we'd need to add a webview version of t_p_u on there as well.
,
Dec 7
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 7
No response to #14... closing this.
,
Dec 7
To #14: I think what needs to be done here is to find a builder that we can add t_p_u, or some simple smoke tests against webview. It would have helped caught issue 906678
,
Dec 7
,
Dec 10
Not every breakage deserves CQ coverage, and we've stepped back from the effort to add webview coverage to the cq ( bug 589168 ) Given how flaky and demanding telemetry_perf_unittests is, and given how little spare android capacity we have, this seems like a wontfix. |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by nedngu...@google.com
, Sep 11 2017