Steps to repro:

$ android_webview/tools/run_cts.py -f="android.webkit.cts.WebViewSslTest#testSecureServerRequiringClientCertDoesCancelRequest" --apk-dir="$HOME/run-cts-apk-dir" --verbose

Expected results:

It only runs 1 test.

Actual results:

It runs every test... If you look at the verbose output (thanks for catching this, arthursonzogni@!) it shows:

I    0.000s Main  command: android_webview/tools/../../build/android/test_runner.py instrumentation -f=android.webkit.cts.WebViewSslTest#testSecureServerRequiringClientCertDoesCancelRequest --verbose -d 005d1ac915b0c7d6 --test-apk /usr/local/google/home/ntfschr/run-cts-apk-dir/arm64/M/4607285/android-cts/repository/testcases/CtsWebkitTestCases.apk -f=-android.webkit.cts.WebViewClientTest#testDoUpdateVisitedHistory:android.webkit.cts.WebViewClientTest#testLoadPage:android.webkit.cts.WebViewClientTest#testOnFormResubmission:android.webkit.cts.WebViewClientTest#testOnReceivedError:android.webkit.cts.WebViewClientTest#testOnReceivedErrorForSubresource:android.webkit.cts.WebViewClientTest#testOnReceivedHttpAuthRequest:android.webkit.cts.WebViewClientTest#testOnReceivedHttpError:android.webkit.cts.WebViewClientTest#testOnReceivedLoginRequest:android.webkit.cts.WebViewClientTest#testOnScaleChanged:android.webkit.cts.WebViewClientTest#testShouldOverrideUrlLoading:android.webkit.cts.WebViewClientTest#testShouldOverrideUrlLoadingOnCreateWindow
I    0.012s Main  condition '<lambda>' met
W    0.012s Main  Test jar not specified. Test runner will not have Java annotation info available. May not handle test timeouts correctly.
...

^ This clearly puts a "-f=" at the beginning for the desired test, but then puts another "-f" at the end (which probably breaks things). I'm running with platform == M, and this list suspiciously looks similar to our "expected failures" list [1], even though I never requested the script to skip expected failures...

Can we fix this? Running a single test is very important to debugging issues like this.

[1] https://cs.chromium.org/chromium/src/android_webview/tools/cts_config/webview_cts_gcs_path.json?l=143&rcl=19a0c1ed924f2a632c6255fe311d6819568c031f