New issue
Advanced search Search tips

Issue 921669 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug

Blocking:
issue 907899



Sign in to add a comment

Don't fail when unnmatched tests are passed via a filter to android instrumentation test runner

Project Member Reported by bpastene@chromium.org, Jan 14

Issue description

The base test launcher exits quietly (ie: w/ a zero exit code) when a filter is passed it that doesn't match any test (eg: `--gtest_filter=slkdfjsdlkfjsdlkfjasdklfj`). This is not true for android instrumentation tests. eg:

$ ./bin/run_webview_instrumentation_test_apk --gtest_filter=slkjfdslkjfsdljkdfsljk
E   16.899s Main  Unrecognized error occurred.
Traceback (most recent call last):
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/test_runner.py", line 1037, in main
    return RunTestsCommand(args)
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/test_runner.py", line 723, in RunTestsCommand
    return RunTestsInPlatformMode(args)
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/test_runner.py", line 869, in RunTestsInPlatformMode
    test_run.RunTests(raw_results)
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/pylib/local/device/local_device_test_run.py", line 58, in RunTests
    tests = self._GetTests()
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/pylib/local/device/local_device_instrumentation_test_run.py", line 380, in _GetTests
    tests = self._test_instance.ProcessRawTests(raw_tests)
  File "/usr/local/google/home/bpastene/ssd/chromium_src/src/build/android/pylib/instrumentation/instrumentation_test_instance.py", line 865, in ProcessRawTests
    raise UnmatchedFilterException(self._test_filter)
UnmatchedFilterException: Test filter "slkjfdslkjfsdljkdfsljk" matched no tests.
$ echo $?
1

Compare that with an android gtest:
$ ./bin/run_storage_unittests --gtest_filter=slkjfdslkjfsdljkdfsljk
$ echo $?
0


Unless there's a good reason for instrumentation tests to fail in this case, we should conform to the base test launcher's behavior. This will also help the case when we try to use the same filter file for different test runs (eg bug 907899).
 
hmm, now that I think about it, this deviation may have been to fail in cases where a dev typos a test in a filter?
Appears to be: bug 662018

Though the implemented fix is a bit more aggressive than the initial bug's suggestion ("failing the test" vs "print something useful"). I think a logging.warning() would be preferred here.

Sign in to add a comment