Unify test_runner.py CLI |
|||||
Issue descriptionThe test runner has slightly different ways of handling command-line options for different test types, and in some cases those also deviate from //base/test/launcher. We should unify them to match //base/test/launcher s.t. external tools etc can run various test types with minimal knowledge of the test runner's implementation details. This should initially include: --gtest_filter --gtest_repeat --gtest_also_run_disabled_tests --test-launcher-retry-limit for gtests and instrumentation tests run through the test runner.
,
Nov 29 2016
,
Nov 29 2016
shenghuazhang@, when this bug is fixed, please kindly send a ping on the blocking bug so that we could move forward on our end. Thanks!
,
Dec 2 2016
,
Dec 2 2016
Started working on it. Landed a CL https://codereview.chromium.org/2544603002/ implementing flag '--gtest_also_run_disabled_tests' logic for gtest runner. Keep fixing the others currently.
,
Dec 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7cd65ac511114dc5ee3ad8d392712ef66a76fdde commit 7cd65ac511114dc5ee3ad8d392712ef66a76fdde Author: shenghuazhang <shenghuazhang@chromium.org> Date: Fri Dec 02 03:30:06 2016 [Android] Add '--gtest_also_run_disabled_tests' for instrumentation tests Add --gtest_also_run_disabled_tests in AddJavaTestOptions for test_runner.py. Add the run_disabled logic in instrumentation_test_instance.py. BUG= 669632 Review-Url: https://codereview.chromium.org/2544533006 Cr-Commit-Position: refs/heads/master@{#435842} [modify] https://crrev.com/7cd65ac511114dc5ee3ad8d392712ef66a76fdde/build/android/pylib/instrumentation/instrumentation_test_instance.py [modify] https://crrev.com/7cd65ac511114dc5ee3ad8d392712ef66a76fdde/build/android/test_runner.py
,
Dec 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/acc5de09b5bb842ff69043b6f2b078b94ae472f7 commit acc5de09b5bb842ff69043b6f2b078b94ae472f7 Author: shenghuazhang <shenghuazhang@chromium.org> Date: Fri Dec 02 21:24:36 2016 [Android] Add name '--test-launcher-retry-limit' to test_runner.py To unify retry-limit option with //base/test/launcher, add '--test-launcher-retry-limit' and '--test_launcher_retry_limit' to common option '--num_retries'. BUG= 669632 Review-Url: https://codereview.chromium.org/2544983002 Cr-Commit-Position: refs/heads/master@{#436028} [modify] https://crrev.com/acc5de09b5bb842ff69043b6f2b078b94ae472f7/build/android/test_runner.py
,
Dec 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/61757ce0ebb1aef762564a649b2266d846f24adf commit 61757ce0ebb1aef762564a649b2266d846f24adf Author: shenghuazhang <shenghuazhang@chromium.org> Date: Thu Dec 08 22:53:01 2016 [Android] Support gtest_filter + gtest_also_run_disabled_tests Support gtest_filter + gtest_also_run_disabled_tests in the same way that //base/test/launcher does. See https://bugs.chromium.org/p/chromium/issues/detail?id=658892 for details. BUG= 669632 Review-Url: https://codereview.chromium.org/2555633002 Cr-Commit-Position: refs/heads/master@{#437383} [modify] https://crrev.com/61757ce0ebb1aef762564a649b2266d846f24adf/build/android/pylib/gtest/gtest_test_instance.py [modify] https://crrev.com/61757ce0ebb1aef762564a649b2266d846f24adf/build/android/pylib/gtest/gtest_test_instance_test.py
,
Dec 8 2016
,
Dec 9 2016
Many thanks for fixing this bug! To double check, after all these changes, what test types on Android will support the four flags in the description above? I assume that gtest and instrumentation are supported now, but telemetry-based tests are not yet.
,
Dec 9 2016
@stgao Yes, this fix will support gtest and instrumentation but not for telemetry-based tests. Should the unified command-line options supporting include telemetry-based tests?
,
Dec 9 2016
@shenghuazhang, thanks for confirm. No, we could do that later, because telemetry hasn't support repeating yet.
,
Dec 9 2016
@stgao Sure. Please let us know when the telemetry-based tests are ready to be supported. |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by jbudorick@chromium.org
, Nov 29 2016