New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 669632 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug

Blocking:
issue 669648



Sign in to add a comment

Unify test_runner.py CLI

Project Member Reported by jbudorick@chromium.org, Nov 29 2016

Issue description

The test runner has slightly different ways of handling command-line options for different test types, and in some cases those also deviate from //base/test/launcher. We should unify them to match //base/test/launcher s.t. external tools etc can run various test types with minimal knowledge of the test runner's implementation details.

This should initially include:
  --gtest_filter
  --gtest_repeat
  --gtest_also_run_disabled_tests
  --test-launcher-retry-limit

for gtests and instrumentation tests run through the test runner.
 
This should also include supporting gtest_filter + gtest_also_run_disabled_tests in the same way that //base/test/launcher does. See https://bugs.chromium.org/p/chromium/issues/detail?id=658892 for details.

Comment 2 by st...@chromium.org, Nov 29 2016

Blocking: 669648

Comment 3 by st...@chromium.org, Nov 29 2016

shenghuazhang@, when this bug is fixed, please kindly send a ping on the blocking bug so that we could move forward on our end. Thanks!
Status: Started (was: Assigned)
Started working on it. Landed a CL https://codereview.chromium.org/2544603002/  implementing flag '--gtest_also_run_disabled_tests' logic for gtest runner.
Keep fixing the others currently.
Project Member

Comment 6 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7cd65ac511114dc5ee3ad8d392712ef66a76fdde

commit 7cd65ac511114dc5ee3ad8d392712ef66a76fdde
Author: shenghuazhang <shenghuazhang@chromium.org>
Date: Fri Dec 02 03:30:06 2016

[Android] Add '--gtest_also_run_disabled_tests' for instrumentation tests

Add --gtest_also_run_disabled_tests in AddJavaTestOptions for test_runner.py. Add the run_disabled logic in instrumentation_test_instance.py.

BUG= 669632 

Review-Url: https://codereview.chromium.org/2544533006
Cr-Commit-Position: refs/heads/master@{#435842}

[modify] https://crrev.com/7cd65ac511114dc5ee3ad8d392712ef66a76fdde/build/android/pylib/instrumentation/instrumentation_test_instance.py
[modify] https://crrev.com/7cd65ac511114dc5ee3ad8d392712ef66a76fdde/build/android/test_runner.py

Project Member

Comment 7 by bugdroid1@chromium.org, Dec 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/acc5de09b5bb842ff69043b6f2b078b94ae472f7

commit acc5de09b5bb842ff69043b6f2b078b94ae472f7
Author: shenghuazhang <shenghuazhang@chromium.org>
Date: Fri Dec 02 21:24:36 2016

[Android] Add name '--test-launcher-retry-limit' to test_runner.py

To unify retry-limit option with //base/test/launcher, add
'--test-launcher-retry-limit' and '--test_launcher_retry_limit' to
common option '--num_retries'.

BUG= 669632 

Review-Url: https://codereview.chromium.org/2544983002
Cr-Commit-Position: refs/heads/master@{#436028}

[modify] https://crrev.com/acc5de09b5bb842ff69043b6f2b078b94ae472f7/build/android/test_runner.py

Project Member

Comment 8 by bugdroid1@chromium.org, Dec 8 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/61757ce0ebb1aef762564a649b2266d846f24adf

commit 61757ce0ebb1aef762564a649b2266d846f24adf
Author: shenghuazhang <shenghuazhang@chromium.org>
Date: Thu Dec 08 22:53:01 2016

[Android] Support gtest_filter + gtest_also_run_disabled_tests

Support gtest_filter + gtest_also_run_disabled_tests in the same way
that //base/test/launcher does.
See https://bugs.chromium.org/p/chromium/issues/detail?id=658892 for
details.

BUG= 669632 

Review-Url: https://codereview.chromium.org/2555633002
Cr-Commit-Position: refs/heads/master@{#437383}

[modify] https://crrev.com/61757ce0ebb1aef762564a649b2266d846f24adf/build/android/pylib/gtest/gtest_test_instance.py
[modify] https://crrev.com/61757ce0ebb1aef762564a649b2266d846f24adf/build/android/pylib/gtest/gtest_test_instance_test.py

Status: Fixed (was: Started)
Cc: chanli@chromium.org
Many thanks for fixing this bug!

To double check, after all these changes, what test types on Android will support the four flags in the description above?
I assume that gtest and instrumentation are supported now, but telemetry-based tests are not yet.
@stgao Yes, this fix will support gtest and instrumentation but not for telemetry-based tests.
Should the unified command-line options supporting include telemetry-based tests?
@shenghuazhang, thanks for confirm.
No, we could do that later, because telemetry hasn't support repeating yet.
@stgao Sure. Please let us know when the telemetry-based tests are ready to be supported.

Sign in to add a comment