New issue
Advanced search Search tips

Issue 674260 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug



Sign in to add a comment

Tests can set command-line flags but can't clear them when set by the testing configuration

Project Member Reported by donnd@chromium.org, Dec 14 2016

Issue description

When a flag is set in fieldtrial_testing_config.json, there appears to be no way to clear it using the @CommandLineFlags annotation in a test.  This makes it very difficult to have tests for both true and false values when the config is set to true.  When the config is set to false, it is possible to override for an individual test.

For Contextual Search while developing CL 2506253002 I spent a lot of time trying different settings, and came to the above conclusion.  There is something called ParameterizedTest that may be able to handle this situation, but it seems complicated and I had trouble finding any good examples to guide its usage.

 
ParameterizedTests manipulating command-line flags also do so via @CommandLineFlags. The primary (and possibly only?) example of this is in webview: https://codesearch.chromium.org/chromium/src/android_webview/javatests/src/org/chromium/android_webview/test/AwTestBase.java?rcl=0&l=57

The java-side command-line flags logic is incapable of removing command-line flags set by other mechanisms because the native CommandLine does not support removing flags. While suboptimal, this was intentional and is documented in @CommandLineFlags.Remove: https://codesearch.chromium.org/chromium/src/base/test/android/javatests/src/org/chromium/base/test/util/CommandLineFlags.java?rcl=0&l=53
(that's not to say we shouldn't do anything here, just an explanation of the current behavior)

Comment 3 by donnd@chromium.org, Dec 14 2016

It's good to have confirmation of the underlying reason, thanks!
Components: Test>Android

Sign in to add a comment