New issue
Advanced search Search tips

Issue 889354 link

Starred by 1 user
Status: Available
Owner: ----
Cc:
derat@chromium.org
hidehiko@chromium.org
nya@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

Proposal: Making "disabled" a special test attribute

Project Member Reported by nya@chromium.org, Sep 26 
Problem:

When we want to run all ARC tests, the most intuitive way is:

 $ tast run DUT 'arc.*'

However this also runs arc.BootForever which never finishes (unless it fails), which is confusing to users.


Proposal:

I propose to exclude disabled tests from wildcard matching. We can accomplish this in several ways. For example:

 a. Treat "disabled" attribute specially on wildcard matching.

 b. Introduce `Enabled` (or `Disabled`) test property and deprecate "disabled" attribute. So a disabled test declaration will be like:

      testing.AddTest(&testing.Test{
          ...
          Attr:    []string{"informational"},
          Enabled: false,
      })

Comment 1 by derat@chromium.org, Sep 26 

Hmm. In general, I think we may have enough tests in most categories at some point (possibly not too far away) that it will no longer be feasible to use e.g. 'category.*' to run all tests from a given category (because it'll take too long).

With that said, I'm not completely opposed to replacing the "disabled" attribute with a Disabled field in testing.Test. (I don't think we want an Enabled field, since tests will default to being disabled in that case.)

More broadly, I'm still hazy on the long-term plans for disabling tests. So far, I think the "disabled" attribute has only been used for loop-forever tests. If a test is breaking the CQ, it's just as easy to add the "informational" attribute to it. I guess we may still want to disable a test if it's causing problems even when running informationally.

I think my preference is to avoid adding more fields to Test unless we think they'll be frequently used, but I can probably be convinced otherwise if wildcard patterns are useful. :-)
►

Sign in to add a comment