May I correct:
- If a test depends on Chrome it should run in Chrome PFQ and CQ.
- If a test depends on Android it should run in Android PFQ, Chrome PFQ and CQ.
(Because a kernel change affect the higher stack and a Chrome change affects Android etc.)

At least at the moment a test (like tast) has no understanding on which builder it runs. And that is not a bad thing. But I can see a case for tast to know which builder it is on to interface with external scheduling decisions. (If I understand the proposal right you are adding commit flow path dependency with this, not just static/idempotent build dependency).

Whoops, sorry. Jet lag, I guess. Yeah, I realized that after posting -- the Chrome PFQ expression would actually be "bvt && (dep:chrome || dep:chrome_login || dep:android)".

I'm not sure I understand the question at the end of the comment. Attribute expressions are the mechanism by which whatever invokes tast tells it which tests to run (i.e. the expressions define test suites). Some tests in the suite may still be skipped if they have unsatisfied software dependencies. Tast doesn't currently do anything about deciding which DUTs tests should be scheduled on, though.

I think I can see now why you insist on "skipping" tests (correct in your more complex scenario, where otherwise failing tests indeed might be skipped instead of run depending on builder name) instead of "passing" them as I suggest (correct only if independent of builder name/function): you are planning to support more powerful scheduling decisions in tast.

Which is a good goal.

My problem is just, who watches tast and ensures it did consider all tests. What I am looking for is some double entry accounting ensuring we can sanity check that nothing is missing (for instance "bvt && (dep:chrome || dep:chrome_login)" was the right formula and computed correctly for lumpy but also for eve). Making sure the number of tests is constant independent of builder/board/model as CTS does is a simple to humans understand version of this check.

How can you add double entry accounting?
How can you ensure the correctness of tast attribute evaluation long term?

This is something autotest struggles with.

Didn't see #2 when I wrote #3.

Can you list/enumerate the tests based on the expressions before running? That might be a version of double entry accounting that I am looking for.

Maybe my concerns in #3 are really overblown? In autotest it is intractable, because different parts of the system can decide to skip tests/jobs in different places and there is no expectation of what should have been executed. Tast in itself is fairly compact, all code local and it could be sufficiently tested by unit tests.

I am struggling with this. Not trying to bike shed.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263

commit d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263
Author: Daniel Erat <derat@chromium.org>
Date: Wed May 09 08:11:04 2018

tast: Automatically add "dep:" test attributes.

Make testing.Test automatically add "dep:"-prefixed
attributes to tests specifying their stated software
dependencies. This makes the existing "chrome" and "arc"
attributes unnecessary, as "dep:chrome", "dep:chrome_login",
and "dep:android" can be used instead.

BUG= chromium:840593 
TEST=added unit tests; also manually verified that "dep:"
     attributes work as expected when included in attribute
     boolean expressions used to select tests and that the
     auto-added attributes are listed in results.json

Change-Id: Ica1b0fe1840769ae214fb161417953f1921432ab
Reviewed-on: https://chromium-review.googlesource.com/1048910
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263/src/chromiumos/tast/testing/test.go
[modify] https://crrev.com/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263/src/chromiumos/tast/testing/test_test.go
[modify] https://crrev.com/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263/docs/test_dependencies.md
[modify] https://crrev.com/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263/docs/running_tests.md
[modify] https://crrev.com/d8c4c4c8a08dc9c67f6c1afec0d216786d4f3263/docs/test_attributes.md

It would be straightforward to update the "tast" Autotest server test to log the list of tests that are being considered. Would that help? I'm not sure if there's any way to make the list more visible than that.