It's not just veyron_minnie. I've seen this on kevin as well. Looking historically, "short HWTest stages marked as passed" seemed to have started around 2017-04-04.

http://shortn/_vDCkWDaNB6

+ xixuan@

copy from email thread:

The suite doesn't find any tests to run since it's command is like: 
/usr/local/autotest/site_utils/run_suite.py --build kevin-paladin/R60-9473.0.0-rc1 --board kevin --suite_name suite_attr_wrapper ... --suite_args "{'attr_filter': u'(suite:b or suite:v or suite:t or suite:- or suite:i or suite:n or suite:l or suite:i or suite:n or suite:e) and (subsystem:default)'}" ...

the suite cannot find tests whose tags includes 'suite_attr_wrapper'. 

A normal command should be like:
/usr/local/autotest/site_utils/run_suite.py --build kevin-paladin/R60-9476.0.0-rc2 --board kevin --suite_name bvt-inline ...
/usr/local/autotest/site_utils/run_suite.py --build link-paladin/R60-9476.0.0-rc1 --board link --suite_name bvt-cq ....

Looks like 'suite_attr_wrapper' is decided by 'subsystems':
https://cs.corp.google.com/chromeos_public/chromite/cbuildbot/commands.py?type=cs&q=suite_attr_wrapper&sq=package:%5Echromeos_(internal%7Cpublic)$&l=1108

What's subsystems and how to decide whether a build should run with 'subsystems'?

+ shuqianz@ to answer the subsystem questions in #4

The problem looks to me like it is splitting "bvt-inline" intended attribute into ['b', 'v', 't', '-', 'i', 'n', 'l', 'i', 'n', 'e'] and treating those as individual attributes!

(otherwise, I think the general subsystem selection logic is correct, just not the command formulaiton)

Besides the splitting error, from the code https://cs.corp.google.com/chromeos_public/chromite/cbuildbot/commands.py?type=cs&q=suite_attr_wrapper&sq=package:%5Echromeos_(internal%7Cpublic)$&l=1108, if subsystems = True, no matter what's the real suite, we will kick off a suite_attr_wrapper suite?

subsystem logic is based on the suite specified in the attr_filter, however, the correct way is "{'attr_filer': '{suite: bvt-inline}'}". Any recent change to run_suite that change the argument parsing?

Reply #8 suite_attr_wrapper is a special suite, which is used to use the attr_filter args to trigger suites based on the attribute boolean expression.

Issue 713322 has been merged into this issue.

The problem is not in the run_suite argument parsing, it's in the arguments being passed to run_suite.

from commands.py:

    if suite != 'suite_attr_wrapper':
      if type(suite) is str:
        suite_attr_str = 'suite:%s' % suite
      else:
        suite_attr_str = ' or '.join(['suite:%s' % x for x in suite])

      attr_value = '(%s) and (%s)' % (suite_attr_str, subsystems_attr_str)
    else:
      attr_value = subsystems_attr_str

type(suite) is str:

should probably be replaced by

isinstance(suite, str):



Also, I don't know why we are supporting the suite argument to be be either a list or a string.

For the HWTest stage, the suite arg will be always to represent one suite. We can remove that iterate code on the suite arg.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/49283aabfba8a0a351394787d0e1245cc502f9c2

commit 49283aabfba8a0a351394787d0e1245cc502f9c2
Author: Shuqian Zhao <shuqianz@chromium.org>
Date: Fri Apr 21 09:34:48 2017

cbuildbot: always treat suite as a string

BUG= chromium:713258 
TEST=unittest

Change-Id: I42365fdb826d150423964a345cfcf025f819d2f2
Reviewed-on: https://chromium-review.googlesource.com/482402
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/49283aabfba8a0a351394787d0e1245cc502f9c2/cbuildbot/commands.py