This can be done self-service. It involves making a change to chromite/cbuildbot/builders/simple_builders.py to ensure the stage is included in the set of stages to run, or to cbuildbot/chromeos_config.py to ensure that the config flags don't cause it to skip itself.

I suggest the gardener should own this, as they will be the one watching the pfq to see if it behaves as they expect.

According to config_lib.py, ImageTest should only be run if 'base' is in the image list. The PFQ builders only build the 'test' image, whereas paladin builders build 'base' and 'test'.

I'm not sure why that's the case -- why can't ImageTest run on a test image?

hmm, it's possible the test image step modifies the rootfs in a way imagetest doesn't like.  it's worth running a test though to see ...

or just enable "base" image for the chrome pfqs.

I can't even get image_test to pass locally on a "base" image. A couple failures:

======================================================================
ERROR: TestValidInterpreter (chromite.cros.test.image_test.BlacklistTest)
Fail if a script's interpreter is not found, or not executable.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/host/source/chromite/cros/test/image_test.py", line 142, in TestValidInterpreter
    with open(full_name, 'rb') as f:
IOError: [Errno 13] Permission denied: 'dir-ROOT-A/bin/passwd'

======================================================================
FAIL: TestImportedSymbolsAreAvailable (chromite.cros.test.image_test.SymbolsTest)
Ensure all ELF files' imported symbols are available in ROOT-A.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/host/source/chromite/cros/test/image_test.py", line 380, in TestImportedSymbolsAreAvailable
    self.assertFalse(failures, '\n'.join(failures))
AssertionError: File dir-ROOT-A/usr/lib64/pppd/2.4.6/winbind.so contains unsatisfied symbols: set(['chap_check_hook', 'chap_mdtype_all', 'notice', 'fatal', 'add_options', 'chap_verify_hook', 'mppe_set_keys2', 'novm', 'safe_fork', 'option_error', 'slprintf', 'pap_auth_hook', 'GenerateAuthenticatorResponse', 'ChallengeHash', 'ipcp_wantoptions', 'allowed_address_hook', 'pap_check_hook', 'mppe_set_keys'])

(the above failure is repeated for a large number of .so files....)

hrm, running as "sudo" passes the test. Trying on a test image now.

image_test passes locally on a test image, after a minor tweak (it needs to be told to look for a test image instead of just a base image). I think I'd rather do that than make the PFQ builders build multiple images.

Steven, does adding the image_test step to the PFQ builders sound sane to you?

+1 to not building multiple images.

As long as image_test doesn't take a significant amount of time to run, I have no objections to adding it to the PFQ.

We should add it to the pfq-informational builders as well if at all possible (again, assuming the test doesn't take very long) so that we have a smaller blamelist on failure.

if it passes on a test image, that sounds OK to me

to be fair, it wouldn't really "build" multiple images ... creating a test image involves first creating a base image and then installing/modifying it to be a test image.  if we requested a base & test image, we'd back up the base image before we turned it into a test image.

image_test itself is pretty fast and is designed to be so.  it tends to be run in parallel with other things, and should be possible to have it run in parallel with say Simple Chrome build steps.  a glance the CQ shows it currently taking less than 5 minutes.

image_test takes 70 seconds locally, on a base or test image. Couldn't find any reason not to run it on a test image, so we'll go that route unless something breaks.

I'll add it to the pfq-informational builders first to verify it works as expected.

Michael: Any update on this?

I suggest having a gardener tackle this while on shift, in case it causes issues in the PFQ.