No, it's not desired.

Should be a straightfoward fix then, to bail out quickly if binhost test fails.

FWIW, you may be able to submit certain CLs (CLs not part of minilayout) in case of binhosttest failure. Probably a future improvement as binhosttest failure should be rare.

Hum....

What if we pull binhost test out of the CQ master, and run it in the PreCQ?

It could be limited to just configuration changes.

Way too risky. Can be broken by manifest changes, config changes, overlay/useflag changes.

(we run it in the precq already for chromite changes btw)

I think this behavior has been there for a long time. one possible improvement is when the CQ master triages failures from slaves, it can check the failures from the master build. If the master build reports failures before the CompletionStage, it can just trigger self-destruction and stop waiting for all slaves as the master is relevant to all CLs. chromium:724281 is another failure in the master build. I think the two bugs can be de-dup'ed if the solution is same. 

Killing the master on test failure will ensure we can't submit bad CLs despite a test failure, and is the right thing to do right away.

If we want to be smarter, we can treat the test like a build slave that config, manifest, overlay/ebuild/portage CLs depend on, and not blame clearly unrelated CLs.

If it helped, we could turn the binhost test into a slave builder that's not board specific instead of running it on the master.

I think that's what I meant in #6, in the completion stage, when CQ master checks the failures reported from slaves, it also checks if there're any failures reported from the master build, it triages the failures and relevant changes, stops waiting for slaves once it notices no CLs will be submitted. and we don't need a separate slave build only for binhost test?

I was just thinking it might fit into the current logic better. Whatever is easier.