Example build:
https://luci-milo.appspot.com/buildbot/chromeos/lars-release/2213

Two different HWTest stages timed out:

https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Flars-release%2F2213%2F%2B%2Frecipes%2Fsteps%2FHWTest__bvt-arc_%2F0%2Fstdout
  Suite timed out. Started on 06-05-2018 [22:30:23], timed out on 06-06-2018 [01:31:43]

https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Flars-release%2F2213%2F%2B%2Frecipes%2Fsteps%2FHWTest__bvt-installer_%2F0%2Fstdout
23:17:48: ERROR: ** Suite timed out before completion **
23:17:48: INFO: Translating result ** Suite timed out before completion ** to fail.

Overall suite details shows 6 DUTs and them sharing duty between both HWTest and Paygen stages:
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/suiteDetails?cidbBuildId=2638292

In particular, the Paygen stages have many many long autoupdate_EndToEnd tests which start to get interspersed with the other stages.  In addition, the devices end up getting provisioned multiple times to the same version wasting needless time.  Once there is a failure or two, there is no room for error and the entire build fails.

A number of other builds on the same canary run failed with what appears to be similar time outs.  I haven't yet investigated if they're the same patterns:
https://luci-milo.appspot.com/buildbot/chromeos/caroline-release/1810
https://luci-milo.appspot.com/buildbot/chromeos/daisy_spring-release/3601
https://luci-milo.appspot.com/buildbot/chromeos/kefka-release/2271
https://luci-milo.appspot.com/buildbot/chromeos/lulu-release/2248
https://luci-milo.appspot.com/buildbot/chromeos/terra-release/2271

While there's a couple of issues here (including provisions and autoupdate tests failing), I think the fact that the timeouts for the suites are too short, and then parallel running of the suites without avoid needless work are the issues this bug should address.