Two suggestions have been made to try to reduce the number of HWtest failures the toolchain testers are seeing:

1). update the suite_min_duts value for the waterfall/rotating builders, so that if there isn't a minimum number of 'good' machines available, the test will say so and not try to run.

2). make sure that the builder timeout for the tests is at least as long as the suite timeout for the tests.