The failures on veyron_minnie informational builds are presumably caused by the same TestLab failure, see the latest run https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8926880441334377840, though all hosts are working fine on veyron_minnie. 

I think it's due to there're only 3 working DUTs, and they cannot finish all the tests in the given suite in time. 

xixuan@xixuan0:~/chromiumos/infra/suite_scheduler$ dut-status -b veyron_minnie -p continuous
hostname                       S   last checked         URL
chromeos4-row9-rack9-host17    OK  2018-12-17 13:21:02  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack9-host17/3007699-cleanup/
chromeos4-row9-rack10-host7    OK  2018-12-17 13:21:57  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack10-host7/3007718-cleanup/
chromeos4-row9-rack10-host9    OK  2018-12-17 13:21:57  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos4-row9-rack10-host9/3007717-cleanup/


xixuan@xixuan0:~/chromiumos/infra/suite_scheduler$ dut-status -b caroline -p continuous
hostname                       S   last checked         URL
chromeos6-row2-rack23-host16   OK  2018-12-17 15:46:54  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos6-row2-rack23-host16/2685911-reset/
chromeos6-row2-rack21-host20   OK  2018-12-17 15:47:12  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos6-row2-rack21-host20/2685913-reset/
chromeos6-row2-rack23-host14   OK  2018-12-17 15:44:59  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos6-row2-rack23-host14/2685896-reset/
chromeos6-row4-rack23-host17   NO  2018-12-17 14:33:54  https://stainless.corp.google.com/browse/chromeos-autotest-results/hosts/chromeos6-row4-rack23-host17/2685630-repair/

There're no available DUTs to balance for veyron_minnie, file b/121159914 to eng-lab.

I add one more DUT to pool:continuuos, file b/121159736 to eng-lab to fix.

Re xixuan@, thanks for the reply! Let's see if it helps. 
But I just found that I made a mistake in the bug report, I checked the last green run and found the host was already broken at that time, so it looks like it was not the host that caused this failure. Sorry for the misleading information.

In the latest caroline run: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8926871626225005072, it failed at TestPlan, HWTest [bvt-inline] and HWTest [bvt-cq] steps, all with TestLabFailure. The same for the latest veyron_minnie: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?id=3240313. So I assume it's caused by the lab failure?

They also both failed at HWTest [chrome-informational] stage with a different error, tast.cryptohome.Login, tast.security.OpenFDs, tast.informat_SERVER_JOB failed, which might be caused by a different cause, I'll take a look at those.

Re #4, Yes, I tend to think the caroline failure https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8926871626225005072 is caused by lack of DUTs, as I see these DUTs work very hard to run these tests, but still can't finish them all:

See the timeline in suite details:
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/suiteDetails?suiteId=268059633
https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/suiteDetails?suiteId=268059636

I see, makes sense. It's hard to understand why the tests now require much more times to finish compared with previous runs. Take caroline informational pfq as an example, it usually takes about 300 minutes to finish, but recently, the number has bumped to more than 350 minutes, given the working hosts number are the same (3 hosts). 

The TestLab failure can on longer be observed on caroline information PFQ. Seems the lab has recovered. 
The replacement of the broken host is tracked in b/121159736.