One solution is to add a console command that allows the host to select a test or specify a test ordering, etc. This is probably the optimal solution but requires the most work.

Another solution could be that the boards could permute the tests themselves and intelligently leave tests which are blocking. Preferably the solution chosen would not involve any communication between boards, since that potentially affect test results and would require extra wiring/use of the same drivers that we are supposed to be testing.

A 'quick and dirty' solution could be for the host to reorder the list of tests saved on the machine and rebuild/reflash the boards. This would take less time to implement but would affect performance significantly, since flashing is a big bottleneck (~30 seconds to flash the test harness each time).

I don't think this is how CTS works (in autotest), but maybe we should chat over coffee tomorrow afternoon?

High level:
a) autotest reboots after each package.
b) inside of each package tradefed choses the order of tests (each time the same). We really are not interested in changing tradefed's order. Way too scary.
c) hangs will be dealt with using retries.

I think there might have been a misunderstanding, I'm actually referring to the Chromium OS Embedded Controller CTS.

We are working on a testing system for the Chromium embedded controllers and we chose to call it CTS, but we are considering renaming it so that it does not get confused with Android CTS. Apologies for the confusion!

Ah, yes. Please rename to avoid confusion with Android CTS.

The assigned owner "twothreecc@google.com" is not able to receive e-mails, please re-triage.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot