Step 2 will be removed by work that phobbs is doing.

Steps 4 and 6 can optionally be the same waterfall restart.

I would really like it if step 7 was possible after step 4, or if it wasn't needed at all (perhaps the slave keeps trying in a loop until it works).

It would also be nice if step 3 wasn't needed and a new slave could somehow add itself to the waterfall.

Also, I was really surprised. I though step 7 could be successfully done after step 4. But that doesn't seem to be true.

{3, 4, 5, 6} should probably be the same CL if a slave is being brought up for a specific purpose. The alternative is what we do now with baremetal, which is to preallocate slaves, add them to the waterfall out of band, and then {5, 6} as needed.

Additionally, 7 could be rendered unnecessary if the cleanup is done or if you switch to ccompute. Basically what you do is have a launcher that repeats:
1) Launch Slave
2) Wait for slave process to terminate.
3) Sleep 10 seconds.
4) Goto 1.

If you do this, (2) will happen immediately if the slave is rejected by the master, so this will basically poll continuously until the master is ready.

I'm hoping that the launcher script you mention into new images soon.

I'm assuming the switch to ccompute will take significant work and thus take time, but that is the hope longer term.

I think hinoka@'s conclusion is that it actually probably wouldn't take too much work/time. He already set up a continuous image builder, although it's failing because the target it's supposed to build doesn't exist: https://uberchromegw.corp.google.com/i/internal.infra.cron/builders/ccompute-chrome-chromeos

Since the move to ccompute, this process it a lot less awkward.