I recently made a change to the COMMIT-QUEUE.ini file in the EC repo and it turns out that I actually made a mistake in the config. However, cbuildbot did not complain (as far as I'm aware). (See  https://crbug.com/642503 )

We should look into having some sort of sanity check for the changes to this file. Something as simple as verifying that the builder config actually exists would have been enough to catch this.