qslib/scheduler exposes a NotifyTask call that allows you to specify that a task is idle, and a separate call to specify that it is aborted. This was based on the initial guess at the right behavior for the scheduler, in which tasks are moved back to the queue immediately when they are preempted, and in which in general we assume that tasks should be enqueued unless we know otherwise.

Following up to  Issue 900664 , the desired behavior is really that tasks should be removed from the scheduler's queue when they are preempted. Swarming will then inform us once the tasks are re-enqueued, and scheduler will do so.