Issue 735388 is a case where I suspect higher priority tasks with _slightly_ different dimensions preempted lower priority tasks enough to make them time out.

It'd be great to make an API to summarize:
- look at the kind of tasks currently run by the bots that are in the set of a set of dimensions and report their (dimensions, priority) outlook
- leverage TaskDimensions* to figure out the current pending tasks.

I'm thinking about implementing this as two separate API, as both can be done concurrently and are querying different kind of entities. I'm not sure how this can made to efficiently scale to >1000 bots, we'll likely have to sample instead of a complete set. Sampling is probably good enough.

Coupled with issue 729565, this would create a quite powerful tool.

* https://github.com/luci/luci-py/blob/master/appengine/swarming/server/task_queues.py