See  bug 84940  for context: tasks were expiring on chromium.fyi and we either didn't get or ignored alerts for them.

Because we should generally never have tasks expiring in the system, we need to fix this.

There may be specific builder/tester configs for which we know we are intentionally and explicitly under-provisioned and as such there are expected to be expiring jobs. Those need to be exempted from the normal monitoring.

There is a related issue in that we need to decide how to prioritize different classes of jobs (in  bug 843737 ) in a cleaner way.