In addition to expired tasks, we also need to measure the times tasks stay in queue. This should be a NonCumulativeDistributionMetric - instantaneous snapshot of how long each task has been waiting.
Started an internal discussion https://goto.google.com/oigum and uploaded https://codereview.chromium.org/2121323002 .
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/a00b8e245035fd044ac2af5007b5523cd71e8ee1 commit a00b8e245035fd044ac2af5007b5523cd71e8ee1 Author: phajdan.jr <phajdan.jr@chromium.org> Date: Mon Jul 11 13:42:51 2016 swarming: add active jobs pending times metric BUG= chromium:624508 Review-Url: https://codereview.chromium.org/2121323002 [modify] https://crrev.com/a00b8e245035fd044ac2af5007b5523cd71e8ee1/appengine/swarming/ts_mon_metrics.py [modify] https://crrev.com/a00b8e245035fd044ac2af5007b5523cd71e8ee1/appengine/swarming/ts_mon_metrics_test.py
Comment 1 by sergeybe...@chromium.org
, Jun 29 2016