Make sustained high provision failure rate into an alert |
|||||||
Issue descriptionIt's already an Omen.
,
Oct 16 2017
,
Oct 17 2017
,
Oct 19 2017
,
Oct 23 2017
There was a typo that made the threshold 100x too low (0.16% instead of 16%). Should be fixed today.
,
Oct 24 2017
Another change collided and made the metric sources change before this landed. Waiting on another change to check that it's fixed.
,
Oct 24 2017
Tentatively declaring this fixed. Changes are up, and the alert is tracking the metric. Declaration is tentative because the Pcon graph for the alert, http://shortn/_EnNA3BNB4a, and the corresponding viceroy Omen, vi/chromeos/provision#_VG_huYBJmlb, don't seem to be tightly coupled.
,
Oct 25 2017
Re #7: I think you're looking at the wrong graph - here it is side by side with the Omen's underlying precomputation: http://shortn/_wY1i7w2w69
,
Oct 25 2017
Oh, excellent. Yes, those match up, that looks like the differences are entirely about how they're doing averaging. I'm not sure how to match up the queries more precisely. That's desirable but, I think, a low priority.
,
Oct 31 2017
Alert fired correctly today. |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by akes...@chromium.org
, Oct 11 2017Labels: -Pri-3 Chase-Pending Pri-1
Owner: ----
Status: Available (was: Untriaged)