Increase trigger duration for SlaveFreeDiskSpace(Very)Low alerts |
||||||
Issue descriptionShould we try increasing the alert trigger duration for SlaveFreeDiskSpace(Very)Low alerts. During the latest shift by katthomas, she saw ~20 which resolved themselves. ToDo: Investigate and implement a new alert trigger duration Assigning to CCI to review. CC'ing katthomas to provide additional comments
,
Feb 7 2018
Especially since the foundation FA seems to own these alerts: https://cs.corp.google.com/piper///depot/google3/configs/monitoring/chrome_ops_foundation/memory_alerts.py?rcl=182129921&l=15
,
Feb 10 2018
@efoo, please rewrite the components of the bug, or if you think it really is infra>Client>Chrome's bug, feel free to add it back.
,
Feb 10 2018
Reassigning to Infra>Platform. My bad. Katie, can you include relevant info based on your shift? Thanks!
,
Feb 12 2018
During my shift I closed ~20 SlaveFreeDiskSpace(Very)Low alerts which appeared to have resolved themselves. They ranged in age from 4 days old to several hours old. Currently the trigger duration is 1h. From my last shift, it appeared as though we weren't addressing these alerts in a timely manner to begin with, so increasing the trigger duration may have the effect of causing fewer alerts to fire because the issues have resolved themselves with little effect on the trooper response.
,
Feb 12 2018
,
Feb 21 2018
,
Feb 21 2018
While annoying, decreasing priority since this is a buildbot-only problem. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by jbudorick@chromium.org
, Feb 7 2018