New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 768676 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner:
Last visit > 30 days ago
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

ts_mon on sysmon continues to report prodrole of servers after they are removed from serverdb, causing alert noise

Project Member Reported by akes...@chromium.org, Sep 26 2017

Issue description

(thought there was an existing bug about this, but couldn't find it, so filing)

When a server is removed from server_db, sysmon on the master notices and stops actively setting the prod role metric for that server.

However, ts_mon continues to latch the previous gauge value until the next time sysmon is restarted, resulting in a potentially very long window in which we report incorrect prodrole information to monarch. This results in prod-role-based alerts being sent about servers that are no longer in production.

-> phobbs who discovered this behavior and is working on a fix
 

Comment 1 by pho...@chromium.org, Sep 26 2017

Mergedinto: 767265
Status: Duplicate (was: Untriaged)
Actually, Allen discovered it :)

Sign in to add a comment