New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 889109 link

Starred by 1 user
Status: Verified
Owner:
█ gu...@chromium.org
OOO
Closed: Sep 25
Cc:
zamorzaev@chromium.org
pprabhu@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

cros-full-0021.mtv unhealthy

Project Member Reported by zamorzaev@chromium.org, Sep 25 
The tick rate dropped to zero last night: http://shortn/_PXr88Vp0qT

Comment 1 by zamorzaev@chromium.org, Sep 25

Owner: zamorzaev@chromium.org

Comment 2 by zamorzaev@chromium.org, Sep 25 

The shard has been continuously restarting: http://shortn/_vRrvWzGRSc

Comment 3 by zamorzaev@chromium.org, Sep 25

Cc: pprabhu@chromium.org

Comment 4 by zamorzaev@chromium.org, Sep 25

Cc: gu...@chromium.org

Comment 5 by gu...@chromium.org, Sep 25

Cc: -gu...@chromium.org zamorzaev@chromium.org
Owner: gu...@chromium.org
The reason was scheduler cannot connect to TKO.

Yesterday, I removed the internal IP and public IP of cros-full-000{1,2} from TKO whitelist. I didn't noticed that cros-full-0021 has SAME public IP with cros-full-0001, see

$ runlocalssh ssh cros-full-0001.mtv.corp.google.com dig +short myip.opendns.com @resolver1.opendns.com
104.132.0.66

$ runlocalssh ssh cros-full-0021.mtv.corp.google.com dig +short myip.opendns.com @resolver1.opendns.com
104.132.0.66

Thus the connection from cros-full-0021 was blocked.

I resolved this issue by re-adding the IP back.

Comment 6 by zamorzaev@chromium.org, Sep 25

Status: Verified (was: Untriaged)
►

Sign in to add a comment