New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 758744 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

6 devservers in the lab's shadow config aren't valid

Reported by jrbarnette@chromium.org, Aug 24 2017

Issue description

A recent attempt to update devservers in the lab turned up 6 devservers
that can't update because of various problems.  Below is the inventory
of the problems:

100.107.126.161
    This host is oneline (answers to ping), but ssh access times out.
    The IP address indicates that this host is in the android1758 lab.

100.107.225.251
    This is up, but it doesn't seem to have the usual chromeos-test
    account set up.
    I can't figure out where this host might be located.

100.107.225.252
    'ssh' fails with 'no route to host'.  'ping' works fine (hunh?)
    I can't figure out where this host might be located.

100.107.227.251
    This is up, but it doesn't seem to have the usual chromeos-test
    account set up.
    I can't figure out where this host might be located.

100.107.227.252
    'ssh' fails with a complaint about 'no matching key exchange method'.
    I can't figure out where this host might be located.

100.115.93.187
    The host is down.
    The IP address is in the same network as chromeos13-infra-devserver,
    but I've no idea where that might be located.

Absent some better idea, I think we should delete the whole lot of
them, since they can't actually be used.

 
I usually file a ticket to the cros lab team to fix this kind of devservers. If we remove them, we may have devserver load issue later.
> I usually file a ticket to the cros lab team to fix this kind of
> devservers. If we remove them, we may have devserver load issue later.

It's unlikely that englab-sys-cros owns any of the cited hosts.

However, as noted, I can't determine the location of most of the
hosts.

We should do some work to find out who owns the IP addresses in
question.  Alternatively, we could take a look at the blamelist
for the changes that added the offending devservers, and see if
that will turn up an owner.

Cc: jashur@chromium.org
100.107.126.161 - As noted, this is in android1758; maybe jashur@ knows about it.

100.115.93.187 - jashur@ added this with https://chrome-internal-review.googlesource.com/409433

The other four servers were added somewhere back in the mists of time.

Comment 4 by jashur@chromium.org, Aug 24 2017

Looks like the following has gone corrupt and it can be removed:
chromeos13-infra-devserver1	100.115.93.187

I increased memory as it was running low on the following:
(Can ping. Can now SSH)

jashur@jashur:~$ ping android1758-infra-devserver9
PING android1758-infra-devserver9.cros.corp.google.com (100.107.126.161) 56(84) bytes of data.
64 bytes from dhcp-100-107-126-161.mtv.corp.google.com (100.107.126.161): icmp_seq=1 ttl=62 time=0.645 ms
64 bytes from dhcp-100-107-126-161.mtv.corp.google.com (100.107.126.161): icmp_seq=2 ttl=62 time=0.615 ms
64 bytes from dhcp-100-107-126-161.mtv.corp.google.com (100.107.126.161): icmp_seq=3 ttl=62 time=0.542 ms
^C
--- android1758-infra-devserver9.cros.corp.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.542/0.600/0.645/0.051 ms
jashur@jashur:~$ ssh chromeos-test@android1758-infra-devserver9
chromeos-test@android1758-infra-devserver9's password: 
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-92-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Thu Aug 24 16:13:19 PDT 2017

  System load: 0.08               Memory usage: 0%   Processes:       156
  Usage of /:  1.6% of 884.77GB   Swap usage:   0%   Users logged in: 0

  Graph this data and manage this system at:
    https://landscape.canonical.com/

Last login: Tue Aug 15 22:02:05 2017 from 172.17.82.182
chromeos-test@android1758-infra-devserver9:~
Project Member

Comment 5 by bugdroid1@chromium.org, Aug 29 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/3e05ee641f614071797bfe97f0dda6a229d69cc3

commit 3e05ee641f614071797bfe97f0dda6a229d69cc3
Author: Richard Barnette <jrbarnette@google.com>
Date: Tue Aug 29 23:04:25 2017

Status: Fixed (was: Untriaged)

Sign in to add a comment