"platform_InstallTestImage" fails with error "AutoservVerifyDependencyError: [Errno 113] No route to host" |
|||||
Issue descriptionHere is a sample failure, 18:54:58 INFO | autoserv| Traceback (most recent call last): 18:54:58 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 572, in run_autoserv 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_server_job.py", line 65, in run 18:54:58 INFO | autoserv| super(site_server_job, self).run(*args, **dargs) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 780, in run 18:54:58 INFO | autoserv| self._execute_code(server_control_file, namespace) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 1280, in _execute_code 18:54:58 INFO | autoserv| execfile(code_file, namespace, namespace) 18:54:58 INFO | autoserv| File "/tmp/test_that_results_KBSIRW/results-1-platform_InstallTestImage/control.srv", line 71, in <module> 18:54:58 INFO | autoserv| parallel_simple(run, machines) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/subcommand.py", line 93, in parallel_simple 18:54:58 INFO | autoserv| function(arg) 18:54:58 INFO | autoserv| File "/tmp/test_that_results_KBSIRW/results-1-platform_InstallTestImage/control.srv", line 36, in run 18:54:58 INFO | autoserv| host = hosts.create_host(machine, servo_args=servo_args) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/factory.py", line 188, in create_host 18:54:58 INFO | autoserv| host_instance = host_class(hostname, **args) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/base_classes.py", line 56, in __init__ 18:54:58 INFO | autoserv| super(Host, self).__init__(*args, **dargs) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/base_classes.py", line 69, in __init__ 18:54:58 INFO | autoserv| self._initialize(*args, **dargs) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/cros_host.py", line 305, in _initialize 18:54:58 INFO | autoserv| try_servo_repair=try_servo_repair) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 846, in create_servo_host 18:54:58 INFO | autoserv| newhost.repair(silent=True) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 618, in repair 18:54:58 INFO | autoserv| self._repair_strategy.repair(self, silent) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/repair.py", line 703, in repair 18:54:58 INFO | autoserv| self._verify_root._verify_host(host, silent) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/repair.py", line 326, in _verify_host 18:54:58 INFO | autoserv| self._verify_dependencies(host, silent) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/repair.py", line 199, in _verify_dependencies 18:54:58 INFO | autoserv| self._verify_list(host, self._dependency_list, silent) 18:54:58 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/repair.py", line 188, in _verify_list 18:54:58 INFO | autoserv| raise AutoservVerifyDependencyError(self, failures) 18:54:58 INFO | autoserv| AutoservVerifyDependencyError: [Errno 113] No route to host 18:54:58 INFO | autoserv| 1 18:54:58 INFO | autoserv| Traceback (most recent call last): 18:54:58 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 773, in main 18:54:58 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 586, in run_autoserv 18:54:58 INFO | autoserv| SystemExit: 1 18:54:58 INFO | autoserv| record_state_duration failed: job_or_task_id=None, hostname=chromeos1-test-station, status=Running 18:54:58 INFO | autoserv| Logging subprocess finished 18:54:58 INFO | autoserv| Logging subprocess finished ----------------------------------------------------------------------------- /tmp/test_that_results_KBSIRW/results-1-platform_InstallTestImage [ FAILED ] ----------------------------------------------------------------------------- Total PASS: 0/1 (0%) Logs attached. test_that_results_KBSIRW
,
Dec 22 2016
I see the servod process running on the servo.
scripts $sshin chromeos1-test-station-servo
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
83:e8:17:ec:1c:de:63:14:db:d1:bf:d3:80:80:43:a0.
Please contact your system administrator.
Add correct host key in /home/aashutoshk/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/aashutoshk/.ssh/known_hosts:36
Password authentication is disabled to avoid man-in-the-middle attacks.
Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
83:e8:17:ec:1c:de:63:14:db:d1:bf:d3:80:80:43:a0.
Please contact your system administrator.
Add correct host key in /home/aashutoshk/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/aashutoshk/.ssh/known_hosts:36
localhost ~ # ps aux | grep -i servod
root 4123 91.1 6.2 17300 15292 ? Rs 17:23 0:06 /usr/bin/python2.7 /usr/lib/python-exec/python2.7/servod --host 0.0.0.0 --board caroline --port 9999
root 4131 0.0 0.1 1500 368 pts/0 S+ 17:23 0:00 grep --colour=auto -i servod
localhost ~ #
,
Dec 22 2016
Hmmm... Something's rotten in the state of Denmark:
$ ssh chromeos1-test-station-servo
Warning: Permanently added 'chromeos1-test-station-servo' (RSA) to the list of known hosts.
localhost ~ # status servod
servod (9999) start/running, process 6286
localhost ~ # cat /var/lib/servod/config_9999
BOARD=caroline
localhost ~ # logout
Connection to chromeos1-test-station-servo closed.
$ servo-stat chromeos1-test-station
chromeos1-test-station ...ABDEFGH not running servod BOARD=lumpy CHROMEOS_RELEASE_VERSION=8708.0.0
The 'servo-stat' command does the same ssh as shown above, yet it's
getting different answers.
,
Dec 22 2016
any tips how this can be fixed?
,
Dec 22 2016
<sigh> the servo-stat command hasn't kept pace with
various recent changes. Fix a few bugs, and you get
this:
$ servo-stat chromeos1-test-station
chromeos1-test-station ...ABDEFGH servod failed BOARD=caroline CHROMEOS_RELEASE_VERSION=8708.0.0
The "servod failed" may not be trustworthy because of lab ACL
restrictions. However:
$ ssh chromeos1-test-station-servo
Warning: Permanently added 'chromeos1-test-station-servo' (RSA) to the list of known hosts.
localhost ~ # dut-control pwr_button
Connection refused
localhost ~ #
So, there's definitely something wrong with the servo.
,
Dec 22 2016
> any tips how this can be fixed?
From inside the chroot, the following command must work
without error:
ssh chromeos1-test-station-servo true
Try that, and fix problems until it succeeds.
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by jrbarnette@chromium.org
, Dec 22 2016