New issue
Advanced search Search tips

Issue 918012 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

daisy_skate-release: fail on tast.ui.ChromeLogin with Lost SSH connection to chromeos4-row9-rack5-host13:2… timed out, Test did not finish

Project Member Reported by mukai@chromium.org, Dec 27

Issue description

https://stainless.corp.google.com/browse/chromeos-autotest-results/270921584-chromeos-test/
https://stainless.corp.google.com/browse/chromeos-autotest-results/269455588-chromeos-test/

It is still not sure why this happens twice recently, but it seems not a device issue. Fails on ChromeLogin due to connection timeout of SSH.

12/26 13:40:06.284 DEBUG|              test:0381| Test failed due to 1 failed: ui.ChromeLogin; 6 missing: ui.ChromeRespawn ui.SessionManagerRespawn ui.SingleProcessMashLogin .... Exception log follows the after_iteration_hooks.
12/26 13:40:06.284 DEBUG|              test:0386| Starting after_iteration_hooks for tast
12/26 13:40:06.294 DEBUG|          ssh_host:0310| Running (ssh) 'test -e "/tmp/sysinfo/autoserv-JDLMM4"' from 'decorated_func|wrapper|_install|path_exists|run|run_very_slowly'
12/26 13:41:09.516 DEBUG|             utils:0287| [stderr] ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out
12/26 13:41:09.518 ERROR|               log:0027| post-test iteration server sysinfo error:
12/26 13:41:09.519 ERROR|         traceback:0013| Traceback (most recent call last):
12/26 13:41:09.519 ERROR|         traceback:0013|   File "/usr/local/autotest/client/common_lib/log.py", line 25, in decorated_func
12/26 13:41:09.519 ERROR|         traceback:0013|     fn(*args, **dargs)
12/26 13:41:09.520 ERROR|         traceback:0013|   File "/usr/local/autotest/server/test.py", line 66, in wrapper
12/26 13:41:09.520 ERROR|         traceback:0013|     host, at, outputdir = self._install()
12/26 13:41:09.520 ERROR|         traceback:0013|   File "/usr/local/autotest/server/test.py", line 142, in _install
12/26 13:41:09.520 ERROR|         traceback:0013|     if not autodir or not self.host.path_exists(autodir):
12/26 13:41:09.520 ERROR|         traceback:0013|   File "/usr/local/autotest/client/common_lib/hosts/base_classes.py", line 520, in path_exists
12/26 13:41:09.521 ERROR|         traceback:0013|     ignore_status=True)
12/26 13:41:09.521 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 335, in run
12/26 13:41:09.521 ERROR|         traceback:0013|     return self.run_very_slowly(*args, **kwargs)
12/26 13:41:09.521 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 324, in run_very_slowly
12/26 13:41:09.521 ERROR|         traceback:0013|     ssh_failure_retry_ok)
12/26 13:41:09.522 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 260, in _run
12/26 13:41:09.522 ERROR|         traceback:0013|     raise error.AutoservSSHTimeout("ssh timed out", result)
12/26 13:41:09.522 ERROR|         traceback:0013| AutoservSSHTimeout: ('ssh timed out', * Command: 
12/26 13:41:09.522 ERROR|         traceback:0013|     /usr/bin/ssh -a -x   -o Protocol=2 -o StrictHostKeyChecking=no -o
12/26 13:41:09.523 ERROR|         traceback:0013|     UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o
12/26 13:41:09.523 ERROR|         traceback:0013|     ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4
12/26 13:41:09.523 ERROR|         traceback:0013|     -l root -p 22 chromeos4-row9-rack5-host13 "export LIBC_FATAL_STDERR_=1; if
12/26 13:41:09.523 ERROR|         traceback:0013|     type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
12/26 13:41:09.523 ERROR|         traceback:0013|     \"server[stack::_install|path_exists|run] -> ssh_run(test -e
12/26 13:41:09.523 ERROR|         traceback:0013|     \\\"/tmp/sysinfo/autoserv-JDLMM4\\\")\";fi; test -e \"/tmp/sysinfo
12/26 13:41:09.523 ERROR|         traceback:0013|     /autoserv-JDLMM4\""
12/26 13:41:09.523 ERROR|         traceback:0013| Exit status: 255
12/26 13:41:09.524 ERROR|         traceback:0013| Duration: 63.2097308636
12/26 13:41:09.524 ERROR|         traceback:0013| 
12/26 13:41:09.524 ERROR|         traceback:0013| stderr:
12/26 13:41:09.524 ERROR|         traceback:0013| ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out)
12/26 13:41:09.524 DEBUG|              test:0391| after_iteration_hooks completed
12/26 13:41:09.525 WARNI|              test:0606| The test failed with the following exception
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/test.py", line 600, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 800, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 464, in execute
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 371, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/server/site_tests/tast/tast.py", line 158, in run_once
    self._parse_results(run_failed)
  File "/usr/local/autotest/server/site_tests/tast/tast.py", line 369, in _parse_results
    raise error.TestFail(failure_msg)
TestFail: 1 failed: ui.ChromeLogin; 6 missing: ui.ChromeRespawn ui.SessionManagerRespawn ui.SingleProcessMashLogin ...
12/26 13:41:09.525 DEBUG|              test:0611| Running cleanup for test.
12/26 13:41:09.526 DEBUG|   logging_manager:0627| Logging subprocess finished
12/26 13:41:09.528 DEBUG|   logging_manager:0627| Logging subprocess finished
12/26 13:41:09.532 INFO |              test:0893| Starting after_hook for tast
12/26 13:41:09.537 DEBUG|          ssh_host:0310| Running (ssh) 'test -e "/tmp/sysinfo/autoserv-JDLMM4"' from 'decorated_func|wrapper|_install|path_exists|run|run_very_slowly'
12/26 13:42:12.744 DEBUG|             utils:0287| [stderr] ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out
12/26 13:42:12.745 ERROR|               log:0027| post-test server sysinfo error:
12/26 13:42:12.745 ERROR|         traceback:0013| Traceback (most recent call last):
12/26 13:42:12.745 ERROR|         traceback:0013|   File "/usr/local/autotest/client/common_lib/log.py", line 25, in decorated_func
12/26 13:42:12.746 ERROR|         traceback:0013|     fn(*args, **dargs)
12/26 13:42:12.746 ERROR|         traceback:0013|   File "/usr/local/autotest/server/test.py", line 66, in wrapper
12/26 13:42:12.746 ERROR|         traceback:0013|     host, at, outputdir = self._install()
12/26 13:42:12.746 ERROR|         traceback:0013|   File "/usr/local/autotest/server/test.py", line 142, in _install
12/26 13:42:12.746 ERROR|         traceback:0013|     if not autodir or not self.host.path_exists(autodir):
12/26 13:42:12.746 ERROR|         traceback:0013|   File "/usr/local/autotest/client/common_lib/hosts/base_classes.py", line 520, in path_exists
12/26 13:42:12.746 ERROR|         traceback:0013|     ignore_status=True)
12/26 13:42:12.746 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 335, in run
12/26 13:42:12.747 ERROR|         traceback:0013|     return self.run_very_slowly(*args, **kwargs)
12/26 13:42:12.747 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 324, in run_very_slowly
12/26 13:42:12.747 ERROR|         traceback:0013|     ssh_failure_retry_ok)
12/26 13:42:12.747 ERROR|         traceback:0013|   File "/usr/local/autotest/server/hosts/ssh_host.py", line 260, in _run
12/26 13:42:12.747 ERROR|         traceback:0013|     raise error.AutoservSSHTimeout("ssh timed out", result)
12/26 13:42:12.747 ERROR|         traceback:0013| AutoservSSHTimeout: ('ssh timed out', * Command: 
12/26 13:42:12.748 ERROR|         traceback:0013|     /usr/bin/ssh -a -x   -o Protocol=2 -o StrictHostKeyChecking=no -o
12/26 13:42:12.748 ERROR|         traceback:0013|     UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o
12/26 13:42:12.748 ERROR|         traceback:0013|     ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4
12/26 13:42:12.748 ERROR|         traceback:0013|     -l root -p 22 chromeos4-row9-rack5-host13 "export LIBC_FATAL_STDERR_=1; if
12/26 13:42:12.748 ERROR|         traceback:0013|     type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
12/26 13:42:12.748 ERROR|         traceback:0013|     \"server[stack::_install|path_exists|run] -> ssh_run(test -e
12/26 13:42:12.748 ERROR|         traceback:0013|     \\\"/tmp/sysinfo/autoserv-JDLMM4\\\")\";fi; test -e \"/tmp/sysinfo
12/26 13:42:12.748 ERROR|         traceback:0013|     /autoserv-JDLMM4\""
12/26 13:42:12.748 ERROR|         traceback:0013| Exit status: 255
12/26 13:42:12.748 ERROR|         traceback:0013| Duration: 63.1978828907
12/26 13:42:12.748 ERROR|         traceback:0013| 
12/26 13:42:12.748 ERROR|         traceback:0013| stderr:
12/26 13:42:12.749 ERROR|         traceback:0013| ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out)
12/26 13:42:12.749 INFO |              test:0897| after_hook completed
12/26 13:42:12.754 DEBUG|          ssh_host:0310| Running (ssh) 'rm -rf /tmp/sysinfo/autoserv-JDLMM4' from 'group_func|runtest|cleanup|uninstall|run|run_very_slowly'
12/26 13:43:15.977 DEBUG|             utils:0287| [stderr] ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out
12/26 13:43:15.988 DEBUG|          ssh_host:0310| Running (ssh) 'rm -rf "/tmp/sysinfo/autoserv-JDLMM4"' from 'cleanup|close|close|close|run|run_very_slowly'
12/26 13:44:19.214 ERROR|             utils:0287| [stderr] ssh: connect to host chromeos4-row9-rack5-host13 port 22: Connection timed out
12/26 13:44:19.217 ERROR|              test:0252| Error cleaning up the sysinfo autotest/host objects, ignoring it
 
Components: Infra>Client>ChromeOS
Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>Test>Platform
Labels: Hotlist-Deputy
Status: Available (was: Untriaged)
I don't think this is actionable, but let's keep tracking it.

Sign in to add a comment