New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682512 link

Starred by 1 user

Issue metadata

Status: Archived
Owner: ----
Closed: Mar 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

moblab stuck in a loop running a single test

Project Member Reported by dchan@google.com, Jan 19 2017

Issue description

moblab ip 100.107.195.216

missing 795-moblab test result in /mnt/moblab/results
moblab UI shows 795 being atttempt over and over https://screenshot.googleplex.com/Mk093CWqpwS.png

looking at one of the reset failure
/mnt/moblab/results/hosts/192.168.231.103/4039-reset/debug/autoserv.INFO
shows

01/18 17:34:24.460 NOTIC|      cros_logging:0037| ts_mon was set up.
01/18 17:35:55.237 ERROR|        site_utils:0937| Timed out waiting for login prompt
01/18 17:35:55.237 ERROR|             reset:0031| Timed out waiting for login prompt
Traceback (most recent call last):
  File "/usr/local/autotest/server/control_segments/reset", line 23, in reset
    target.cleanup()
  File "/usr/local/autotest/server/hosts/cros_host.py", line 1402, in cleanup
    self._restart_ui()
  File "/usr/local/autotest/server/hosts/cros_host.py", line 1313, in _restart_ui
    cros_ui.wait_for_chrome_ready(prompt, self)
  File "/usr/local/autotest/client/cros/cros_ui.py", line 65, in wait_for_chrome_ready
    timeout=timeout, sleep_interval=1.0)
  File "/usr/local/autotest/client/common_lib/site_utils.py", line 938, in poll_for_condition
    raise exception
TimeoutError: Timed out waiting for login prompt
01/18 17:35:55.239 INFO |        server_job:0153| END FAIL      ----    reset   timestamp=1484789755    localtime=Jan 18 17:35:55
01/18 17:35:56.249 INFO |     ts_mon_config:0150| Waiting for ts_mon flushing process to finish...
01/18 17:35:56.262 ERROR|         traceback:0013| Traceback (most recent call last):
01/18 17:35:56.262 ERROR|         traceback:0013|   File "/usr/local/autotest/server/autoserv", line 539, in run_autoserv
01/18 17:35:56.263 ERROR|         traceback:0013|     job.reset(job_labels)
01/18 17:35:56.263 ERROR|         traceback:0013|   File "/usr/local/autotest/server/server_job.py", line 480, in reset
01/18 17:35:56.263 ERROR|         traceback:0013|     self._execute_code(RESET_CONTROL_FILE, namespace, protect=False)
01/18 17:35:56.264 ERROR|         traceback:0013|   File "/usr/local/autotest/server/server_job.py", line 1280, in _execute_code
01/18 17:35:56.264 ERROR|         traceback:0013|     execfile(code_file, namespace, namespace)
01/18 17:35:56.265 ERROR|         traceback:0013|   File "/usr/local/autotest/server/control_segments/reset", line 45, in <module>
01/18 17:35:56.265 ERROR|         traceback:0013|     job.parallel_simple(reset, machines)
01/18 17:35:56.265 ERROR|         traceback:0013|   File "/usr/local/autotest/server/server_job.py", line 604, in parallel_simple
01/18 17:35:56.266 ERROR|         traceback:0013|     return_results=return_results)
01/18 17:35:56.266 ERROR|         traceback:0013|   File "/usr/local/autotest/server/subcommand.py", line 93, in parallel_simple
01/18 17:35:56.266 ERROR|         traceback:0013|     function(arg)
01/18 17:35:56.267 ERROR|         traceback:0013|   File "/usr/local/autotest/server/control_segments/reset", line 35, in reset
01/18 17:35:56.267 ERROR|         traceback:0013|     raise Exception('')
01/18 17:35:56.268 ERROR|         traceback:0013| Exception
01/18 17:35:56.268 ERROR|          autoserv:0797| 1
Traceback (most recent call last):
  File "/usr/local/autotest/server/autoserv", line 793, in main
    use_ssp)
  File "/usr/local/autotest/server/autoserv", line 584, in run_autoserv
    sys.exit(exit_code)
SystemExit: 1


I manually reboot the DUT and test seems to continue.
 

Comment 2 by ntang@google.com, Mar 1 2017

Status: Archived (was: Untriaged)
Most of the time is the DUT could not get back into READY state. Close the issue for now since I could not get into 100.107.195.216 to look at the results log. Please reopen if you see the error again.

Sign in to add a comment