New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 614068 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

[B1758] Replace Lars EVT with DVT

Project Member Reported by tienchang@chromium.org, May 23 2016

Issue description

Steps:
  Verify device (asset tag: C073530) powers on and boots
  Switch to dev mode
  Set gbb_flags for fast boot
  Install ToT test image
  Add servo cable
  Install in android1758-row4-rack3-host2
  Run network_WiFi_VerifyRouter in B1758
  Run platform_InstallTestImage in B1758
  Return Lars EVT
 
19:17:52 INFO | autoserv| Client complete
19:17:54 INFO | autoserv| GOOD  network_WiFi_VerifyRouter       network_WiFi_VerifyRouter       timestamp=1464049074    localtime=May 23 19:17:54   completed successfully
19:17:54 INFO | autoserv| END GOOD      network_WiFi_VerifyRouter       network_WiFi_VerifyRouter       timestamp=1464049074    localtime=May 23 19:17:54
19:17:54 INFO | autoserv| Finished processing control file
19:17:54 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_w5OaROssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmpHd9Alf -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=300 -l root -p 22 android1758-row4-rack3-host2.cros'
19:17:55 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x   -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_E83mP1ssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 android1758-row4-rack3-host2.cros'
19:17:56 INFO | autoserv| There are no orphaned crashes; deleting /tmp/test_that_results_j8znGC/results-1-network_WiFi_VerifyRouter/crashinfo.android1758-row4-rack3-host2.cros
19:17:57 INFO | autoserv| record_state_duration failed: job_or_task_id=None, hostname=android1758-row4-rack3-host2.cros, status=Running
-------------------------------------------------------------------------------------------------------
/tmp/test_that_results_j8znGC/results-1-network_WiFi_VerifyRouter                           [  PASSED  ]
/tmp/test_that_results_j8znGC/results-1-network_WiFi_VerifyRouter/network_WiFi_VerifyRouter [  PASSED  ]

Servo is pingable and servod process is running but platform_install test is failing for some reason. 

ping android1758-row4-rack3-host2-servo.cros
PING android1758-row4-rack3-host2-servo.cros.corp.google.com (100.107.127.99) 56(84) bytes of data.
64 bytes from dhcp-100-107-127-99.mtv.corp.google.com (100.107.127.99): icmp_seq=1 ttl=60 time=0.982 ms
64 bytes from dhcp-100-107-127-99.mtv.corp.google.com (100.107.127.99): icmp_seq=2 ttl=60 time=0.826 ms
^C
--- android1758-row4-rack3-host2-servo.cros.corp.google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.826/0.904/0.982/0.078 ms


ps -ef | grep -i servod
root      1615     1  0 00:28 ?        00:00:00 /usr/bin/python2.7 /usr/lib/python-exec/python2.7/servod --host 0.0.0.0 --board lars --port 9999
root      1622  1586  0 00:28 pts/0    00:00:00 grep --colour=auto -i servod


7:21:01 INFO | Began logging to /tmp/test_that_results_bMB7TC
Adding labels [u'cros-version:ad_hoc_build', u'board:lars'] to host android1758-row4-rack3-host2.cros
19:21:01 INFO | Fetching suite for job named platform_InstallTestImage...
19:21:04 INFO | Scheduling suite for job named platform_InstallTestImage...
19:21:04 INFO | ... scheduled 1 job(s).
19:21:04 INFO | autoserv| Results placed in /tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage
19:21:04 INFO | autoserv| Logged pid 1211 to /tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/.autoserv_execute
19:21:04 INFO | autoserv| I am PID 1211
19:21:04 INFO | autoserv| Not checking if job_repo_url contains autotest packages on ['android1758-row4-rack3-host2.cros']
19:21:04 INFO | autoserv| Processing control file
19:21:04 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_EWgvD9ssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmp6ELvky -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=300 -l root -p 22 android1758-row4-rack3-host2.cros'
19:21:06 INFO | autoserv| Pinging servo host at android1758-row4-rack3-host2-servo.cros
19:21:08 INFO | autoserv| Pinging host android1758-row4-rack3-host2-servo.cros
19:21:08 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_47uLgfssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=180 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 android1758-row4-rack3-host2-servo.cros'
19:21:09 INFO | autoserv| Host (ssh) android1758-row4-rack3-host2-servo.cros is alive
19:21:09 INFO | autoserv| Applying an update to the servo host, if necessary.
19:21:09 INFO | autoserv| servo host android1758-row4-rack3-host2-servo.cros does not require an update.
19:21:10 INFO | autoserv| servod is running, PID=3491
19:21:10 INFO | autoserv| Attempting to repair servo host android1758-row4-rack3-host2-servo.cros.
19:21:10 INFO | autoserv| START ----    reboot  timestamp=1464049270    localtime=May 23 19:21:10
19:21:10 INFO | autoserv| GOOD  ----    reboot.start    timestamp=1464049270    localtime=May 23 19:21:10
19:22:04 INFO | autoserv| [stderr] mux_client_request_session: read from master failed: Broken pipe
19:22:05 INFO | autoserv| [stderr] Warning: Permanently added 'android1758-row4-rack3-host2-servo.cros' (RSA) to the list of known hosts.
19:22:05 INFO | autoserv| [stderr] Warning: Permanently added 'android1758-row4-rack3-host2-servo.cros' (RSA) to the list of known hosts.
19:22:06 INFO | autoserv| Master ssh connection to android1758-row4-rack3-host2-servo.cros is down.
19:22:06 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_syP_3zssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=180 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 android1758-row4-rack3-host2-servo.cros'
19:22:07 INFO | autoserv| GOOD  ----    reboot.verify   timestamp=1464049327    localtime=May 23 19:22:07
19:22:07 INFO | autoserv| END GOOD      ----    reboot  kernel=3.8.13   localtime=May 23 19:22:07       timestamp=1464049327
19:22:27 INFO | autoserv| Pinging host android1758-row4-rack3-host2-servo.cros
19:22:27 INFO | autoserv| Host (ssh) android1758-row4-rack3-host2-servo.cros is alive
19:22:27 INFO | autoserv| Applying an update to the servo host, if necessary.
19:22:28 INFO | autoserv| servo host android1758-row4-rack3-host2-servo.cros does not require an update.
19:22:28 INFO | autoserv| servod is running, PID=623
19:22:28 INFO | autoserv| Failed to repair servo: [Errno 111] Connection refused
19:22:28 INFO | autoserv| Exception escaped control file, job aborting:
19:22:28 INFO | autoserv| Traceback (most recent call last):
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 684, in run
19:22:28 INFO | autoserv| self._execute_code(server_control_file, namespace)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 1182, in _execute_code
19:22:28 INFO | autoserv| execfile(code_file, namespace, namespace)
19:22:28 INFO | autoserv| File "/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/control.srv", line 72, in <module>
19:22:28 INFO | autoserv| parallel_simple(run, machines)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/subcommand.py", line 93, in parallel_simple
19:22:28 INFO | autoserv| function(arg)
19:22:28 INFO | autoserv| File "/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/control.srv", line 37, in run
19:22:28 INFO | autoserv| host = hosts.create_host(machine, servo_args=servo_args)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/factory.py", line 160, in create_host
19:22:28 INFO | autoserv| host_instance = host_class(hostname, **args)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/base_classes.py", line 58, in __init__
19:22:28 INFO | autoserv| super(Host, self).__init__(*args, **dargs)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/base_classes.py", line 70, in __init__
19:22:28 INFO | autoserv| self._initialize(*args, **dargs)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/cros_host.py", line 306, in _initialize
19:22:28 INFO | autoserv| try_lab_servo=try_lab_servo)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 789, in create_servo_host
19:22:28 INFO | autoserv| required_by_test=required_by_test)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/base_classes.py", line 58, in __init__
19:22:28 INFO | autoserv| super(Host, self).__init__(*args, **dargs)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/base_classes.py", line 70, in __init__
19:22:28 INFO | autoserv| self._initialize(*args, **dargs)
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 173, in _initialize
19:22:28 INFO | autoserv| self.repair()
19:22:28 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 708, in repair
19:22:28 INFO | autoserv| '\n'.join(errors))
19:22:28 INFO | autoserv| ServoHostRepairTotalFailure: All attempts at repairing the servo failed:
19:22:28 INFO | autoserv| [Errno 111] Connection refused
19:22:28 INFO | autoserv| INFO  ----    ----    timestamp=1464049348    job_abort_reason=All attempts at repairing the servo failed: [Errno 111] Connection refused  localtime=May 23 19:22:28       All attempts at repairing the servo failed:
19:22:28 INFO | autoserv| [Errno 111] Connection refused
19:22:28 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_vcbHVmssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmpr5t4Y7 -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=300 -l root -p 22 android1758-row4-rack3-host2.cros'
19:22:29 INFO | autoserv| Starting master ssh connection '/usr/bin/ssh -a -x   -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_2NZaCYssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 android1758-row4-rack3-host2.cros'
19:22:30 INFO | autoserv| Collecting crash information...
19:22:30 INFO | autoserv| There are no orphaned crashes; deleting /tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/crashinfo.android1758-row4-rack3-host2.cros
19:22:30 INFO | autoserv| android1758-row4-rack3-host2.cros already up, collecting crash info
19:22:30 INFO | autoserv| Collecting /var/tmp/messages.autotest_start...
19:22:30 INFO | autoserv| Collecting /var/log/messages...
19:22:30 INFO | autoserv| Collecting 'dmesg' ...
19:22:30 INFO | autoserv| Collecting /var/log...
19:22:30 INFO | autoserv| Collecting /dev/pstore/console-ramoops...
19:22:30 INFO | autoserv| trying scp, rsync failed: Command <rsync -L  --timeout=1800 --rsh='/usr/bin/ssh -a -x   -o ControlPath=/tmp/_autotmp_2NZaCYssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22' -az --no-o --no-g root@android1758-row4-rack3-host2.cros:"/dev/pstore/console-ramoops" "/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/crashinfo.android1758-row4-rack3-host2.cros/console-ramoops"> failed, rc=23, Command returned non-zero exit status
19:22:30 INFO | autoserv| * Command:
19:22:30 INFO | autoserv| rsync -L  --timeout=1800 --rsh='/usr/bin/ssh -a -x   -o ControlPath=/tmp
19:22:30 INFO | autoserv| /_autotmp_2NZaCYssh-master/socket -o StrictHostKeyChecking=no -o
19:22:30 INFO | autoserv| UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o
19:22:30 INFO | autoserv| ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4
19:22:30 INFO | autoserv| -o Protocol=2 -l root -p 22' -az --no-o --no-g
19:22:30 INFO | autoserv| root@android1758-row4-rack3-host2.cros:"/dev/pstore/console-ramoops" "/tmp
19:22:30 INFO | autoserv| /test_that_results_bMB7TC/results-1-platform_InstallTestImage/crashinfo.an
19:22:30 INFO | autoserv| droid1758-row4-rack3-host2.cros/console-ramoops"
19:22:30 INFO | autoserv| Exit status: 23
19:22:30 INFO | autoserv| Duration: 0.037220954895
19:22:30 INFO | autoserv| 
19:22:30 INFO | autoserv| stderr:
19:22:30 INFO | autoserv| rsync: link_stat "/dev/pstore/console-ramoops" failed: No such file or directory (2)
19:22:30 INFO | autoserv| rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1637) [Receiver=3.1.1]
19:22:30 INFO | autoserv| rsync: [Receiver] write error: Broken pipe (32)
19:22:31 INFO | autoserv| Collection of /dev/pstore/console-ramoops failed: scp -rq -o ControlPath=/tmp/_autotmp_2NZaCYssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmpqLRMhs -P 22 root@android1758-row4-rack3-host2.cros:"/dev/pstore/console-ramoops" '/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/crashinfo.android1758-row4-rack3-host2.cros/console-ramoops'
19:22:31 INFO | autoserv| * Command:
19:22:31 INFO | autoserv| scp -rq -o ControlPath=/tmp/_autotmp_2NZaCYssh-master/socket -o
19:22:31 INFO | autoserv| StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmpqLRMhs -P 22
19:22:31 INFO | autoserv| root@android1758-row4-rack3-host2.cros:"/dev/pstore/console-ramoops" '/tmp
19:22:31 INFO | autoserv| /test_that_results_bMB7TC/results-1-platform_InstallTestImage/crashinfo.an
19:22:31 INFO | autoserv| droid1758-row4-rack3-host2.cros/console-ramoops'
19:22:31 INFO | autoserv| Exit status: 1
19:22:31 INFO | autoserv| Duration: 0.196501970291
19:22:31 INFO | autoserv| 
19:22:31 INFO | autoserv| stderr:
19:22:31 INFO | autoserv| scp: /dev/pstore/console-ramoops: No such file or directory
19:22:31 INFO | autoserv| Collecting /sys/kernel/debug/dri/0/i915_error_state...
19:22:32 INFO | autoserv| Traceback (most recent call last):
19:22:32 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 547, in run_autoserv
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_server_job.py", line 65, in run
19:22:32 INFO | autoserv| super(site_server_job, self).run(*args, **dargs)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 684, in run
19:22:32 INFO | autoserv| self._execute_code(server_control_file, namespace)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/server_job.py", line 1182, in _execute_code
19:22:32 INFO | autoserv| execfile(code_file, namespace, namespace)
19:22:32 INFO | autoserv| File "/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/control.srv", line 72, in <module>
19:22:32 INFO | autoserv| parallel_simple(run, machines)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/subcommand.py", line 93, in parallel_simple
19:22:32 INFO | autoserv| function(arg)
19:22:32 INFO | autoserv| File "/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage/control.srv", line 37, in run
19:22:32 INFO | autoserv| host = hosts.create_host(machine, servo_args=servo_args)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/factory.py", line 160, in create_host
19:22:32 INFO | autoserv| host_instance = host_class(hostname, **args)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/base_classes.py", line 58, in __init__
19:22:32 INFO | autoserv| super(Host, self).__init__(*args, **dargs)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/base_classes.py", line 70, in __init__
19:22:32 INFO | autoserv| self._initialize(*args, **dargs)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/cros_host.py", line 306, in _initialize
19:22:32 INFO | autoserv| try_lab_servo=try_lab_servo)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 789, in create_servo_host
19:22:32 INFO | autoserv| required_by_test=required_by_test)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/base_classes.py", line 58, in __init__
19:22:32 INFO | autoserv| super(Host, self).__init__(*args, **dargs)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/hosts/base_classes.py", line 70, in __init__
19:22:32 INFO | autoserv| self._initialize(*args, **dargs)
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 173, in _initialize
19:22:32 INFO | autoserv| self.repair()
19:22:32 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/servo_host.py", line 708, in repair
19:22:32 INFO | autoserv| '\n'.join(errors))
19:22:32 INFO | autoserv| ServoHostRepairTotalFailure: All attempts at repairing the servo failed:
19:22:32 INFO | autoserv| [Errno 111] Connection refused
19:22:32 INFO | autoserv| 1
19:22:32 INFO | autoserv| Traceback (most recent call last):
19:22:32 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 769, in main
19:22:32 INFO | autoserv| File "../third_party/autotest/files/server/autoserv", line 561, in run_autoserv
19:22:32 INFO | autoserv| SystemExit: 1
19:22:32 INFO | autoserv| record_state_duration failed: job_or_task_id=None, hostname=android1758-row4-rack3-host2.cros, status=Running
-----------------------------------------------------------------------------
/tmp/test_that_results_bMB7TC/results-1-platform_InstallTestImage [  FAILED  ]
-----------------------------------------------------------------------------
Total PASS: 0/1 (0%)
Cc: krisr@chromium.org aashuto...@chromium.org
 Issue 601491  has been merged into this issue.
Status: Fixed (was: Assigned)
Lars DVT has been running test successfully. Closing this. 
https://wmatrix.googleplex.com/platform/wifi_matfunc?platforms=lars&days_back=10&releases=53
Labels: VerifyIn-53
Status: Verified (was: Fixed)

Sign in to add a comment