New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 714225 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

x86-zgb-paladin: tests were scheduled but aborted

Project Member Reported by nxia@chromium.org, Apr 21 2017

Issue description

https://luci-milo.appspot.com/buildbot/chromeos/x86-zgb-paladin/9669

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/113674399-chromeos-test/chromeos6-row2-rack6-host14/


provision_AutoUpdate.double


START	provision_AutoUpdate.double	provision_AutoUpdate.double	timestamp=1492790263	localtime=Apr 21 08:57:43	
INFO	----	----	Job aborted by autotest_system on 2017-04-21 08:58:04


The test was scheduled at 08:57 but got aborted at 8:58, didn't find meaningful info logs.




 
the log files in debug folder shows a ssh failure and log parse failure.

04/21 08:57:43.395 INFO |        server_job:0184| START	provision_AutoUpdate.double	provision_AutoUpdate.double	timestamp=1492790263	localtime=Apr 21 08:57:43	
04/21 08:57:58.094 INFO |          autotest:0247| Installing autotest on chromeos6-row2-rack6-host14
04/21 08:57:58.095 INFO |          autotest:0252| Using installation dir /tmp/sysinfo/autoserv-qa7yaf
04/21 08:58:08.313 INFO |     site_autotest:0050| Get job repo url from host attributes: http://100.115.185.228:8082/static/x86-zgb-paladin/R60-9482.0.0-rc1/autotest/packages
04/21 08:58:16.204 INFO |          autoserv:0687| Results placed in /usr/local/autotest/results/113674399-chromeos-test/chromeos6-row2-rack6-host14
04/21 08:58:16.204 INFO |           pidfile:0016| Logged pid 29476 to /usr/local/autotest/results/113674399-chromeos-test/chromeos6-row2-rack6-host14/.collect_crashinfo_execute
04/21 08:58:16.285 INFO |    connectionpool:0188| Starting new HTTP connection (1): metadata.google.internal
04/21 08:58:16.831 NOTIC|      cros_logging:0037| ts_mon was set up.
04/21 08:58:18.251 INFO |        server_job:0719| I am PID 29476
04/21 08:58:18.567 INFO |      abstract_ssh:0809| Starting master ssh connection '/usr/bin/ssh -a -x -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_iMDLdpssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/tmpludo2Z -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=300 -l root -p 22 chromeos6-row2-rack6-host14'
04/21 08:58:23.707 INFO |      abstract_ssh:0824| Timed out waiting for master-ssh connection to be established.
04/21 08:58:31.110 INFO |      abstract_ssh:0809| Starting master ssh connection '/usr/bin/ssh -a -x   -N -o ControlMaster=yes -o ControlPath=/tmp/_autotmp_TZhUTUssh-master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22 chromeos6-row2-rack6-host14'
04/21 08:58:36.246 INFO |      abstract_ssh:0824| Timed out waiting for master-ssh connection to be established.
04/21 08:59:27.884 INFO | get_network_stats:0048| get_network_stats: at-start RXbytes 130197239 TXbytes 6763795
04/21 08:59:27.888 ERROR|   logging_manager:0626| tko parser: {'aborted_by': 'autotest_system', 'job_started': 1492789882, 'offload_failures_only': 'True', 'parent_job_id': 113662534, 'user': 'chromeos-test', 'builds': "{'cros-version': 'x86-zgb-paladin/R60-9482.0.0-rc1'}", 'aborted_on': 1492790284, 'job_finished': 1492790284, 'hostname': 'chromeos6-row2-rack6-host14', 'status_version': 1, 'label': 'x86-zgb-paladin/R60-9482.0.0-rc1/bvt-inline/provision_AutoUpdate.double', 'drone': 'chromeos-server108.mtv.corp.google.com', 'build': 'x86-zgb-paladin/R60-9482.0.0-rc1', 'suite': 'bvt-inline', 'retry_original_job_id': 113663066, 'experimental': 'False', 'job_queued': 1492789400}
04/21 08:59:27.890 ERROR|   logging_manager:0626| tko parser: MACHINE NAME: chromeos6-row2-rack6-host14
04/21 08:59:27.891 ERROR|   logging_manager:0626| tko parser: MACHINE GROUP: x86-zgb
04/21 08:59:27.891 ERROR|   logging_manager:0626| tko parser: parsing partial test ---- SERVER_JOB
04/21 08:59:27.892 ERROR|   logging_manager:0626| tko parser: parsing partial test provision_AutoUpdate.double provision_AutoUpdate.double
04/21 08:59:27.893 ERROR|   logging_manager:0626| tko parser: RUNNING: RUNNING
04/21 08:59:27.893 ERROR|   logging_manager:0626| Subdir: provision_AutoUpdate.double
04/21 08:59:27.893 ERROR|   logging_manager:0626| Testname: provision_AutoUpdate.double
04/21 08:59:27.894 ERROR|   logging_manager:0626| 
04/21 08:59:27.894 ERROR|   logging_manager:0626| tko parser: Unexpected indent: aborting log parse
04/21 08:59:27.895 ERROR|   logging_manager:0626| tko parser: parsing test provision_AutoUpdate.double provision_AutoUpdate.double
04/21 08:59:27.895 ERROR|   logging_manager:0626| tko parser: ADD: ABORT
04/21 08:59:27.895 ERROR|   logging_manager:0626| Subdir: provision_AutoUpdate.double
04/21 08:59:27.896 ERROR|   logging_manager:0626| Testname: provision_AutoUpdate.double
04/21 08:59:27.896 ERROR|   logging_manager:0626| None
04/21 08:59:27.897 ERROR|   logging_manager:0626| tko parser: parsing test ---- SERVER_JOB

Comment 2 by nxia@chromium.org, Apr 21 2017

SSH connection might be the cause, but the connection failure log happened after  08:58:04 (when the test was marked as aborted)

Comment 3 by nxia@chromium.org, Apr 21 2017

I guess the lab's still experiencing some network issues, just not in a large scale.

Comment 4 by vapier@chromium.org, Oct 25 2017

Status: WontFix (was: Untriaged)
zgb is EOL now, so don't think we care anymore

Sign in to add a comment