Issue description
Description #1 (aashuto...@chromium.org, Nov 19 2018)
Description #2 (aashuto...@chromium.org, Dec 15 2018)
This test case fails consistently. I see similar failure for other perf tests, but they pass on multiple re-runs.
Traceback (most recent call last):
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 800, in _call_test_function
15:01:49 INFO | autoserv| return func(*args, **dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 464, in execute
15:01:49 INFO | autoserv| postprocess_profiled_run, args, dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 371, in _call_run_once
15:01:49 INFO | autoserv| self.run_once(*args, **dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 156, in run_once
15:01:49 INFO | autoserv| self.do_run(ap_config, session, power_save, governor)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 111, in do_run
15:01:49 INFO | autoserv| results = session.run(config)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/cros/network/netperf_session.py", line 113, in run
15:01:49 INFO | autoserv| result = runner.run(ignore_failures=self._ignore_failures)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/cros/network/netperf_runner.py", line 563, in run
15:01:49 INFO | autoserv| timeout=timeout)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/ssh_host.py", line 335, in run
15:01:49 INFO | autoserv| return self.run_very_slowly(*args, **kwargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/ssh_host.py", line 331, in run_very_slowly
15:01:49 INFO | autoserv| raise error.AutoservRunError(timeout_message, cmderr.args[1])
15:01:49 INFO | autoserv| AutoservRunError: Timeout encountered: /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_8Rqvygssh-master/socket -o Protocol=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -l root -p 22 chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1; if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\" \"server[stack::run|run|run] -> ssh_run(/usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866)\";fi; /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866"
15:01:49 INFO | autoserv| * Command:
15:01:49 INFO | autoserv| /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_8Rqvygssh-master/socket
15:01:49 INFO | autoserv| -o Protocol=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
15:01:49 INFO | autoserv| -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
15:01:49 INFO | autoserv| ServerAliveCountMax=3 -o ConnectionAttempts=4 -l root -p 22
15:01:49 INFO | autoserv| chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1; if type
15:01:49 INFO | autoserv| \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
15:01:49 INFO | autoserv| \"server[stack::run|run|run] -> ssh_run(/usr/local/bin/netperf -H
15:01:49 INFO | autoserv| 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866)\";fi;
15:01:49 INFO | autoserv| /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P
15:01:49 INFO | autoserv| 0,12866"
15:01:49 INFO | autoserv| Exit status: 255
15:01:49 INFO | autoserv| Duration: 70.0761048794
This test case fails consistently. I see similar failure for other perf tests, but they pass on multiple re-runs.
Traceback (most recent call last):
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 800, in _call_test_function
15:01:49 INFO | autoserv| return func(*args, **dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 464, in execute
15:01:49 INFO | autoserv| postprocess_profiled_run, args, dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/client/common_lib/test.py", line 371, in _call_run_once
15:01:49 INFO | autoserv| self.run_once(*args, **dargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 156, in run_once
15:01:49 INFO | autoserv| self.do_run(ap_config, session, power_save, governor)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 111, in do_run
15:01:49 INFO | autoserv| results = session.run(config)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/cros/network/netperf_session.py", line 113, in run
15:01:49 INFO | autoserv| result = runner.run(ignore_failures=self._ignore_failures)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/cros/network/netperf_runner.py", line 563, in run
15:01:49 INFO | autoserv| timeout=timeout)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/ssh_host.py", line 335, in run
15:01:49 INFO | autoserv| return self.run_very_slowly(*args, **kwargs)
15:01:49 INFO | autoserv| File "/mnt/host/source/src/third_party/autotest/files/server/hosts/ssh_host.py", line 331, in run_very_slowly
15:01:49 INFO | autoserv| raise error.AutoservRunError(timeout_message, cmderr.args[1])
15:01:49 INFO | autoserv| AutoservRunError: Timeout encountered: /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_8Rqvygssh-master/socket -o Protocol=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -l root -p 22 chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1; if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\" \"server[stack::run|run|run] -> ssh_run(/usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866)\";fi; /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866"
15:01:49 INFO | autoserv| * Command:
15:01:49 INFO | autoserv| /usr/bin/ssh -a -x -o ControlPath=/tmp/_autotmp_8Rqvygssh-master/socket
15:01:49 INFO | autoserv| -o Protocol=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
15:01:49 INFO | autoserv| -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
15:01:49 INFO | autoserv| ServerAliveCountMax=3 -o ConnectionAttempts=4 -l root -p 22
15:01:49 INFO | autoserv| chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1; if type
15:01:49 INFO | autoserv| \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
15:01:49 INFO | autoserv| \"server[stack::run|run|run] -> ssh_run(/usr/local/bin/netperf -H
15:01:49 INFO | autoserv| 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P 0,12866)\";fi;
15:01:49 INFO | autoserv| /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 10 -- -P
15:01:49 INFO | autoserv| 0,12866"
15:01:49 INFO | autoserv| Exit status: 255
15:01:49 INFO | autoserv| Duration: 70.0761048794
SAMPLE failure for network_WiFi_Perf.vht80,
Note: The exit status is different for this failure.
Traceback (most recent call last):
File "/usr/local/autotest/client/common_lib/test.py", line 800, in _call_test_function
return func(*args, **dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 464, in execute
postprocess_profiled_run, args, dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 371, in _call_run_once
self.run_once(*args, **dargs)
File "/usr/local/autotest/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 156, in run_once
self.do_run(ap_config, session, power_save, governor)
File "/usr/local/autotest/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py", line 102, in do_run
session.warmup_stations()
File "/usr/local/autotest/server/cros/network/netperf_session.py", line 95, in warmup_stations
self.warmup_wifi_part(warmup_client=True)
File "/usr/local/autotest/server/cros/network/netperf_session.py", line 76, in warmup_wifi_part
warmup_history.append(runner.run())
File "/usr/local/autotest/server/cros/network/netperf_runner.py", line 563, in run
timeout=timeout)
File "/usr/local/autotest/server/hosts/ssh_host.py", line 335, in run
return self.run_very_slowly(*args, **kwargs)
File "/usr/local/autotest/server/hosts/ssh_host.py", line 331, in run_very_slowly
raise error.AutoservRunError(timeout_message, cmderr.args[1])
AutoservRunError: Timeout encountered: /usr/bin/ssh -a -x -o Protocol=2 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4 -l root -p 22 chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1; if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\" \"server[stack::warmup_wifi_part|run|run] -> ssh_run(/usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 2 -- -P 0,12866)\";fi; /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l 2 -- -P 0,12866"
* Command:
/usr/bin/ssh -a -x -o Protocol=2 -o StrictHostKeyChecking=no -o
UserKnownHostsFile=/dev/null -o BatchMode=yes -o ConnectTimeout=30 -o
ServerAliveInterval=900 -o ServerAliveCountMax=3 -o ConnectionAttempts=4
-l root -p 22 chromeos15-row4-rack11-host5 "export LIBC_FATAL_STDERR_=1;
if type \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
\"server[stack::warmup_wifi_part|run|run] ->
ssh_run(/usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t TCP_STREAM -l
2 -- -P 0,12866)\";fi; /usr/local/bin/netperf -H 192.168.0.254 -p 12865 -t
TCP_STREAM -l 2 -- -P 0,12866"
Exit status: 143
Duration: 62.0722532272
Comment 1 by aashuto...@chromium.org
, Dec 14