New issue
Advanced search Search tips

Issue 751168 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Sep 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug



Sign in to add a comment

Canary build AUTEST fail: autoupdate_EndToEndTest_npo_delta_9792.0.0

Project Member Reported by jkwang@google.com, Aug 1 2017 
This happens a few times:
07/30 04:35:19.241 ERROR|             utils:0280| [stderr] mux_client_request_session: read from master failed: Broken pipe
07/30 04:35:24.952 ERROR|             utils:0280| [stderr] 2017-07-30 04:35:24 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/client-autotest.tar.bz2 [11925045/11925045] -> "/usr/local/autotest/packages/client-autotest.tar.bz2" [1]
07/30 04:35:27.264 ERROR|             utils:0280| [stderr] 2017-07-30 04:35:26 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/packages.checksum [39039/39039] -> "/usr/local/autotest/packages.checksum" [1]
07/30 04:35:29.132 ERROR|             utils:0280| [stderr] cat: /usr/local/autotest/.checksum: No such file or directory
07/30 04:35:54.127 ERROR|             utils:0280| [stderr] bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:35:54.168 ERROR|            runner:0118| Non-critical failure: Failed to create directory summary for /usr/local/autotest/results/default.
Traceback (most recent call last):
  File "/usr/local/autotest/client/bin/result_tools/runner.py", line 112, in run_on_client
    timeout=_BUILD_DIR_SUMMARY_TIMEOUT)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 458, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_cnXkXussh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row10-rack6-host5 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::run_on_client|run|wrapper] ->
    ssh_run(/usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default )\";fi;
    /usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default "
Exit status: 127
Duration: 0.296881914139

stderr:
bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:35:59.938 ERROR|             utils:0280| [stderr] bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:35:59.979 ERROR|            runner:0118| Non-critical failure: Failed to create directory summary for /usr/local/autotest/results/default.
Traceback (most recent call last):
  File "/usr/local/autotest/client/bin/result_tools/runner.py", line 112, in run_on_client
    timeout=_BUILD_DIR_SUMMARY_TIMEOUT)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 458, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_cnXkXussh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row10-rack6-host5 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::run_on_client|run|wrapper] ->
    ssh_run(/usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default )\";fi;
    /usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default "
Exit status: 127
Duration: 0.296575069427

stderr:
bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:40:40.692 ERROR|             utils:0280| [stderr] 2017-07-30 04:40:40 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/client-autotest.tar.bz2 [11925045/11925045] -> "/usr/local/autotest/packages/client-autotest.tar.bz2" [1]
07/30 04:40:43.570 ERROR|             utils:0280| [stderr] 2017-07-30 04:40:43 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/packages.checksum [39039/39039] -> "/usr/local/autotest/packages.checksum" [1]
07/30 04:40:45.706 ERROR|             utils:0280| [stderr] cat: /usr/local/autotest/.checksum: No such file or directory
07/30 04:41:09.522 ERROR|             utils:0280| [stderr] bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:41:09.566 ERROR|            runner:0118| Non-critical failure: Failed to create directory summary for /usr/local/autotest/results/default.
Traceback (most recent call last):
  File "/usr/local/autotest/client/bin/result_tools/runner.py", line 112, in run_on_client
    timeout=_BUILD_DIR_SUMMARY_TIMEOUT)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 458, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_XZ3U5_ssh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row10-rack6-host5 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::run_on_client|run|wrapper] ->
    ssh_run(/usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default )\";fi;
    /usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default "
Exit status: 127
Duration: 0.313153982162

stderr:
bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:41:14.599 ERROR|             utils:0280| [stderr] bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:41:14.639 ERROR|            runner:0118| Non-critical failure: Failed to create directory summary for /usr/local/autotest/results/default.
Traceback (most recent call last):
  File "/usr/local/autotest/client/bin/result_tools/runner.py", line 112, in run_on_client
    timeout=_BUILD_DIR_SUMMARY_TIMEOUT)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 458, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_XZ3U5_ssh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row10-rack6-host5 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::run_on_client|run|wrapper] ->
    ssh_run(/usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default )\";fi;
    /usr/local/autotest/result_tools/utils.py -p
    /usr/local/autotest/results/default "
Exit status: 127
Duration: 0.295881986618

stderr:
bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:41:23.962 ERROR|             utils:0280| [stderr] 2017-07-30 04:41:23 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/client-autotest.tar.bz2 [11925045/11925045] -> "/tmp/sysinfo/autoserv-D0Y09k/packages/client-autotest.tar.bz2" [1]
07/30 04:41:26.233 ERROR|             utils:0280| [stderr] 2017-07-30 04:41:25 URL:http://100.115.219.133:8082/static/auron_yuna-release/R61-9765.11.0/autotest/packages/packages.checksum [39039/39039] -> "/tmp/sysinfo/autoserv-D0Y09k/packages.checksum" [1]
07/30 04:41:28.094 ERROR|             utils:0280| [stderr] cat: /tmp/sysinfo/autoserv-D0Y09k/.checksum: No such file or directory
07/30 04:41:43.607 ERROR|             utils:0280| [stderr] bash: /usr/local/autotest/result_tools/utils.py: No such file or directory
07/30 04:41:43.649 ERROR|            runner:0118| Non-critical failure: Failed to create directory summary for /tmp/sysinfo/autoserv-D0Y09k/results/default.
Traceback (most recent call last):
  File "/usr/local/autotest/client/bin/result_tools/runner.py", line 112, in run_on_client
    timeout=_BUILD_DIR_SUMMARY_TIMEOUT)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 316, in run
    return self.run_very_slowly(*args, **kwargs)
  File "/usr/local/autotest/site-packages/chromite/lib/metrics.py", line 458, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 307, in run_very_slowly
    ssh_failure_retry_ok)
  File "/usr/local/autotest/server/hosts/ssh_host.py", line 258, in _run
    raise error.AutoservRunError("command execution error", result)
AutoservRunError: command execution error
* Command: 
    /usr/bin/ssh -a -x    -o ControlPath=/tmp/_autotmp_XZ3U5_ssh-
    master/socket -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
    -o BatchMode=yes -o ConnectTimeout=30 -o ServerAliveInterval=900 -o
    ServerAliveCountMax=3 -o ConnectionAttempts=4 -o Protocol=2 -l root -p 22
    chromeos4-row10-rack6-host5 "export LIBC_FATAL_STDERR_=1; if type
    \"logger\" > /dev/null 2>&1; then logger -tag \"autotest\"
    \"server[stack::run_on_client|run|wrapper] ->
    ssh_run(/usr/local/autotest/result_tools/utils.py -p /tmp/sysinfo
    /autoserv-D0Y09k/results/default )\";fi;
    /usr/local/autotest/result_tools/utils.py -p /tmp/sysinfo/autoserv-
    D0Y09k/results/default "
Exit status: 127
Duration: 0.301286935806

stderr:
bash: /usr/local/autotest/result_tools/utils.py: No such file or directory

Comment 1 by dhadd...@chromium.org, Aug 1 2017

Components: Internals>Installer
Labels: OS-Chrome
Example builder:
https://uberchromegw.corp.google.com/i/chromeos/builders/celes-release/builds/1357

The test actually passes:
08/01 05:03:18.798 INFO |autoupdate_EndToEn:1175| Target active slot changed as expected: /dev/mmcblk0p3
08/01 05:03:18.799 INFO |autoupdate_EndToEn:1177| Update successful, test completed

But it fails because autotest wants to do that ssh call and the utils.py file doesn't exist

Comment 2 by dhadd...@chromium.org, Aug 1 2017 

Seems to be something unrelated to the test

Comment 3 by dhadd...@chromium.org, Sep 18 2017

Status: WontFix (was: Untriaged)
This is obsolete now that the AUTest stage has been removed 

For details see   issue 764038   and CL https://chromium-review.googlesource.com/#/c/chromiumos/chromite/+/671685/
►

Sign in to add a comment