testbed_DummyTest.install_apk always failed due to error 'num of 2 DUTs are required, 0 get' |
||||||||||
Issue description
We have 2 shamu testbed setup in chromeos-autotest.hot. However, when running testing push, it always failed at testbed_DummyTest.install_apk, due to error:
2/24 15:07:41.277 DEBUG| adb_host:0397| MainThread| Command: timeout --signal=9 3601 adb -s 021ea8a6f0a6a681 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?"
02/24 15:07:41.277 DEBUG| ssh_host:0284| MainThread| Running (ssh) 'timeout --signal=9 3601 adb -s 021ea8a6f0a6a681 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?"'
02/24 15:07:41.548 DEBUG| base_utils:0280| MainThread| [stdout] hammerhead
02/24 15:07:41.549 DEBUG| base_utils:0280| MainThread| [stdout] ADB_CMD_OUTPUT:0
02/24 15:07:41.554 DEBUG| ssh_host:0284| MainThread| Running (ssh) 'pgrep -f "adb.*server"'
02/24 15:07:41.752 DEBUG| base_utils:0280| MainThread| [stdout] 6710
02/24 15:07:41.753 DEBUG| base_utils:0280| MainThread| [stdout] 11907
02/24 15:07:41.754 DEBUG| adb_host:0335| MainThread| ADB Server PID: 6710
11907
02/24 15:07:41.754 DEBUG| adb_host:0397| MainThread| Command: timeout --signal=9 3601 adb -s 0d70854b0369e155 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?"
02/24 15:07:41.755 DEBUG| ssh_host:0284| MainThread| Running (ssh) 'timeout --signal=9 3601 adb -s 0d70854b0369e155 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?"'
02/24 15:07:42.018 DEBUG| base_utils:0280| MainThread| [stdout] hammerhead
02/24 15:07:42.018 DEBUG| base_utils:0280| MainThread| [stdout] ADB_CMD_OUTPUT:0
02/24 15:07:42.024 ERROR|provision_TestbedU:0097| MainThread| Expected number of DUTs for name shamu is 2, got 0
Traceback (most recent call last):
File "/usr/local/autotest/server/site_tests/provision_TestbedUpdate/provision_TestbedUpdate.py", line 95, in run_once
host, image=value)
File "/usr/local/autotest/server/afe_utils.py", line 234, in machine_install_and_update_labels
image_name, host_attributes = host.machine_install(*args, **dargs)
File "/usr/local/autotest/server/hosts/testbed.py", line 354, in machine_install
serial_build_map = self.locate_devices(images)
File "/usr/local/autotest/server/hosts/testbed.py", line 269, in locate_devices
(name, len(builds), len(duts) if duts else 0))
InstallError: Expected number of DUTs for name shamu is 2, got 0
02/24 15:07:42.026 DEBUG| test:0389| MainThread| Test failed due to Expected number of DUTs for name shamu is 2, got 0. Exception log follows the after_iteration_hooks.
02/24 15:07:42.026 DEBUG| test:0392| MainThread| starting after_iteration_hooks
02/24 15:07:42.026 DEBUG| test:0066| MainThread| host client install not supported, skipping after_iteration_hook:
02/24 15:07:42.026 DEBUG| test:0395| MainThread| after_iteration_hooks completed
02/24 15:07:42.027 WARNI| test:0615| MainThread| The test failed with the following exception
Traceback (most recent call last):
File "/usr/local/autotest/client/common_lib/test.py", line 609, in _exec
_call_test_function(self.execute, *p_args, **p_dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 817, in _call_test_function
return func(*args, **dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 470, in execute
dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 347, in _call_run_once_with_retry
postprocess_profiled_run, args, dargs)
File "/usr/local/autotest/client/common_lib/test.py", line 380, in _call_run_once
self.run_once(*args, **dargs)
File "/usr/local/autotest/server/site_tests/provision_TestbedUpdate/provision_TestbedUpdate.py", line 98, in run_once
raise error.TestFail(str(e))
TestFail: Expected number of DUTs for name shamu is 2, got 0
02/24 15:07:42.028 DEBUG| test:0620| MainThread| Running cleanup for test.
02/24 15:07:42.029 DEBUG| logging_manager:0627| Logging subprocess finished
02/24 15:07:42.038 DEBUG| logging_manager:0627| Logging subprocess finished
02/24 15:07:42.048 DEBUG| test:0066| MainThread| host client install not supported, skipping after_hook:
02/24 15:07:42.049 INFO | server_job:0184| MainThread| FAIL provision_TestbedUpdate provision_TestbedUpdate timestamp=1487977662 localtime=Feb 24 15:07:42 Expected number of DUTs for name shamu is 2, got 0
02/24 15:07:42.050 INFO | server_job:0184| MainThread| END FAIL provision_TestbedUpdate provision_TestbedUpdate timestamp=1487977662 localtime=Feb 24 15:07:42
02/24 15:07:42.050 ERROR| control:0075| MainThread|
Traceback (most recent call last):
File "/usr/local/autotest/results/hosts/android1758-row1-rack7-test-station-5.cros/11188-provision/control.srv", line 51, in provision_machine
provision.Provision)
File "/usr/local/autotest/server/cros/provision.py", line 370, in run_special_task_actions
raise SpecialTaskActionException()
SpecialTaskActionException
02/24 15:07:42.051 INFO | server_job:0184| MainThread| END FAIL ---- provision timestamp=1487977662 localtime=Feb 24 15:07:42
02/24 15:07:42.051 ERROR| server_job:0809| MainThread| Exception escaped control file, job aborting:
Traceback (most recent call last):
File "/usr/local/autotest/server/server_job.py", line 801, in run
self._execute_code(server_control_file, namespace)
File "/usr/local/autotest/server/server_job.py", line 1301, in _execute_code
execfile(code_file, namespace, namespace)
File "/usr/local/autotest/results/hosts/android1758-row1-rack7-test-station-5.cros/11188-provision/control.srv", line 109, in <module>
job.parallel_simple(provision_machine, machines)
File "/usr/local/autotest/server/server_job.py", line 625, in parallel_simple
return_results=return_results)
File "/usr/local/autotest/server/subcommand.py", line 93, in parallel_simple
function(arg)
File "/usr/local/autotest/results/hosts/android1758-row1-rack7-test-station-5.cros/11188-provision/control.srv", line 100, in provision_machine
raise Exception('')
Exception
,
Feb 24 2017
Kris, android1758-row1-rack7-test-station-5.cros and android1758-row1-rack7-test-station-7.cros are what we used to test Autotest before pushing to lab. http://chromeos-autotest.hot.corp.google.com/afe/#tab_id=hosts It seems that the devices connected to that teststation were changed from shamu to sailfish? Are these 2 testbeds used for any other tests? I think they should be dedicated to test push. Charlene, We might need to change test push to replace shamu build to sailfish.
,
Feb 24 2017
duts connected to testbed android1758-row1-rack7-test-station-7.cros are both flo. root@android1758-row1-rack7-ubuntu-server:~# adb -s 06da7cd6 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?" flo ADB_CMD_OUTPUT:0 root@android1758-row1-rack7-ubuntu-server:~# adb -s 08f0a1c0 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?" flo ADB_CMD_OUTPUT:0 We need to change them to sailfish to be consistent. I will lock -7 for now.
,
Feb 24 2017
I replaced the sailfish DUTs for shamu DUTs
,
Feb 25 2017
Hmm, it seems the two devices I added into ndroid1758-row1-rack7-test-station-5.cros just disappeared. root@android1758-row1-rack7-ubuntu-server:~# adb devices -l List of devices attached 00beba72386ea497 device usb:4-1.3.1 product:bullhead model:Nexus_5X device:bullhead 38007A98 device usb:4-1.3.3 product:fugu model:Nexus_Player device:fugu 00ba1969e2909823 device usb:5-1.4.2 product:bullhead model:Nexus_5X device:bullhead 16EED4D2 device usb:5-1.1 product:fugu model:Nexus_Player device:fugu 08f0a1c0 device usb:7-1.4.2 product:razor model:Nexus_7 device:flo 06da7cd6 device usb:7-1.4.4 product:razor model:Nexus_7 device:flo ZX1G22LCV5 device usb:8-1.4.3 HT6540300178 device usb:9-1.3 product:sailfish model:Pixel device:sailfish HT6620300052 device usb:9-1.4.3 product:sailfish model:Pixel device:sailfish 021ea8a6f0a6a681 device usb:19-1.4.2 product:hammerhead model:Nexus_5 device:hammerhead 0d70854b0369e155 device usb:19-1.4.4 product:hammerhead model:Nexus_5 device:hammerhead Can we lock these two devices just used for ChromeOS infra team for testing push? Testing push have been failed for this kind of failures several times before. Every time I need to manually add two new shamu devices to the testing push master. And, dshi@, how can I find a good sailfish build I can use?
,
Feb 27 2017
ping
,
Feb 27 2017
This is blocking push to prod, which is blocking resolution of CQ issues.
,
Feb 27 2017
,
Feb 27 2017
,
Feb 27 2017
The team will: - Verify the serial numbers for android1758-row1-rack7-test-station-4 - Install new devices and re-create android1758-row1-rack7-test-station-5 - android1758-row1-rack7-test-station-4 and 5 will be dedicated to lab push
,
Feb 27 2017
dschimmels, krisr, Do we know why shamu were removed from these two testbeds? Refer to comment #2 meanwhile, we will disable testbed test in test push for now: https://chromium-review.googlesource.com/#/c/447116/
,
Feb 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff92802456f39d7b939b841318021d65e38f3ff6 commit ff92802456f39d7b939b841318021d65e38f3ff6 Author: Dan Shi <dshi@google.com> Date: Mon Feb 27 18:54:31 2017 [autotest] Disable testbed in test push before testbed is restored BUG= chromium:696061 TEST=unittest Change-Id: Ida6a8a0a285bb0e3422d527cb57009cfb44c5d5a Reviewed-on: https://chromium-review.googlesource.com/447116 Commit-Queue: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Shuqian Zhao <shuqianz@chromium.org> [modify] https://crrev.com/ff92802456f39d7b939b841318021d65e38f3ff6/site_utils/test_push.py
,
Feb 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff92802456f39d7b939b841318021d65e38f3ff6 commit ff92802456f39d7b939b841318021d65e38f3ff6 Author: Dan Shi <dshi@google.com> Date: Mon Feb 27 18:54:31 2017 [autotest] Disable testbed in test push before testbed is restored BUG= chromium:696061 TEST=unittest Change-Id: Ida6a8a0a285bb0e3422d527cb57009cfb44c5d5a Reviewed-on: https://chromium-review.googlesource.com/447116 Commit-Queue: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Shuqian Zhao <shuqianz@chromium.org> [modify] https://crrev.com/ff92802456f39d7b939b841318021d65e38f3ff6/site_utils/test_push.py
,
Feb 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff92802456f39d7b939b841318021d65e38f3ff6 commit ff92802456f39d7b939b841318021d65e38f3ff6 Author: Dan Shi <dshi@google.com> Date: Mon Feb 27 18:54:31 2017 [autotest] Disable testbed in test push before testbed is restored BUG= chromium:696061 TEST=unittest Change-Id: Ida6a8a0a285bb0e3422d527cb57009cfb44c5d5a Reviewed-on: https://chromium-review.googlesource.com/447116 Commit-Queue: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Shuqian Zhao <shuqianz@chromium.org> [modify] https://crrev.com/ff92802456f39d7b939b841318021d65e38f3ff6/site_utils/test_push.py
,
Feb 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff92802456f39d7b939b841318021d65e38f3ff6 commit ff92802456f39d7b939b841318021d65e38f3ff6 Author: Dan Shi <dshi@google.com> Date: Mon Feb 27 18:54:31 2017 [autotest] Disable testbed in test push before testbed is restored BUG= chromium:696061 TEST=unittest Change-Id: Ida6a8a0a285bb0e3422d527cb57009cfb44c5d5a Reviewed-on: https://chromium-review.googlesource.com/447116 Commit-Queue: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Shuqian Zhao <shuqianz@chromium.org> [modify] https://crrev.com/ff92802456f39d7b939b841318021d65e38f3ff6/site_utils/test_push.py
,
Feb 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/ff92802456f39d7b939b841318021d65e38f3ff6 commit ff92802456f39d7b939b841318021d65e38f3ff6 Author: Dan Shi <dshi@google.com> Date: Mon Feb 27 18:54:31 2017 [autotest] Disable testbed in test push before testbed is restored BUG= chromium:696061 TEST=unittest Change-Id: Ida6a8a0a285bb0e3422d527cb57009cfb44c5d5a Reviewed-on: https://chromium-review.googlesource.com/447116 Commit-Queue: Dan Shi <dshi@google.com> Tested-by: Dan Shi <dshi@google.com> Reviewed-by: Shuqian Zhao <shuqianz@chromium.org> [modify] https://crrev.com/ff92802456f39d7b939b841318021d65e38f3ff6/site_utils/test_push.py
,
Feb 27 2017
@comment 11, I am not sure why those devices were removed. David and I will look into it.
,
Feb 27 2017
Verify the serial numbers for android1758-row1-rack7-test-station-4 - Done Host android1758-row1-rack7-test-station-4 Reverify Repair Platform: testbed Other labels: camera-hal, os:android, board:shamu-1, board:shamu-2 ACLs: "Everyone" Status: Ready - Install new devices and re-create android1758-row1-rack7-test-station-5 - Done Host android1758-row1-rack7-test-station-5 Reverify Repair Platform: testbed Other labels: camera-hal, board:angler-1, os:android, board:angler-2 ACLs: "Everyone" Status: Ready - android1758-row1-rack7-test-station-4 and 5 will be dedicated to lab push - in regards to this what labels should we add to dedicate these 2 test stations to lab push?
,
Mar 1 2017
Latest test_push fails android test: http://chromeos-autotest.hot.corp.google.com/afe/#tab_id=view_job&object_id=6115 03/01 12:18:35.786 DEBUG| ssh_host:0284| MainThread| Running (ssh) 'timeout --signal=9 3601 adb -s 84B7N16625000002 shell "getprop ro.product.device; echo ADB_CMD_OUTPUT:\$?"' 03/01 12:18:36.047 DEBUG| base_utils:0280| MainThread| [stdout] angler 03/01 12:18:36.052 DEBUG| base_utils:0280| MainThread| [stdout] ADB_CMD_OUTPUT:0 03/01 12:18:36.055 ERROR|provision_TestbedU:0097| MainThread| Expected number of DUTs for name sailfish is 2, got 0 Traceback (most recent call last): File "/usr/local/autotest/server/site_tests/provision_TestbedUpdate/provision_TestbedUpdate.py", line 95, in run_once host, image=value) File "/usr/local/autotest/server/afe_utils.py", line 234, in machine_install_and_update_labels image_name, host_attributes = host.machine_install(*args, **dargs) File "/usr/local/autotest/server/hosts/testbed.py", line 354, in machine_install serial_build_map = self.locate_devices(images) File "/usr/local/autotest/server/hosts/testbed.py", line 269, in locate_devices (name, len(builds), len(duts) if duts else 0)) InstallError: Expected number of DUTs for name sailfish is 2, got 0
,
Mar 2 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/chromeos-admin/+/67a822a3179caac9515e5a4a17261ffcf7fdd1c2 commit 67a822a3179caac9515e5a4a17261ffcf7fdd1c2 Author: Dan Shi <dshi@google.com> Date: Thu Mar 02 10:45:33 2017
,
Mar 7 2017
We set up android1758-row1-rack7-test-station-4.cros and -5 to have 2 angler each for test_push test. The duts are locked in cautotest.
,
May 30 2017
,
Aug 1 2017
,
Jan 22 2018
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by dshi@chromium.org
, Feb 24 2017