New issue
Advanced search Search tips

Issue 853337 link

Starred by 5 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2018
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

When running tests in lab phantom DUT is being found

Project Member Reported by haddowk@chromium.org, Jun 15 2018

Issue description


There are only 2 DUT's connected to each moblab in the lab, however it detected 3 in this case.

https://screenshot.googleplex.com/SfBSa0GJpWT

Make the DUT detection more robust so we never get phantom DUT.

Phantom DUT can lead to issues like 06/15 09:16:40.241 WARNI|              test:0637| The test failed with the following exception
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/test.py", line 631, in _exec
    _call_test_function(self.execute, *p_args, **p_dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 837, in _call_test_function
    raise error.UnhandledTestFail(e)
UnhandledTestFail: Unhandled KeyError: 'board'
Traceback (most recent call last):
  File "/usr/local/autotest/client/common_lib/test.py", line 831, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 495, in execute
    dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 362, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/client/common_lib/test.py", line 400, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/server/site_tests/moblab_RunSuite/moblab_RunSuite.py", line 61, in run_once
    board = labels['board']
  File "/usr/local/autotest/utils/labellib.py", line 73, in __getitem__
    return self._keyval_map[key]
KeyError: 'board'

As the phantom DUT has no labels if it is selected to get the "board" label it will crash the test.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b0648b713966f36052079a410ea41b1a5f3e353f

commit b0648b713966f36052079a410ea41b1a5f3e353f
Author: Keith Haddow <haddowk@chromium.org>
Date: Tue Jun 19 04:14:28 2018

[autotest] Do a better job of detecting DUT's in moblab host

Change the command that attempts to find the DUT's connected
to a moblab, and make it so that device has to be reachable
before it will be added to autotest.

BUG= chromium:853337 
TEST=None

Change-Id: I46149f8e81b85c9c80204edfcb715adca3a2e53a
Reviewed-on: https://chromium-review.googlesource.com/1103187
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Matt Mallett <mattmallett@chromium.org>

[modify] https://crrev.com/b0648b713966f36052079a410ea41b1a5f3e353f/server/hosts/moblab_host.py

Project Member

Comment 2 by bugdroid1@chromium.org, Jun 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/2aadf6e7775a838a9bb0aebc790fc65671292c39

commit 2aadf6e7775a838a9bb0aebc790fc65671292c39
Author: Keith Haddow <haddowk@chromium.org>
Date: Wed Jun 20 08:43:23 2018

[autotest] Fix host check to allow vm DUT's to be found.

Stricter checks have made the moblab VM flaky, relax the check
a little

BUG= chromium:853337 
TEST=manual testing

Change-Id: I783fb498ea152952dfa971e76975ab05c631eb8c
Reviewed-on: https://chromium-review.googlesource.com/1106056
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>

[modify] https://crrev.com/2aadf6e7775a838a9bb0aebc790fc65671292c39/server/hosts/moblab_host.py

Project Member

Comment 3 by bugdroid1@chromium.org, Jun 22 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c8d7d6a6430a6d6f5ae0d385f6ac8749e0fb5416

commit c8d7d6a6430a6d6f5ae0d385f6ac8749e0fb5416
Author: Keith Haddow <haddowk@chromium.org>
Date: Fri Jun 22 18:37:52 2018

[autotest] Use fping for moblab DUT discovery.

arp or ip n commands seem to be unreliable at times, showing
phantom IP's when using cisco managed router in the lab but showing
IP addresses as DELAY rather than REACHABLE in the VM.

Switch to a new strategy, just ping the devices and include devices
that have a 0% ping loss.

The change drops detection of the mac address, there is no known
need for it.  Also moblab should always have fping installed it is
in the moblab ebuild as an RDEPEND.

Tryjobs passed guado_moblab-paladin,
https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Try/b8943249388377048672

moblab-generic-vm-paladin
https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Try/b8943249380296023168

BUG= chromium:853337 
TEST=trybot

Change-Id: I96bdf3c70b42081225bbbff69b519216ce166fd9
Reviewed-on: https://chromium-review.googlesource.com/1107238
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>

[modify] https://crrev.com/c8d7d6a6430a6d6f5ae0d385f6ac8749e0fb5416/server/hosts/moblab_host.py

Status: Fixed (was: Started)

Sign in to add a comment