autotest: ideas to do wifi/networking smoke-testing without needing a wificell |
|||
Issue descriptionMotivation: (a) not requiring a wificell frees us up to use any pool, and running on the cq pools is a huge plus. (b) the success of network_WlanDriver in bvt-cq shows that there is value in testing DUTs in cq pools (we found unsupported wifi chip configurations). (c) we need a smoke-test for networking and wificell-pre-cq hasn't worked. (d) we can reach a larger number of DUTs this way. for recent new devices, we've had to wait a long time for a device to show up in pool:wificell. (e) we'll learn new stuff while trying to test wifi without a router (like how far can you stretch a class 1 frame's info..) Tests that are easy candidates: 1. network_WlanDriver (already runs on bvt-cq) 2. test that checks that shill doesn't respawn every 10 seconds and that wpa-supplicant responds to the GetInterface dbus command. These tests check that the wifi interface is up and shill/supplicant recognize it, and nothing in the networking stack is making shill crash repeatedly. Further ideas: 3. "Class 1" 80211 frames can be sent by a station that is not authenticated to any AP. See https://en.wikipedia.org/wiki/IEEE_802.11w-2009#Classes We could use this to test tx on the DUT. Perhaps check that the DUT can send out probe requests or RTS on each channel? 4. iw commands for regulatory? 5. PCI reset related test? 6. turn off/on power to the wifi card? 7. Can we use "send out class 1 frames" idea to check that the DUT has 2 antennas connected properly?
,
Aug 23
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/fba14e2b1401f2bb171da6c05076db4b2093a0fa commit fba14e2b1401f2bb171da6c05076db4b2093a0fa Author: Kirtika Ruchandani <kirtika@google.com> Date: Thu Aug 23 14:36:52 2018 autotest-tests-shill: Add shill stability smoke-test While there, upgrade the EAPI, so that pre-submit hooks are happy. BUG=chromium:854914 TEST=None Change-Id: Ibb3bc119b0e52072cedee40f064b9aee2c491de9 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1182408 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/fba14e2b1401f2bb171da6c05076db4b2093a0fa/chromeos-base/autotest-tests-shill/autotest-tests-shill-9999.ebuild
,
Aug 23
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/90c8538deaee1879527324cf3cf7b3281d8ba95a commit 90c8538deaee1879527324cf3cf7b3281d8ba95a Author: Kirtika Ruchandani <kirtika@google.com> Date: Thu Aug 23 14:36:53 2018 autotest: wifi: Add networking smoke-test Quick check to guard against bad CLs that break shill and cause it to respawn. CQ-DEPEND=CL:1182408 BUG=chromium:854914 TEST=`test_that -b samus ${DUT_IP} network_ShillStability` Run without any activity: test passes Run while ssh-ed to DUT and 'restart shill': test fails with the sample message: "FAIL: shill pid changed: 8415 -> 8656" Change-Id: I7e29229b9ab2e522741b11134c620a2d6d12e780 Reviewed-on: https://chromium-review.googlesource.com/1158384 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [add] https://crrev.com/90c8538deaee1879527324cf3cf7b3281d8ba95a/client/site_tests/network_ShillStability/control.bvt [add] https://crrev.com/90c8538deaee1879527324cf3cf7b3281d8ba95a/client/site_tests/network_ShillStability/control [add] https://crrev.com/90c8538deaee1879527324cf3cf7b3281d8ba95a/client/site_tests/network_ShillStability/network_ShillStability.py
,
Sep 22
Hi Harpreet, network_ShillStability has been running in bvt-perbuild fine for ~3 weeks now. Could you help me with migrating to bvt-cq? https://stainless.corp.google.com/search?test=%5Enetwork%5C_ShillStability%24&exclude_non_release=true&exclude_cts=true&col=build&row=board_model&view=matrix&first_date=2018-09-16&last_date=2018-09-22 All the reds in here are "ABORT" which look like the entire suite was aborted. If I need to do something to diagnose those, will need your help. Also, do I need to put in the forgive_flaky and retries that Brian had to for network_WlanDriver to get into bvt?
,
Oct 14
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0e97b0c52aa51cb9b5cfe17e00bc574321f7beae commit 0e97b0c52aa51cb9b5cfe17e00bc574321f7beae Author: harpreet <harpreet@google.com> Date: Sun Oct 14 08:09:40 2018 network_ShillStability: Move to bvt-cq This test has been running in bvt-perbuild for over a month. During this time it is only aborted or failed (twice) due to DUT issues that had nothing to do with the test itself. Please see stainless links below: Status (everything besides Fail and Abort): go/network_ShillStability-pass Status Fail: go/network_ShillStability-fail Status Abort: go/network_ShillStability-abort BUG=chromium:854914 TEST=Running in bvt-perbuild Change-Id: I9197577547ef0ebd27ba5117bdd3426a92d3f94a Reviewed-on: https://chromium-review.googlesource.com/1278353 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Harpreet Grewal <harpreet@chromium.org> Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/0e97b0c52aa51cb9b5cfe17e00bc574321f7beae/client/site_tests/network_ShillStability/control.bvt
,
Oct 22
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/75e920467e30ae0e7d8c4489221f8884749f6c2d commit 75e920467e30ae0e7d8c4489221f8884749f6c2d Author: Kirtika Ruchandani <kirtika@google.com> Date: Mon Oct 22 01:31:10 2018 autotest-tests-shill: Add network_WiFiResume Add a new smoke test checks for wifi interface available after a suspend-resume cycle. CQ-DEPEND=CL:1189140 BUG=chromium:854914 TEST=Ran network_WiFiResume octopus (JfP2), samus (WP2), coral with QCA6174A and eve (StP2) at my desk. Change-Id: I089a39e1e7fcd36345317a181cabfd9cb3852a9f Reviewed-on: https://chromium-review.googlesource.com/1189425 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Grant Grundler <grundler@chromium.org> [modify] https://crrev.com/75e920467e30ae0e7d8c4489221f8884749f6c2d/chromeos-base/autotest-tests-shill/autotest-tests-shill-9999.ebuild
,
Oct 22
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/74349150c64cb18fb3acbdee280aca5d45805ddb commit 74349150c64cb18fb3acbdee280aca5d45805ddb Author: Kirtika Ruchandani <kirtika@google.com> Date: Mon Oct 22 01:31:10 2018 autotest: wifi: Add wifi resume smoke test. Reviving network_WiFiResume for this purpose -it was found in an old and abandoned state, without a control file. As documented in crbug.com/854914, we'd like to have wifi/networking smoke tests that detect bad CLs in the CQ/faster than suite:wifi_matfunc. This test just checks that the wifi interface (mostly wlan0 or mlan0) is availble after a suspend-resume cycle. CQ-DEPEND=CL:1189425 BUG=chromium:854914 TEST=Ran this test on octopus (JfP2), samus (WP2), coral with QCA6174A and eve (StP2) at my desk. Change-Id: I9ff76a9e5d0ce91acad9bdfc2dbee0cba8c2f066 Reviewed-on: https://chromium-review.googlesource.com/1189140 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Grant Grundler <grundler@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/74349150c64cb18fb3acbdee280aca5d45805ddb/client/site_tests/network_WiFiResume/network_WiFiResume.py [add] https://crrev.com/74349150c64cb18fb3acbdee280aca5d45805ddb/client/site_tests/network_WiFiResume/control
,
Oct 24
|
|||
►
Sign in to add a comment |
|||
Comment 1 by kirtika@google.com
, Aug 1