We currently have no networking test coverage in the bvt-inline autotest suite (expected?), and very minimal coverage in bvt-cq [1]. This has caused problems every so often, where we completely break shill or Wifi on the CQ. This normally gets caught by unsuspecting developers, who unreasonably expect basic features to be working on ToT.
We should find or develop some basic tests that can improve our chances of not breaking things too badly [2].
If we want to run this kind of test on the general CQ or PreCQ (to prevent landing completely b0rken CLs), we need to consider that we can't have dependencies on any special lab resources. This includes any given known Wifi networks (AFAIK) or a dedicated AP [3].
Possible tests include:
* ensure a Wifi device shows up and is accessible in shill
* test basic operations that don't require a particular AP (e.g., scans? other control/maintenance operations?)
* check that shill isn't repeatedly crashing while doing any of the above
* more? I'm sure we can be creative here...
We might also consider how to make sure this only runs on boards with Wifi (do we support any on the CQ without it?).
This effort should be orthogonal to any wificell-pre-cq, as that is currently opt-in, and fairly resource constrained, so it can't really fit the needs above (to prevent major breakage in ToT), unless developers are very diligent.
---
[1] I see:
network_CheckCriticalProcesses
network_DefaultProfileCreation
network_DestinationVerification
network_DefaultProfileServices ** used to be, but not since bug 609840
network_WiFiInvalidParameters
---
[2] Might derive some inspiration from previous failures:
https://bugs.chromium.org/p/chromium/issues/detail?id=698394
https://bugs.chromium.org/p/chromium/issues/detail?id=729693
Or note the fact that we don't notice that Wifi is going totally berserk until it fully crashes the machine:
https://bugs.chromium.org/p/chromium/issues/detail?id=697274
---
[3] It does appear that we provision some kind of a "ChromeOS_lab_..." Wifi network near many of the lab machines. Can this be utilized?
Comment 1 by marcuskoehler@chromium.org
, Jan 15