Add 5 GHz & 802.11ac coverage for various autotests in suite:wifi_matfunc |
||||
Issue descriptionsuite:wifi_matfunc has tests that mostly use 2.4 GHz bands to test functionality. For some of these, we care about the same test on 5 GHz as well. e.g. network_WiFi_BgscanBackoff For others, 5 GHz may or may not apply, i.e. network_WiFi_OverlappingBSSScan. Using this bug to: (a) add a 5 GHz option/coverage where possible. (b) in particular, within 5 GHz, exercise 5745 MHz / 5765 MHz (channels 149 and 153) due to known issues in that frequency range on Intel SoCs (SKL, and now possibly KBL). [See internal bug b/35583398, b/77340053]. (c) track findings from trying to update/migrate these tests. I'll start with network_WiFi_OverlappingBSSScan.py -choosing WIFI_FREQUENCY as 5765 or 5180 instead of 2437 makes this test fail. - the failure comes from this snippet of code below, which was neither documented / explained nor reviewed in the original commit: def detect_client_coexistence_report(self, client_mac, instance=0): """Detects whether hostapd has logged an action frame from |client_mac| indicating information about 20/40MHz BSS coexistence. @param client_mac string the MAC address of the client to detect. @param instance int indicating which hostapd instance to query. """ coex_msg = ('nl80211: MLME event frame - hexdump(len=.*): ' '.. .. .. .. .. .. .. .. .. .. %s ' '.. .. .. .. .. .. .. .. 04 00.*48 01 ..' % ' '.join(client_mac.split(':'))) log_file = self.hostapd_instances[instance].log_file logging.error('log file is at %s.', log_file) logging.error('coex_msg is %s.', coex_msg) grep_result = self.router.run("grep -i '%s' %s" % (coex_msg, log_file), ignore_status=False) logging.error('grep result: %s', grep_result.stdout) result = self.router.run("grep -qi '%s' %s" % (coex_msg, log_file), ignore_status=True) return result.exit_status == 0 - its not clear why the test is looking for this frame, and removing this check makes the test pass on 5 GHz channels - looking at the log file where this is being search (/tmp/hostapd-test-managed0.log), gives me the following "MLME event" frames that match the client's formatted mac address ("b4 d5 bd c8 a1 6d"): 1522537922.135983: nl80211: MLME event frame - hexdump(len=30): b0 00 3c 00 30 b5 c2 34 11 76 b4 d5 bd c8 a1 6d 30 b5 c2 34 11 76 a0 02 00 00 01 00 00 00 1522537922.142169: nl80211: MLME event frame - hexdump(len=30): b0 00 00 00 b4 d5 bd c8 a1 6d 30 b5 c2 34 11 76 30 b5 c2 34 11 76 00 00 00 00 02 00 00 00 1522537922.143984: nl80211: MLME event frame - hexdump(len=119): 00 00 3c 00 30 b5 c2 34 11 76 b4 d5 bd c8 a1 6d 30 b5 c2 34 11 76 b0 02 01 00 0a 00 00 20 4f 76 65 72 6c 61 70 70 69 6e 67 42 53 53 53 63 61 6e 5f 62 5f 37 7a 66 6f 63 5f 63 68 31 35 33 01 08 0c 12 18 24 30 48 60 6c 2d 1a ef 11 17 ff ff 00 00 00 00 00 00 00 00 2c 01 01 00 00 00 00 00 00 00 00 00 00 7f 08 04 00 00 00 01 00 00 40 dd 07 00 50 f2 02 00 01 00 1522537922.156816: nl80211: MLME event frame - hexdump(len=149): 10 00 00 00 b4 d5 bd c8 a1 6d 30 b5 c2 34 11 76 30 b5 c2 34 11 76 00 00 01 00 00 00 01 c0 01 08 8c 12 98 24 b0 48 60 6c 2d 1a 0e 00 1b ff ff ff 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 4a 0e 14 00 0a 00 0a 00 c8 00 14 00 05 00 19 00 3d 16 99 07 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7f 08 05 00 00 02 00 00 00 40 5a 03 24 01 00 dd 18 00 50 f2 02 01 01 00 00 03 a4 00 00 27 a4 00 00 42 43 5e 00 62 32 2f 00 1522538034.340431: nl80211: MLME event frame - hexdump(len=26): c0 00 3c 00 30 b5 c2 34 11 76 b4 d5 bd c8 a1 6d 30 b5 c2 34 11 76 f0 02 03 00 - of these, none match the reg-exp that the test is looking for: coex_msg = ('nl80211: MLME event frame - hexdump(len=.*): ' '.. .. .. .. .. .. .. .. .. .. %s ' '.. .. .. .. .. .. .. .. 04 00.*48 01 ..' % ' '.join(client_mac.split(':'))) Questions: - what do the "04 00.*48 01" stand for here? They should be properly documented in this test. This test should also explain why the coexistence report matters. - Is this test relevant to 5 GHz in the first place? a quick search shows overlapping BSS restrictions are more stringent in 2.4 GHz.
,
Apr 1 2018
Fixed #1 here: https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/989395/ Next peeve: hostap_config in all wifi autotests uses MODE_11B by default. Revisit that and other defaults. There is an if statement to switch to MODE_11N_PURE but its not clear if its ever triggered. Fix that. def __init__(self, mode=MODE_11B, channel=None, frequency=None, n_capabilities=[], hide_ssid=None, beacon_interval=None, dtim_period=None, frag_threshold=None, ssid=None, bssid=None, force_wmm=None, security_config=None, pmf_support=PMF_SUPPORT_DISABLED, obss_interval=None, vht_channel_width=None, vht_center_channel=None, ac_capabilities=[], beacon_footer='', spectrum_mgmt_required=None, scenario_name=None, min_streams=None, nas_id=None, mdid=None, r1kh_id=None, r0kh=None, r1kh=None, bridge=None):
,
Apr 1 2018
,
Apr 2 2018
Took a stab at a fix for #2 here: https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/989409/ Current failures are here: https://paste.googleplex.com/5635432553381888?raw will continue work on fixing if team approves. Next peeve: Notice that most failures are on channel 1, because most tests are written for channel 1. A lot of coverage for a channel that has 12% of our users, little to none for the rest. See https://screenshot.googleplex.com/WRPdtSoLKde for last month's stable channel users' freq distribution. We need to add more coverage for channels 6,11,36,40(? or whatever 5220 is), 149 and 157.
,
Apr 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/76721b5581a24f1173e23dfe8714241f3a7ee159 commit 76721b5581a24f1173e23dfe8714241f3a7ee159 Author: Kirtika Ruchandani <kirtika@google.com> Date: Wed Apr 04 06:44:47 2018 network_WiFi_BgscanBackoff: Add support for 5GHz This test currently uses 2.4 GHz channels. Refactor it to make channels configurable. Keep the control file for the current configuration of channels 1 and 11. Add a new control file to test channels on 5 GHz, especially channels 149 and 153 which we care about on certain Intel SoCs. BUG=chromium:827829 TEST=This is a test, confirm that both control.wifi_bgscan_backoff and control.5760_noise_check pass on my Soraka. Change-Id: I26d52fa5f0a3363237b10da83a324584f9b632f9 Reviewed-on: https://chromium-review.googlesource.com/989395 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Harpreet Grewal <harpreet@chromium.org> [modify] https://crrev.com/76721b5581a24f1173e23dfe8714241f3a7ee159/server/site_tests/network_WiFi_BgscanBackoff/network_WiFi_BgscanBackoff.py [add] https://crrev.com/76721b5581a24f1173e23dfe8714241f3a7ee159/server/site_tests/network_WiFi_BgscanBackoff/control.5760noise_check [modify] https://crrev.com/76721b5581a24f1173e23dfe8714241f3a7ee159/server/site_tests/network_WiFi_BgscanBackoff/control.wifi_bgscan_backoff
,
Apr 6 2018
Next peeve: Chrome OS's hostap config doesn't require setting the number of spatial streams. Someone needs to understand what hostap's default is, if one doesn't set it. For 802.11n and 802.11ac, we want to go for 2x2 MIMO and we probably want to explicitly specify Rx STBC12: https://w1.fi/cgit/hostap/plain/hostapd/hostapd.conf Chrome OS autotests have a reference to "min_streams" in hostap_config, it may be obsolete.
,
Apr 7 2018
Adding VHT80 configs to network_WiFi_AttenuatedPerf and other changes based on our wifi-test sync meeting today: https://chromium-review.googlesource.com/#/c/chromiumos/third_party/autotest/+/1000963/ At this point, this bug is a kitchensink of all changes wanted/done for (a) 5 GHz support (b) 802.11ac support (c) noise at 5760 MHz testing.
,
Apr 10 2018
For the "why" about min_streams, see: https://chromium-review.googlesource.com/283621 It looks like it's not about configuring hostapd, but just about choosing the right wiphy. In general, we probably want to look for more streams, but there are cases where we don't care, so we look for fewer.
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b554398ab0d71a0f926acc2895c627fb057c7a3c commit b554398ab0d71a0f926acc2895c627fb057c7a3c Author: Brian Norris <briannorris@chromium.org> Date: Tue Apr 17 04:26:42 2018 perf dashboard: add VHT80 to perf dashboards BUG=chromium:827829 TEST=none Change-Id: I2ddbd24d3d2a8bc827a532f477dce10534216d12 Signed-off-by: Brian Norris <briannorris@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1013262 Reviewed-by: Harpreet Grewal <harpreet@chromium.org> [modify] https://crrev.com/b554398ab0d71a0f926acc2895c627fb057c7a3c/tko/perf_upload/perf_dashboard_config.json
,
Apr 17 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/860ad5164a6e1e11f4ec99a1c05ae7e021384d00 commit 860ad5164a6e1e11f4ec99a1c05ae7e021384d00 Author: Brian Norris <briannorris@chromium.org> Date: Tue Apr 17 22:53:11 2018 network_WiFi_Perf: log VHT80 correctly in keyval results Wifi perf results generate keys by using the hostap config's printable_mode() method. This method didn't handle VHT80 correctly and instead interpreted it as 11A. Add special handling for VHT. I don't currently plumb the corresponding packet_capturer parameters, because (a) our current 'iw' doesn't have a clear way of configuring this (or at least, I was too lazy for this at the moment) and (b) we should probably plumb packet capture modes (i.e., 'iw dev XXX set freq ...') differently than "printable mode" anyway. Note that this means we probably aren't capturing network_WiFi_SimpleConnect.wifi_check5VHT80 correctly. Fortunately, that's the only other test that uses VHT rates, and it doesn't actually utilize those captures for anything. BUG=chromium:827829, chromium:774831 TEST=network_WiFi_Perf.vht80 (and others); check that the results/keyval file contains results like: ch044_modeVHT80_none_PSon_tcp_tx_throughput{perf}=191.98+-0.56 instead of: ch044_mode11A_none_PSon_tcp_tx_throughput{perf}=193.73+-5.60 Change-Id: I038eeeb1a54998324458ffe9609b6ea06d4766f2 Signed-off-by: Brian Norris <briannorris@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1013261 [modify] https://crrev.com/860ad5164a6e1e11f4ec99a1c05ae7e021384d00/server/cros/network/hostap_config.py
,
May 15 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/df17b001a89486b3e2a90b89377e72aef7247ba9 commit df17b001a89486b3e2a90b89377e72aef7247ba9 Author: Brian Norris <briannorris@chromium.org> Date: Tue May 15 20:32:25 2018 perf dashboard: add WiFi VHT80 SimpleConnect stats to perf dashboard The association times for 5VHT80 were not being populated in chromeperf. Fix that. For completeness: these are also these network_WiFi_SimpleConnect results that are missing. I don't see that they are extremely important, but I could be convinced otherwise: network_WiFi_SimpleConnect.wifi_checkNonAsciiSSID network_WiFi_SimpleConnect.wifi_checkOddWPAPassphrases network_WiFi_SimpleConnect.wifi_checkRawPMK network_WiFi_SimpleConnect.wifi_checkSGI network_WiFi_SimpleConnect.wifi_checkSSIDLimits network_WiFi_SimpleConnect.wifi_checkTruncatedBeacon BUG=chromium:827829 TEST=none Change-Id: I46559a7995cf413b55a773a2c518facdc54e5b26 Signed-off-by: Brian Norris <briannorris@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/1060124 Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org> [modify] https://crrev.com/df17b001a89486b3e2a90b89377e72aef7247ba9/tko/perf_upload/perf_dashboard_config.json
,
May 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/753498b613de52e58ae9b9e01ce278d1f6a2cd30 commit 753498b613de52e58ae9b9e01ce278d1f6a2cd30 Author: Kirtika Ruchandani <kirtika@google.com> Date: Fri May 18 12:32:35 2018 autotest: wifi: Add checks to support pure VHT80 testing As of this change, autotest doesn't have tests with MODE_11AC_PURE. We'd like to start testing 802.11ac VHT80 rates exclusively. This change introduces a check to enforce that the client can support these rates. Pieces involved are: a. Change the require_vht variable in hostap_config to make it appear not-private. b. Check for require_vht in the ap config, in wifi_test_context_manager, then error out if the client does not support AC rates. c. Remove duplicate checks from existing wifi tests, since we added one in wifi_test_context_manager. network_WiFi_BluetoothStreamPerf has this check, but it currently only runs on 802.11a and 802.11b configs. Remove it. network_WiFi_Perf has this check, but its not the same. It enforces that the client support 802.11ac VHT only rates, even when the AP is asking for a "mixed" that is both 802.11ac VHT and 802.11n HT. Add a comment explaining that, and leave the check intact. d. Check the require_capabilities function logging in site_linux_system to not assume that its running on an AP. BUG=chromium:827829 TEST=Run a modified network_WiFi_SimpleConnect with VHT80 config on a client that does not support 802.11ac, and confirm that the test passes with TestNA. See the TEST section in the commit message of CL:1060648 for more details. Change-Id: I6af1a403981a134c4a6c7a0d3f29456118e89467 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1065060 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/753498b613de52e58ae9b9e01ce278d1f6a2cd30/server/cros/network/hostap_config.py [modify] https://crrev.com/753498b613de52e58ae9b9e01ce278d1f6a2cd30/server/site_tests/network_WiFi_BluetoothStreamPerf/network_WiFi_BluetoothStreamPerf.py [modify] https://crrev.com/753498b613de52e58ae9b9e01ce278d1f6a2cd30/server/cros/network/wifi_test_context_manager.py [modify] https://crrev.com/753498b613de52e58ae9b9e01ce278d1f6a2cd30/server/site_tests/network_WiFi_Perf/network_WiFi_Perf.py [modify] https://crrev.com/753498b613de52e58ae9b9e01ce278d1f6a2cd30/server/site_linux_system.py
,
May 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/de693fcc7ea4009639301e7d082d402f83c0863f commit de693fcc7ea4009639301e7d082d402f83c0863f Author: Kirtika Ruchandani <kirtika@google.com> Date: Fri May 18 17:05:21 2018 network_WiFi_SimpleConnect: Add 802.11ac 'pure' mode All the current wifi tests use MODE_11AC_MIXED on the AP side for 802.11ac tests. The mixed mode allows the test to pass even if the client doesn't support 802.11ac/vht80 rates and uses ht40 rates instead. Just to add some coverage with true VHT80 rates, add a network_WiFi_SimpleConnect variant that uses MODE_11AC_PURE. BUG=chromium:827829 TEST= 1. test_that -b cave ${DUT_IP} network_WiFi_SimpleConnect.wifi_check5VHT80_pure 2. test_that -b cave ${DUT_IP} network_WiFi_SimpleConnect.wifi_check5VHT80_mixed Run both a and b with and without 'iwlwifi.disable_11ac=Y' added to kernel command line (/usr/share/vboot/bin/make_dev_ssd.sh --edit_config --partitions 2) Run `watch -n0.1 "iw wlan0 station dump | tee -a /var/log/rate.txt"` in all cases, and check that `grep 'tx bitrate:' rate.txt` gives you "80MHz VHT-NSS" in the 'pure' case, and "40MHz" in the 802.11ac disabled case. Change-Id: Id9a6d5747073473c4f960d7ba2376f5a610d8b16 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1060648 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [copy] https://crrev.com/de693fcc7ea4009639301e7d082d402f83c0863f/server/site_tests/network_WiFi_SimpleConnect/control.wifi_check5VHT80_pure [rename] https://crrev.com/de693fcc7ea4009639301e7d082d402f83c0863f/server/site_tests/network_WiFi_SimpleConnect/control.wifi_check5VHT80_mixed [modify] https://crrev.com/de693fcc7ea4009639301e7d082d402f83c0863f/tko/perf_upload/perf_dashboard_config.json
,
May 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/877a810dc1db5dd85e008d58b221d7447a4d39b6 commit 877a810dc1db5dd85e008d58b221d7447a4d39b6 Author: Kirtika Ruchandani <kirtika@google.com> Date: Wed May 23 04:57:05 2018 autotest: wifi: fix broken CAPABILITY_VHT check Two mistakes: - 'self.client.require_capabilities', not self.context.client, since 'self' is the context. - require_capabilities expects a list BUG=chromium:827829 TEST=The below passes of grunt instead of throwing an exception - `test_that -b grunt grunt network_WiFi_SimpleConnect.wifi_check5VHT80_pure` Fixes: 753498b61 ("autotest: wifi: Add checks to support pure VHT80 testing") Change-Id: I1620ff77e02c1b98038e141fb7cc7039f43c7895 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1069413 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/877a810dc1db5dd85e008d58b221d7447a4d39b6/server/cros/network/wifi_test_context_manager.py
,
Sep 7
,
Jan 15
|
||||
►
Sign in to add a comment |
||||
Comment 1 by kirtika@chromium.org
, Mar 31 2018#2: network_WiFi_BgscanBackoff The default HostapConfig that chrome os is using doesn't like 5 GHz channels. That's why the below patch fails the test with "Tried to set an invalid frequency: 5745". We'll need to fix this to allow any channel on 2.4/5 GHz band to be used without setup effort, and to use 802.11n/ac as default where possible. --- a/server/site_tests/network_WiFi_BgscanBackoff/network_WiFi_BgscanBackoff.py +++ b/server/site_tests/network_WiFi_BgscanBackoff/network_WiFi_BgscanBackoff.py @@ -23,6 +23,8 @@ class network_WiFi_BgscanBackoff(wifi_cell_test_base.WiFiCellTestBase): PING_INTERVAL_SECONDS = 0.1 LATENCY_MARGIN_MS = 200 THRESHOLD_BASELINE_LATENCY_MS = 100 + CHAN_FIRST_AP = 149 + CHAN_SECOND_AP = 1 def run_once(self): @@ -34,7 +36,8 @@ class network_WiFi_BgscanBackoff(wifi_cell_test_base.WiFiCellTestBase): self.context.get_wifi_addr(), interval=self.PING_INTERVAL_SECONDS, count=int(period / self.PING_INTERVAL_SECONDS)) - self.context.configure(hostap_config.HostapConfig(channel=1)) + self.context.configure( + hostap_config.HostapConfig(channel=self.CHAN_FIRST_AP)) bgscan_config = xmlrpc_datatypes.BgscanConfiguration( short_interval=self.CONFIGURED_BGSCAN_INTERVAL_SECONDS, long_interval=self.CONFIGURED_BGSCAN_INTERVAL_SECONDS, @@ -47,7 +50,7 @@ class network_WiFi_BgscanBackoff(wifi_cell_test_base.WiFiCellTestBase): logging.info('Ping statistics with bgscan: %r', result_bgscan) # Bring up a second AP, make sure that it shows up in bgscans. self.context.configure( - hostap_config.HostapConfig(channel=11, + hostap_config.HostapConfig(channel=self.CHAN_SECOND_AP, min_streams=1, ssid=self.context.router.get_ssid()), multi_interface=True) @@ -71,7 +74,8 @@ class network_WiFi_BgscanBackoff(wifi_cell_test_base.WiFiCellTestBase): self.context.client.shill.disconnect( self.context.router.get_ssid(instance=0)) # Reconfigure AP, so the new bgscan setting can be correctly applied. - self.context.configure(hostap_config.HostapConfig(channel=1)) + self.context.configure( + hostap_config.HostapConfig(channel=self.CHAN_FIRST_AP)) # Gather some statistics about ping latencies without scanning going on. self.context.assert_connect_wifi(get_assoc_params(None)) logging.info('Pinging router without background scans for %d seconds.',