Move to new stable whirlwind testbed-ap build and deploy to lab |
||
Issue descriptionNow that we have a new whirlwind testbed-ap build succeeding with wireless-4.2 ( bug 702007 ), need to: 1) Produce a new build (at gs://chromeos-image-archive/trybot-whirlwind-test-ap/) using the test_ap builder (https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/test_ap/, cbuildbot --remote whirlwind-test-ap). 2) Verify build by checking wifi_matfunc results for a few DUTs. 3) Update the whirlwind stable version in network_WiFi_UpdateRouter to the new build. 4) Check that lab whirlwinds move to the new version. 5) Check for any variation in wifi test results.
,
Oct 9 2017
New build: R63-10007.0.0-b5 https://pantheon.corp.google.com/storage/browser/chromeos-image-archive/trybot-whirlwind-test-ap/R63-10007.0.0-b5/ I ran wifi_matfunc on chromeos9-row1-cart1-host1 (kefka) and chromeos9-row1-cart2-host2 (kevin). Results look good. Test details: chromeos9-row1-cart1-host1 (kefka) pass: network_WiFi_BeaconInterval.wifi_bintval network_WiFi_BgscanBackoff.wifi_bgscan_backoff network_WiFi_ChannelScanDwellTime network_WiFi_ConnectionIdentifier network_WiFi_CSADisconnect network_WiFi_DisableEnable network_WiFi_DisableRandomMACAddress network_WiFi_DisconnectClearsIP network_WiFi_DisconnectReason.ap_gone network_WiFi_DisconnectReason.deauth_client network_WiFi_DisconnectReason.disable_client_wifi network_WiFi_DisconnectReason.switch_ap network_WiFi_DTIMPeriod.wifi_DTIM_period network_WiFi_GTK network_WiFi_HiddenRemains network_WiFi_HiddenScan network_WiFi_LinkMonitorFailure network_WiFi_LowInitialBitrates network_WiFi_MalformedProbeResp network_WiFi_MaskedBSSID.wifi_masked_bssid network_WiFi_MissingBeacons network_WiFi_MultiAuth network_WiFi_OverlappingBSSScan network_WiFi_PMKSACaching network_WiFi_Powersave.wifi_ps network_WiFi_ProfileBasic network_WiFi_ProfileGUID network_WiFi_PTK network_WiFi_RandomMACAddress network_WiFi_Reassociate network_WiFi_Regulatory network_WiFi_Roam.wifi_roam1xTLS network_WiFi_Roam.wifi_roamNone network_WiFi_Roam.wifi_roamWEP network_WiFi_Roam.wifi_roamWPA network_WiFi_RoamDbus network_WiFi_ScanPerformance network_WiFi_SecChange network_WiFi_SetOptionalDhcpProperties network_WiFi_SimpleConnect.wifi_check11a network_WiFi_SimpleConnect.wifi_check11b network_WiFi_SimpleConnect.wifi_check11g network_WiFi_SimpleConnect.wifi_check1x_PEAP network_WiFi_SimpleConnect.wifi_check1x_TTLS network_WiFi_SimpleConnect.wifi_check1x_WEP network_WiFi_SimpleConnect.wifi_check1x_WPA network_WiFi_SimpleConnect.wifi_check24HT20 network_WiFi_SimpleConnect.wifi_check24HT40 network_WiFi_SimpleConnect.wifi_check5HT20 network_WiFi_SimpleConnect.wifi_check5HT40 network_WiFi_SimpleConnect.wifi_check5VHT80 network_WiFi_SimpleConnect.wifi_checkDFS network_WiFi_SimpleConnect.wifi_checkHidden network_WiFi_SimpleConnect.wifi_checkHiddenWEP network_WiFi_SimpleConnect.wifi_checkHiddenWPA network_WiFi_SimpleConnect.wifi_checkMixedWPA network_WiFi_SimpleConnect.wifi_checkNonAsciiSSID network_WiFi_SimpleConnect.wifi_checkOddWPAPassphrases network_WiFi_SimpleConnect.wifi_checkRawPMK network_WiFi_SimpleConnect.wifi_checkSSIDLimits network_WiFi_SimpleConnect.wifi_checkTruncatedBeacon network_WiFi_SimpleConnect.wifi_checkWEP104 network_WiFi_SimpleConnect.wifi_checkWEP40 network_WiFi_SimpleConnect.wifi_checkWPA2 network_WiFi_SimpleConnect.wifi_checkWPA2_CCMP_PMF network_WiFi_SimpleConnect.wifi_checkWPA2_TKIP network_WiFi_SimpleConnect.wifi_checkWPA_CCMP network_WiFi_SimpleConnect.wifi_checkWPA_multi network_WiFi_SimpleConnect.wifi_checkWPA_TKIP network_WiFi_SuspendStress.11a network_WiFi_SuspendStress.11b network_WiFi_SuspendStress.11g network_WiFi_SuspendStress.24HT40 network_WiFi_SuspendStress.5HT40 network_WiFi_SuspendStress.Hidden network_WiFi_SuspendStress.WEP40 network_WiFi_SuspendStress.WPA2 network_WiFi_VisibleScan network_WlanDriver NA: network_WiFi_IBSS.wifi_IBSS network_WiFi_RateControl network_WiFi_Reset network_WiFi_RxFrag.wifi_rxfrag network_WiFi_TDLSPing fail: network_WiFi_Prefer5Ghz (flaky with both old and new whirlwind builds) network_WiFi_WakeOnWiFiThrottling (servo error) chromeos9-row1-cart2-host2 (kevin) pass: network_WiFi_BeaconInterval.wifi_bintval network_WiFi_BgscanBackoff.wifi_bgscan_backoff network_WiFi_ChannelScanDwellTime network_WiFi_ConnectionIdentifier network_WiFi_CSADisconnect network_WiFi_DisableEnable network_WiFi_DisableRandomMACAddress network_WiFi_DisconnectClearsIP network_WiFi_DisconnectReason.ap_gone network_WiFi_DisconnectReason.deauth_client network_WiFi_DisconnectReason.disable_client_wifi network_WiFi_DisconnectReason.switch_ap network_WiFi_DTIMPeriod.wifi_DTIM_period network_WiFi_GTK network_WiFi_HiddenRemains network_WiFi_HiddenScan network_WiFi_LinkMonitorFailure network_WiFi_LowInitialBitrates network_WiFi_MalformedProbeResp network_WiFi_MaskedBSSID.wifi_masked_bssid network_WiFi_MissingBeacons network_WiFi_MultiAuth network_WiFi_OverlappingBSSScan network_WiFi_PMKSACaching network_WiFi_Powersave.wifi_ps network_WiFi_Prefer5Ghz network_WiFi_ProfileBasic network_WiFi_ProfileGUID network_WiFi_RandomMACAddress network_WiFi_Reassociate network_WiFi_Regulatory network_WiFi_Reset network_WiFi_Roam.wifi_roam1xTLS network_WiFi_Roam.wifi_roamNone network_WiFi_Roam.wifi_roamWEP network_WiFi_Roam.wifi_roamWPA network_WiFi_RoamDbus network_WiFi_ScanPerformance network_WiFi_SecChange network_WiFi_SetOptionalDhcpProperties network_WiFi_SimpleConnect.wifi_check11a network_WiFi_SimpleConnect.wifi_check11b network_WiFi_SimpleConnect.wifi_check11g network_WiFi_SimpleConnect.wifi_check1x_PEAP network_WiFi_SimpleConnect.wifi_check1x_TTLS network_WiFi_SimpleConnect.wifi_check1x_WEP network_WiFi_SimpleConnect.wifi_check1x_WPA network_WiFi_SimpleConnect.wifi_check24HT20 network_WiFi_SimpleConnect.wifi_check24HT40 network_WiFi_SimpleConnect.wifi_check5HT20 network_WiFi_SimpleConnect.wifi_check5HT40 network_WiFi_SimpleConnect.wifi_check5VHT80 network_WiFi_SimpleConnect.wifi_checkDFS network_WiFi_SimpleConnect.wifi_checkHidden network_WiFi_SimpleConnect.wifi_checkHiddenWEP network_WiFi_SimpleConnect.wifi_checkHiddenWPA network_WiFi_SimpleConnect.wifi_checkMixedWPA network_WiFi_SimpleConnect.wifi_checkNonAsciiSSID network_WiFi_SimpleConnect.wifi_checkOddWPAPassphrases network_WiFi_SimpleConnect.wifi_checkRawPMK network_WiFi_SimpleConnect.wifi_checkSSIDLimits network_WiFi_SimpleConnect.wifi_checkTruncatedBeacon network_WiFi_SimpleConnect.wifi_checkWEP104 network_WiFi_SimpleConnect.wifi_checkWEP40 network_WiFi_SimpleConnect.wifi_checkWPA2 network_WiFi_SimpleConnect.wifi_checkWPA2_CCMP_PMF network_WiFi_SimpleConnect.wifi_checkWPA2_TKIP network_WiFi_SimpleConnect.wifi_checkWPA_CCMP network_WiFi_SimpleConnect.wifi_checkWPA_multi network_WiFi_SimpleConnect.wifi_checkWPA_TKIP network_WiFi_SuspendStress.11a network_WiFi_SuspendStress.11b network_WiFi_SuspendStress.11g network_WiFi_SuspendStress.24HT40 network_WiFi_SuspendStress.5HT40 network_WiFi_SuspendStress.WEP40 network_WiFi_SuspendStress.WPA2 network_WiFi_VisibleScan network_WlanDriver NA: network_WiFi_IBSS.wifi_IBSS network_WiFi_PTK (also NA recently with 7849 whirlwind build, lost ping packets) network_WiFi_RateControl network_WiFi_RxFrag.wifi_rxfrag network_WiFi_TDLSPing fail: network_WiFi_SuspendStress.Hidden (also fail recently with 7849 whirlwind build, lost ping packets) network_WiFi_WakeOnWiFiThrottling (servo error)
,
Oct 11 2017
Above results were for chromeos9-row1-cart1-host1 whirlwind 3.18 Intel 7265 kefka chromeos9-row1-cart2-host2 whirlwind 4.4 Marvell 88W8997 PCIE kevin Results for: chromeos9-row4-rack6-host3 whirlwind 3.18 Marvell 88W8897 SDIO elm All wifi_matfunc tests pass or expected NA, except these failures: network_WiFi_Reset (also failing with 7849 whirlwind build) network_WiFi_SuspendStress.11a (flaky with both old and new whirlwind builds) network_WiFi_SuspendStress.11b (flaky with both old and new whirlwind builds) network_WiFi_SuspendStress.11g (flaky with both old and new whirlwind builds) network_WiFi_SuspendStress.5HT40 (flaky with both old and new whirlwind builds) network_WiFi_SuspendStress.Hidden (flaky with both old and new whirlwind builds) network_WiFi_SuspendStress.WEP40 (flaky with both old and new whirlwind builds) network_WiFi_WakeOnWiFiThrottling (servo error) Results for: chromeos9-row4-rack4-host1 whirlwind 3.14 Intel 7260 gandof All wifi_matfunc tests pass or expected NA, except these failures: network_WiFi_BgscanBackoff.wifi_bgscan_backoff (also failing with 7849 whirlwind build) network_WiFi_MaskedBSSID.wifi_masked_bssid (also failing with 7849 whirlwind build) network_WiFi_Prefer5Ghz (flaky with both old and new whirlwind builds) network_WiFi_WakeOnWiFiThrottling (servo error) Results for: chromeos9-row1-cart1-host3 whirlwind 4.4 Intel 7265 eve All wifi_matfunc tests pass or expected NA, except these failures: network_WiFi_WakeOnWiFiThrottling (servo error) Results for: chromeos9-row4-rack5-host3 whirlwind 3.14 Broadcom BCM4354 SDIO veyron_minnie All wifi_matfunc tests pass or expected NA, except these failures: network_WiFi_BgscanBackoff.wifi_bgscan_backoff (also failing with 7849 whirlwind build) network_WiFi_LowInitialBitrates (also failing with 7849 whirlwind build) network_WiFi_PMKSACaching (also failing with 7849 whirlwind build) network_WiFi_Prefer5Ghz (also failing with 7849 whirlwind build) network_WiFi_SuspendStress.Hidden (also failing with 7849 whirlwind build) network_WiFi_WakeOnWiFiThrottling (servo error)
,
Oct 11 2017
Ran wifi_perf on chromeos9-row1-cart1-host1 (kefka) with R63-10007.0.0-b5, and compared to previous kefka wifi_perf run with R50-7849.0.0-b13. Attached image plots old (red) and new (blue) throughput numbers. Looks good to me.
,
Oct 15 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/902ed0d55ce220c0b4ae6bcaaaa340910369a67d commit 902ed0d55ce220c0b4ae6bcaaaa340910369a67d Author: Edward Hill <ecgh@chromium.org> Date: Sat Oct 14 02:43:02 2017 network_WiFi_UpdateRouter: change whirlwind version to R63-10007.0.0-b5 BUG= chromium:770255 TEST=wifi_matfunc Change-Id: Id4d7df0e7f599504a31139cd31988c4d61b6a2a5 Reviewed-on: https://chromium-review.googlesource.com/707795 Commit-Ready: Edward Hill <ecgh@chromium.org> Tested-by: Edward Hill <ecgh@chromium.org> Reviewed-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Brian Norris <briannorris@chromium.org> [modify] https://crrev.com/902ed0d55ce220c0b4ae6bcaaaa340910369a67d/server/site_tests/network_WiFi_UpdateRouter/network_WiFi_UpdateRouter.py
,
Oct 30 2017
As of Oct 29, 34 lab whirlwinds have updated to build 10007, and I don't see any change in test results before and after, which is good. There are 5 whirlwinds still on build 7849 because network_WiFi_UpdateRouter hasn't run on them since the version change: chromeos2-row11-rack3-host4 chromeos9-row4-rack1-host3 chromeos9-row4-rack3-host3 chromeos9-row4-rack7-host1 chromeos9-row4-rack8-host1 There are also 19 hosts that have not run network_WiFi_UpdateRouter in Oct: chromeos1-grover-host1 chromeos1-grover-host10 chromeos1-grover-host11 chromeos1-grover-host2 chromeos1-grover-host3 chromeos1-grover-host4 chromeos1-grover-host5 chromeos1-grover-host6 chromeos1-grover-host7 chromeos1-grover-host8 chromeos1-grover-host9 chromeos1-shelf5-host2 chromeos1-shelf5-host4 chromeos1-shelf5-host7 chromeos1-shelf5-host8 chromeos2-row11-rack6-host4 chromeos9-row1-cart1-host8 chromeos9-row1-cart2-host6 chromeos9-row1-cart3-host2
,
Oct 30 2017
Nice! Thanks Ed! I think those grovers don't run whirlwind. Dunno if they're supposed to be running the update-router test though. Not sure about all the rest, but perhaps some were locked for a while? I might have been holding onto at least one or two (somewhat accidentally).
,
Oct 30 2017
Great work Ed! Yes, the grover cells are setup with stumpy or panther APs but also looks like they are not setup to run the update router test. Also the following duts don't have whirlwind. chromeos2-row11-rack6-host4 - stumpy chromeos1-shelf5-host2 - stumpy chromeos1-shelf5-host4 - stumpy chromeos1-shelf5-host7 - stumpy |
||
►
Sign in to add a comment |
||
Comment 1 by ecgh@chromium.org
, Sep 29 2017