wifi autotests: Kill/Trim existing tests to make time for pool:wificell to run a reboot-test |
||
Issue descriptionI plan to add network_WiFi_Reboot test that checks we find wlan0 (or mlan0 or whatever the equivalent is) after reboots. 10 iterations should be good nightly, and if we find time/space ina stress pool, we can increase that. This would be helpful for cyan and newer designs (like <redacted> where the wlan load switch is being removed). The DUTs in pool:wificell currently seem loaded - not sure how much idle time they have between nightly runs. Chatted with Harpreet and we agreed that we can trim the fat from some existing tests: - network_WiFi_SuspendStress.11a/b : 802.11a and 802.11b together comprise less than 0.1% of our population. (Google internal screenshot: https://screenshot.googleplex.com/xSFfhg2zMCZ) - Harpreet pointed out that network_WiFi_SuspendStress.stress_24HT40 and network_WiFi_SuspendStress.stress_WPA2 run 690 iterations each. Can we shorten this to 100?
,
Aug 24
pool:wificell or pool:stress-wifi? > - Harpreet pointed out that network_WiFi_SuspendStress.stress_24HT40 and network_WiFi_SuspendStress.stress_WPA2 run 690 iterations each. > Can we shorten this to 100? Those are only run in 'pool:stress-wifi' (suite:wifi_stress). We can shorten them if we have a reason, but it does seem fitting that the "stress" pool would do more iterations... And more importantly: shortening those won't free you up any time on pool:wificell > - network_WiFi_SuspendStress.11a/b : 802.11a and 802.11b together Sure, as long as we still keep some coverage on this test for more popular configurations.
,
Aug 27
> Those are only run in 'pool:stress-wifi' (suite:wifi_stress). We can shorten them if we have a reason, but it does seem fitting that the "stress" pool would do more iterations... Correct. There are only 3 devices in this pool and we haven't been tracking these results. So in its current state its not really useful. I could use this space for other 'new' device installs if we are ok with removing the stress pool / suite. But if we see value in keeping it, it'll be best to expand the pool to at least cover a decent subset of kernels / wifi modules and start tracking test results. Note - I did not account for expanding this pool so we'll need to wait till early 2019 to get additional RF enclosures to expand this pool.
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0edd4fefffd73192552d0208e201fa1d20922cb7 commit 0edd4fefffd73192552d0208e201fa1d20922cb7 Author: Kirtika Ruchandani <kirtika@google.com> Date: Wed Aug 29 18:15:13 2018 network_WiFi_SuspendStress: Remove dead code This test doesn't rely on servo anymore, and doesn't need a reboot to clean up state. BUG= chromium:877652 TEST=Ran network_WiFi_SuspendStress on eve after change. Change-Id: I39912766590754007c5157bf7d7ce79233835411 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1188993 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Harpreet Grewal <harpreet@chromium.org> [modify] https://crrev.com/0edd4fefffd73192552d0208e201fa1d20922cb7/server/site_tests/network_WiFi_SuspendStress/network_WiFi_SuspendStress.py
,
Aug 29
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/c00409f990d28b116a2612c8643b474bf0ea8789 commit c00409f990d28b116a2612c8643b474bf0ea8789 Author: Kirtika Ruchandani <kirtika@google.com> Date: Wed Aug 29 18:15:14 2018 network_WiFi_SuspendStress: Kill 802.11a/b variants Based on observing autoserv.DEBUG timestamps from these test runs, they have highly variable runtime. Each one can take from 10 mins to 60 mins, the suspend uses a random number for time to suspend/stay awake. Both 802.11a and 80211b are old protocols that see very little usage in the field (less than 0.1% combined of total users). It makes sense to kill these tests to allow pool:wificell to spend time on more relevant tests (prioritizing HT40, VHT80, adding reboot tests etc). 802.11a and 802.11b will continue getting some coverage via network_WiFi_SimpleConnect. BUG= chromium:877652 TEST=Ran network_WiFi_SuspendStress on eve after change. Change-Id: Ia3f6393cc8134d2a0dc1737bcbc3b3d11e624a50 Signed-off-by: Kirtika Ruchandani <kirtika@google.com> Reviewed-on: https://chromium-review.googlesource.com/1188994 Commit-Ready: Kirtika Ruchandani <kirtika@chromium.org> Tested-by: Kirtika Ruchandani <kirtika@chromium.org> Reviewed-by: Harpreet Grewal <harpreet@chromium.org> [delete] https://crrev.com/0edd4fefffd73192552d0208e201fa1d20922cb7/server/site_tests/network_WiFi_SuspendStress/control.11b [delete] https://crrev.com/0edd4fefffd73192552d0208e201fa1d20922cb7/server/site_tests/network_WiFi_SuspendStress/control.11a
,
Sep 7
|
||
►
Sign in to add a comment |
||
Comment 1 by kirtika@chromium.org
, Aug 24