Issue metadata
Sign in to add a comment
|
Infra middleware regression testing |
||||||||||||||||||||||||
Issue descriptionInfra critical features like recovery, repair, Ethernet should be included in regression testing since we would like this to continue working and be reliable. Recommend to add to bvt: * platform_ServoPowerStateController_USBPluggedin * some kind of Ethernet load or bandwidth test * Anything else that's needed by infra to continue working.
,
Jun 4 2018
I don't believe ethernet testing should be part of BVT. If ethernet isn't working (~100% failure), current tests will fail after provisioning and look like a test failure, not infrastructure failure. If ethernet is working (~100% success), we won't detect failures until after ~24 or so of testing, possibly due to other causes (e.g. USB host controller bugs). If ethernet is "mostly working" (< 99% success), BVT likely won't catch that either since it will take hours of testing. We do run "functional stress test" almost daily: search for "Test Name: network_Ether" on stainless.corp. (And I'm surprised it's mostly failing on Relm - by default it only does 1000 iterations - should pass. I'll look into this). Without setting up dedicated HW (servers and network), a load test run on every build would risk havoc in the test lab. I think it would be smarter to set up better (if it's not doing this already) monitoring of BW to/from servers and DUTs in the lab and flagging DUTs that aren't getting the throughput we expect while they are updating/provisioning.
,
Jun 7 2018
Are we talking about testing for regressions in the ChromeOS image, or in the infrastructure code? For the latter, we have a staging lab which provides some functional coverage. If the former, then adding something like platform_ServoPowerStateController_USBPluggedin to bvt might be a good idea. Though unclear to me if it is really ChromeOS image issues that introduce regressions here, or if it is labstation or servod issues (which are not pushed / updated as part of ChromeOS image bvt).
,
Jun 7 2018
,
Jun 8 2018
I think regression testing is needed for infra support in Chrome OS images (which do sometimes break). platform_ServoPowerStateController_USBPluggedin in bvt would be a good sanity check. I think Labstation and servod also need testing, as mentioned in the separate bugcrbug.com/848538
,
Jun 14 2018
The first is easily accomplished: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/1101911
,
Jun 14 2018
> The first is easily accomplished: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/1101911 I've added a comment to the CL. Testing this in BVT may not have the desired impact.
,
Jun 21 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330 commit 8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330 Author: Aviv Keshet <akeshet@chromium.org> Date: Thu Jun 21 23:31:18 2018 add platform_ServoPowerStateController to bvt-perbuild BUG= chromium:848543 TEST=None Change-Id: I4503cb407ee4e3510a566936efc9f7649354bc0a Reviewed-on: https://chromium-review.googlesource.com/1101911 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Richard Barnette <jrbarnette@google.com> Reviewed-by: Nick Sanders <nsanders@chromium.org> [modify] https://crrev.com/8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330/server/site_tests/platform_ServoPowerStateController/control.usb
,
Oct 12
Marking dup of crbug/848538 and part of 2019 planning.
,
Oct 12
|
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by nsanders@chromium.org
, Jun 1 2018