New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 848543 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 848538
Owner: ----
Closed: Oct 12
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature


Participants' hotlists:
SIE-infra-request


Sign in to add a comment

Infra middleware regression testing

Project Member Reported by nsanders@chromium.org, Jun 1 2018

Issue description

Infra critical features like recovery, repair, Ethernet should be included in regression testing since we would like this to continue working and be reliable.

Recommend to add to bvt:

* platform_ServoPowerStateController_USBPluggedin
* some kind of Ethernet load or bandwidth test
* Anything else that's needed by infra to continue working.
 
Summary: Infra middleware regression testing (was: Infra middleare regression testing)
I don't believe ethernet testing should be part of BVT.

If ethernet isn't working (~100% failure), current tests will fail after provisioning and look like a test failure, not infrastructure failure.

If ethernet is working (~100% success), we won't detect failures until after ~24 or so of testing, possibly due to other causes (e.g. USB host controller bugs).

If ethernet is "mostly working" (< 99% success), BVT likely won't catch that either since it will take hours of testing.

We do run "functional stress test" almost daily: search for "Test Name: network_Ether" on stainless.corp.  (And I'm surprised it's mostly failing on Relm - by default it only does 1000 iterations - should pass. I'll look into this).

Without setting up dedicated HW (servers and network), a load test run on every build would risk havoc in the test lab.

I think it would be smarter to set up better (if it's not doing this already) monitoring of BW to/from servers and DUTs in the lab and flagging DUTs that aren't getting the throughput we expect while they are updating/provisioning.
Cc: gu...@chromium.org
Labels: -Type-Bug Type-Feature
Owner: ----
Status: Available (was: Untriaged)
Are we talking about testing for regressions in the ChromeOS image, or in the infrastructure code? For the latter, we have a staging lab which provides some functional coverage.

If the former, then adding something like platform_ServoPowerStateController_USBPluggedin to bvt might be a good idea. Though unclear to me if it is really ChromeOS image issues that introduce regressions here, or if it is labstation or servod issues (which are not pushed / updated as part of ChromeOS image bvt).
Components: Infra>Client>ChromeOS>Test
Labels: -Infra-ChromeOS
I think regression testing is needed for infra support in Chrome OS images (which do sometimes break). platform_ServoPowerStateController_USBPluggedin in bvt would be a good sanity check.

I think Labstation and servod also need testing, as mentioned in the separate bugcrbug.com/848538 
> The first is easily accomplished: https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/1101911

I've added a comment to the CL.  Testing this in BVT may not have the
desired impact.
Project Member

Comment 8 by bugdroid1@chromium.org, Jun 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330

commit 8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330
Author: Aviv Keshet <akeshet@chromium.org>
Date: Thu Jun 21 23:31:18 2018

add platform_ServoPowerStateController to bvt-perbuild

BUG= chromium:848543 
TEST=None

Change-Id: I4503cb407ee4e3510a566936efc9f7649354bc0a
Reviewed-on: https://chromium-review.googlesource.com/1101911
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
Reviewed-by: Nick Sanders <nsanders@chromium.org>

[modify] https://crrev.com/8dcf7ce024bae63d84a6fe1a2fb00c5ed3c33330/server/site_tests/platform_ServoPowerStateController/control.usb

Mergedinto: 848538
Status: Duplicate (was: Available)
Marking dup of crbug/848538 and part of 2019 planning.
Cc: matth...@chromium.org

Sign in to add a comment