New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 889214 link

Starred by 1 user

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

caroline-chrome-pfq is not running HWTest

Project Member Reported by steve...@chromium.org, Sep 25

Issue description

This was mentioned but not resolved in issue 759096. Moving this specific discussion to a separate issue.

To the best of my knowledge, there is no documentation as to which PFQ builders run HWTests.

I am aware that the reason is often to do with availability of DUTs, however without documentation, it is difficult to know this. Also, there is the possibility that HWTests are not running even when DUTs are available.

Case in point, caroline-arcnext-chrome-pfq was recently added to the PFQ and *is* running HWTests, even though caroline-chrome-pfq is not. The reason for this is not clear.

 
Cc: jrbarnette@chromium.org
The definition of hardware tests in the CQ is around https://cs.corp.google.com/chromeos_public/chromite/config/chromeos_config.py?rcl=7fd8b20a597af3ee31ed185a8c0d6229ecabf2af&l=3904

The explanations as to why certain boards have tests are more nuanced, as you suggest these are often related to DUT counts in the lab, but also about stability and such.

We probably should clean the code around this up a bit more, for the CQ this is organized into a table at  https://cs.corp.google.com/chromeos_public/chromite/config/chromeos_config.py?rcl=7fd8b20a597af3ee31ed185a8c0d6229ecabf2af&l=2734

Current DUT allocations are:
bhthompson@ragnarok:/usr/local/ssd960gb/chromiumos-1/src/third_party/autotest/files/contrib$ atest host list -b model:caroline | ./count_labels -p
     10 bvt
      4 continuous
      7 cq
      1 crosperf
      1 crosperf_arcnext
     32 cts
      3 faft-test
      1 groamer
      1 hctsai_bisect_group
      1 performance
     13 suites
      1 wificell
     75 total

I believe the Chrome PFQ uses the bvt pool, I am not entirely certain if the pool can take having both caroline and caroline-arcnext running HWTest in the PFQ, it will probably work but may begin to be flaky.

We may want to select some other SKL-Y board if we want another SKL-Y board running tests with N and not P.
Is there any action needed here then?
Do we want more coverage for caroline in the PFQ?

10 DUTs can't handle two PFQs and release. If we want caroline-pfq along with caroline-arcnext-pfq, we'll need to expand that pool.
Also, this may still cause some swapping as the two PFQs will try to get all the DUTs in the pool for their HWTests.
I guess I mostly just have questions:

1. What can we provide information so that a gardener can understand which builders are running which HW test suites and why? (Other than inspecting a builder from each of the 22 builders, or parsing through the linked script, which isn't really a table, and doesn't explicitly mention caroline-chrome-pfq at all).

2. Why are we prioritizing arcnext for running HWTests for caroline? (See 1).

Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>CI
Labels: -Pri-1 Pri-2

Sign in to add a comment