caroline-chrome-pfq is not running HWTest |
|||
Issue descriptionThis was mentioned but not resolved in issue 759096. Moving this specific discussion to a separate issue. To the best of my knowledge, there is no documentation as to which PFQ builders run HWTests. I am aware that the reason is often to do with availability of DUTs, however without documentation, it is difficult to know this. Also, there is the possibility that HWTests are not running even when DUTs are available. Case in point, caroline-arcnext-chrome-pfq was recently added to the PFQ and *is* running HWTests, even though caroline-chrome-pfq is not. The reason for this is not clear.
,
Sep 25
The definition of hardware tests in the CQ is around https://cs.corp.google.com/chromeos_public/chromite/config/chromeos_config.py?rcl=7fd8b20a597af3ee31ed185a8c0d6229ecabf2af&l=3904 The explanations as to why certain boards have tests are more nuanced, as you suggest these are often related to DUT counts in the lab, but also about stability and such. We probably should clean the code around this up a bit more, for the CQ this is organized into a table at https://cs.corp.google.com/chromeos_public/chromite/config/chromeos_config.py?rcl=7fd8b20a597af3ee31ed185a8c0d6229ecabf2af&l=2734 Current DUT allocations are: bhthompson@ragnarok:/usr/local/ssd960gb/chromiumos-1/src/third_party/autotest/files/contrib$ atest host list -b model:caroline | ./count_labels -p 10 bvt 4 continuous 7 cq 1 crosperf 1 crosperf_arcnext 32 cts 3 faft-test 1 groamer 1 hctsai_bisect_group 1 performance 13 suites 1 wificell 75 total I believe the Chrome PFQ uses the bvt pool, I am not entirely certain if the pool can take having both caroline and caroline-arcnext running HWTest in the PFQ, it will probably work but may begin to be flaky. We may want to select some other SKL-Y board if we want another SKL-Y board running tests with N and not P.
,
Sep 25
Apologies, the first line should say PFQ for https://cs.corp.google.com/chromeos_public/chromite/config/chromeos_config.py?rcl=7fd8b20a597af3ee31ed185a8c0d6229ecabf2af&l=3904
,
Sep 25
Is there any action needed here then? Do we want more coverage for caroline in the PFQ? 10 DUTs can't handle two PFQs and release. If we want caroline-pfq along with caroline-arcnext-pfq, we'll need to expand that pool. Also, this may still cause some swapping as the two PFQs will try to get all the DUTs in the pool for their HWTests.
,
Sep 25
I guess I mostly just have questions: 1. What can we provide information so that a gardener can understand which builders are running which HW test suites and why? (Other than inspecting a builder from each of the 22 builders, or parsing through the linked script, which isn't really a table, and doesn't explicitly mention caroline-chrome-pfq at all). 2. Why are we prioritizing arcnext for running HWTests for caroline? (See 1).
,
Oct 25
,
Oct 25
|
|||
►
Sign in to add a comment |
|||
Comment 1 by dgarr...@chromium.org
, Sep 25