Missing tests/variations in BVT runs without clear messaging |
||||||
Issue descriptionTake a look here: https://wmatrix.googleplex.com/bvt-cq?tests=video*&hide_missing=false One would expect test counts to add up to a consistent number within a platform column but there seems to be some dips (for instance 'Banon' and 'Butterfly' on Wed Sep 14, 20:35). Drilling in further doesn't expose any details on why those runs went missing. Not enough machines? Provisioning errors? rohitbm@ also found logs here that point to a Timeout error during provisioning: https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/76979198-chromeos-test/chromeos4-row8-rack5-host11. Is this a common issue?
,
Sep 16 2016
,
Sep 19 2016
I suspect the test runs just did not happen cause of DUT contention or other lab issues. I suggest looking at the suite jobs on cautotest if you believe test runs are missing.
,
Sep 21 2016
Rohit found that the provisioning failed. However that require too much digging and should have surface the error to wmatrix instead.
,
Sep 21 2016
,
Sep 21 2016
Big problem is theres no clear owner of wmatrix to do feature requests.
,
Sep 22 2016
I think we need to decide if wmatrix should be rebuild or keep, either way we need to have resource working on it.
,
Sep 22 2016
We could add a graph of historic test results for each board to visualize a dip better. The necessary data should be already imported into wmatrix. We would need to allocate eng resource on it though.
,
Jul 18 2017
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by dchan@chromium.org
, Sep 16 2016Components: Infra>Client>ChromeOS