Need a tool to display builder test coverage in the lab
Reported by
jrbarnette@chromium.org,
May 2 2017
|
||||
Issue description
We need a tool that will help in evaluating current HWTest
coverage in the lab, and finding gaps. BVT test coverage
requirements for release builders is relatively consistent
and well understood, so the tool should focus on non-release
builders, especially the CQ and PFQ.
The tool needs to be able to gather and report the following
information about every non-release builder that is configured
to run tests:
* From the chromite config: The board the builder builds for.
* From the chromite config: All test suites the builder will run.
* From the chromite config: The pool the suites will run against.
* From the AFE database: The number of DUTs for the selected pool
and board.
* From the AFE database: The number of DUTs in the board's
'suites' pool.
* From CIDB: various run time statistics for HWTest phases;
at minimum, include average run time over the past month or so.
* From CIDB: for each HWTest phase statistic, the analogous
statistic for the builder as a whole.
* From GoldenEye: The board family of the builder.
The tool should also gather information about coverage gaps by
listing the following information about all boards _not_ tested
in a non-release builder:
* From Goldeneye: The board family.
* From the AFE database: The number of DUTs in the board's
'suites' pool.
Additional information that may be helpful:
* The number of tests in each suite.
The board->family information from GoldenEye can be found
here (among other locations):
gs://chromeos-build-release-console/boards.json
Each entry in the file has a "reference_group" field.
,
May 5 2017
,
Jun 21 2017
Probably, I'll wind up doing this, because no one else will care enough. But it's not in my queue yet, and I want this marked as "someone else should feel entitled to work on this."
,
Oct 11 2017
,
Jun 8 2018
Hi, this bug has not been updated recently. Please acknowledge the bug and provide status within two weeks (6/22/2018), or the bug will be archived. Thank you. |
||||
►
Sign in to add a comment |
||||
Comment 1 by jrbarnette@chromium.org
, May 2 20172.0 KB
2.0 KB View Download