Firmware version of DUTs under test should be accessible via structured data |
|||||
Issue descriptionWe should be able to use the firmware version as part of build/test comparison and will want it in structured data to facilitate that. Ideally it should end up in datastore with each suite run/DUT.
,
Mar 2 2017
I'd like the information to be available through the build tags comparisons (and in the future, test comparisons), where a sheriff can look at two sets of builds/tests (typically failing and succeeding) and have key differences (eg all succeeding builds have FW A and all failing builds have FW B) surfaced automatically. It would also be nice to display it on our suite details of test runs (and should be findable in the logs somewhere).
,
Mar 2 2017
Having all tests report "I ran with firmware version V" is relatively easy; we can include that information in keyvals somewhere. Note that that's different from reporting on upgrade checks, which happen during special tasks, but not tests. Reporting FW version relative to suite runs is likely not to be meaningful. In principle, a firmware upgrade can occur silently during the middle of a suite. So there's no guarantee there won't be more than one version of firmware to report, or that different DUTs won't report different versions at different times. A "all firmware versions reported during testing" value might be useful, in that it would make an upgrade during testing visible.
,
Mar 2 2017
I'd be happy if each test run had a firmware version associated with it, and then collapsing that into a set of unique firmware versions when reporting at the suite or build level.
,
Mar 8 2017
Do we have a process that already dumps boilerplate info about a test and dut run to the tests's keyvals? If so, the next step here is to update it to also dump firmware info.
,
Mar 15 2017
Do we have an immediate owner for this? If not, I'll add it to the FixIt list.
,
Mar 21 2017
,
Mar 31 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by jrbarnette@chromium.org
, Mar 2 2017