New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 697664 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Firmware version of DUTs under test should be accessible via structured data

Project Member Reported by davidri...@chromium.org, Mar 1 2017

Issue description

We should be able to use the firmware version as part of build/test comparison and will want it in structured data to facilitate that.  Ideally it should end up in datastore with each suite run/DUT.
 
Could you spell out exactly where/how the information would be reported,
and how it would be used?

The principle utility I can think of would be that we could more easily
detect manual updates to firmware version assignment.  Automated version
assignments are already logged.

I'd like the information to be available through the build tags comparisons (and in the future, test comparisons), where a sheriff can look at two sets of builds/tests (typically failing and succeeding) and have key differences (eg all succeeding builds have FW A and all failing builds have FW B) surfaced automatically.

It would also be nice to display it on our suite details of test runs (and should be findable in the logs somewhere).
Status: Available (was: Untriaged)
Having all tests report "I ran with firmware version V" is
relatively easy; we can include that information in keyvals
somewhere.  Note that that's different from reporting on
upgrade checks, which happen during special tasks, but not
tests.

Reporting FW version relative to suite runs is likely not to
be meaningful.  In principle, a firmware upgrade can occur
silently during the middle of a suite.  So there's no guarantee
there won't be more than one version of firmware to report, or
that different DUTs won't report different versions at different
times.  A "all firmware versions reported during testing" value
might be useful, in that it would make an upgrade during testing
visible.

I'd be happy if each test run had a firmware version associated with it, and then collapsing that into a set of unique firmware versions when reporting at the suite or build level.
Owner: ----
Do we have a process that already dumps boilerplate info about a test and dut run to the tests's keyvals? If so, the next step here is to update it to also dump firmware info.

Comment 6 by autumn@chromium.org, Mar 15 2017

Do we have an immediate owner for this? If not, I'll add it to the FixIt list. 

Comment 7 by autumn@chromium.org, Mar 21 2017

Labels: Hotlist-Fixit
Components: -Infra>Client>ChromeOS Infra>Client>ChromeOS>Test

Sign in to add a comment