New issue
Advanced search Search tips

Issue 910445 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature

Blocked on:
issue 910391


Show other hotlists

Hotlists containing this issue:
CrOSParallelCQ


Sign in to add a comment

Consider submitting Tast changes when only Autotest stages have failed

Project Member Reported by derat@chromium.org, Nov 30

Issue description

It'd be nice if the CQ would submit changes in src/platform/tast and src/platform/tast-tests when the only builder failures are in HWTest stages that run Autotest tests. Tast changes seem like they should only cause failures in BuildPackages/UnitTests/etc., TastVMTest, and "HWTest [bvt-tast-"-prefixed stages.

I tried to look into whether this could be done using COMMIT-QUEUE.ini (http://goo.gl/5J7oND) via the "ignored-stages" directive.

In chromite, I think that the relevant code is CanIgnoreFailures (lib/triage_lib.py). It calls GetFailingStages (lib/build_failure_message.py) and GetStagesToIgnore (lib/cq_config.py) and checks if the former is a subset of the latter.

GetStagesToIgnore claimns that the returned stages are "stage name prefixes":

    """Get a list of stages that the CQ should ignore for self.change.

    The section and option in the config file COMMIT-QUEUE.ini would be like:

    [GENERAL]
      ignored-stages: HWTest VMTest

    The CQ will submit changes to the given project even if the listed stages
    failed. These strings are stage name prefixes, meaning that "HWTest" would
    match any HWTest stage (e.g. "HWTest [bvt]" or "HWTest [foo]")

    Returns:
      A list of stages (strings) to ignore for self.change.
    """

However, I think that this can't be used to do a true prefix match. GetFailingStages looks like it's getting stage names from StageFailureMessage.stage_prefix_name (lib/failure_message_lib.py), and I think they're just e.g. 'HWTest'.

So unless I'm missing something, we'd probably need some chromite changes to be able to ignore failures in e.g. "HWTest [bvt-inline]" while still blocking a change on failures in e.g. "HWTest [bvt-tast-cq]".

As tests are ported from Autotest to Tast (issue 885910), this will hopefully become less of a problem, so I'm not sure if modifying chromite is worth the trouble.
 
Components: Tests>Tast
Components: -Infra>Client>ChromeOS>Test Infra>Client>ChromeOS>CI
Cc: jclinton@chromium.org
Exactly what builds/tests do you think are needed to validate a change to Tast?

IE: If all tast changes were submitted from the PreCQ, what builds would need to run in the PreCQ?
Good question. We'd probably want tests to run on VM (already happens via betty-pre-cq), x86, and ARM DUTs. We'd want to run on DUTs that support ARC (both P and earlier) and DUTs that don't support ARC. We'd also want to make sure that we're running on at least one DUT that supports Crostini (tests aren't on CQ yet, IIRC, but probably will be at some point), and that we have coverage of the different video encode/decode functionality that's tested, and that we support CUPS, and any new features that are added in the future and that might have tests targeting them.

So in short, I think it'd be risky to not run on the full set of paladin builders. :-/
Blockedon: 910391
Labels: CrOSParallelCQ
Status: Available (was: Untriaged)
This will become possible when Parallel CQ launches because every paladin will run for every CL at that time. On a per-directory basis, changes can be opted-out of hardware testing.

Sign in to add a comment