Consider submitting Tast changes when only Autotest stages have failed |
||||
Issue descriptionIt'd be nice if the CQ would submit changes in src/platform/tast and src/platform/tast-tests when the only builder failures are in HWTest stages that run Autotest tests. Tast changes seem like they should only cause failures in BuildPackages/UnitTests/etc., TastVMTest, and "HWTest [bvt-tast-"-prefixed stages. I tried to look into whether this could be done using COMMIT-QUEUE.ini (http://goo.gl/5J7oND) via the "ignored-stages" directive. In chromite, I think that the relevant code is CanIgnoreFailures (lib/triage_lib.py). It calls GetFailingStages (lib/build_failure_message.py) and GetStagesToIgnore (lib/cq_config.py) and checks if the former is a subset of the latter. GetStagesToIgnore claimns that the returned stages are "stage name prefixes": """Get a list of stages that the CQ should ignore for self.change. The section and option in the config file COMMIT-QUEUE.ini would be like: [GENERAL] ignored-stages: HWTest VMTest The CQ will submit changes to the given project even if the listed stages failed. These strings are stage name prefixes, meaning that "HWTest" would match any HWTest stage (e.g. "HWTest [bvt]" or "HWTest [foo]") Returns: A list of stages (strings) to ignore for self.change. """ However, I think that this can't be used to do a true prefix match. GetFailingStages looks like it's getting stage names from StageFailureMessage.stage_prefix_name (lib/failure_message_lib.py), and I think they're just e.g. 'HWTest'. So unless I'm missing something, we'd probably need some chromite changes to be able to ignore failures in e.g. "HWTest [bvt-inline]" while still blocking a change on failures in e.g. "HWTest [bvt-tast-cq]". As tests are ported from Autotest to Tast (issue 885910), this will hopefully become less of a problem, so I'm not sure if modifying chromite is worth the trouble.
,
Dec 1
,
Dec 3
Exactly what builds/tests do you think are needed to validate a change to Tast? IE: If all tast changes were submitted from the PreCQ, what builds would need to run in the PreCQ?
,
Dec 3
Good question. We'd probably want tests to run on VM (already happens via betty-pre-cq), x86, and ARM DUTs. We'd want to run on DUTs that support ARC (both P and earlier) and DUTs that don't support ARC. We'd also want to make sure that we're running on at least one DUT that supports Crostini (tests aren't on CQ yet, IIRC, but probably will be at some point), and that we have coverage of the different video encode/decode functionality that's tested, and that we support CUPS, and any new features that are added in the future and that might have tests targeting them. So in short, I think it'd be risky to not run on the full set of paladin builders. :-/
,
Dec 3
This will become possible when Parallel CQ launches because every paladin will run for every CL at that time. On a per-directory basis, changes can be opted-out of hardware testing. |
||||
►
Sign in to add a comment |
||||
Comment 1 by derat@chromium.org
, Nov 30