betty-release shouldn't fail due to failures in informational Tast tests |
||
Issue descriptionPer http://doc/1ZyozTa907IJqw7QJNHaWIQ07v0szpUOzdLG2K6cURII and https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/test_attributes.md, my intent is for failures in tests with the "informational" attribute to not bother tree guardians, i.e. they shouldn't run inline on -release, -paladin, and -pfq builders. However, betty-release is a bit different from other -release builders since it doesn't run against DUTs in the lab. Tast doesn't get wrapped in an Autotest test there; instead, there's a special TastVMTest stage that runs the tast command directly in a VM. config/chromeos_config.py in chromite currently adds a tast_vm_canary_tests stage using the expression '(!disabled)', which matches informational tests in addition to non-informational ones. This results in sheriffs getting nagged about informational failures like the one in issue 865117 . I should probably update TastVMTest and TastVMTestStage to support a warn_only attribute like what VMTest uses. Then we can have two Tast stages on betty-release, one that runs informational tests with warn_only=True and another that runs non-informational tests with warn_only=False.
,
Jul 20
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/c8f285131ec2d0a0de7b12b45632bb8b177136f0 commit c8f285131ec2d0a0de7b12b45632bb8b177136f0 Author: Daniel Erat <derat@chromium.org> Date: Fri Jul 20 03:12:55 2018 cbuildbot/stages: Make TastVMTest support "informational". Make TastVMTestStage disregard failures in tests with "informational" attributes. This repurposes earlier unused code that watched for "flaky" attributes. BUG= chromium:865256 TEST=updated unit tests pass Change-Id: Ia5d6c6d5318fc274146b1d9f4c2804f198ddf511 Reviewed-on: https://chromium-review.googlesource.com/1142891 Commit-Ready: Dan Erat <derat@chromium.org> Tested-by: Dan Erat <derat@chromium.org> Reviewed-by: Don Garrett <dgarrett@chromium.org> Reviewed-by: Shuhei Takahashi <nya@chromium.org> [modify] https://crrev.com/c8f285131ec2d0a0de7b12b45632bb8b177136f0/cbuildbot/stages/tast_test_stages.py [modify] https://crrev.com/c8f285131ec2d0a0de7b12b45632bb8b177136f0/cbuildbot/stages/tast_test_stages_unittest.py
,
Jul 20
|
||
►
Sign in to add a comment |
||
Comment 1 by derat@chromium.org
, Jul 19