New issue
Advanced search Search tips

Issue 865256 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jul 20
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

betty-release shouldn't fail due to failures in informational Tast tests

Project Member Reported by derat@chromium.org, Jul 19

Issue description

Per http://doc/1ZyozTa907IJqw7QJNHaWIQ07v0szpUOzdLG2K6cURII and https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/test_attributes.md, my intent is for failures in tests with the "informational" attribute to not bother tree guardians, i.e. they shouldn't run inline on -release, -paladin, and -pfq builders.

However, betty-release is a bit different from other -release builders since it doesn't run against DUTs in the lab. Tast doesn't get wrapped in an Autotest test there; instead, there's a special TastVMTest stage that runs the tast command directly in a VM.

config/chromeos_config.py in chromite currently adds a tast_vm_canary_tests stage using the expression '(!disabled)', which matches informational tests in addition to non-informational ones. This results in sheriffs getting nagged about informational failures like the one in  issue 865117 .

I should probably update TastVMTest and TastVMTestStage to support a warn_only attribute like what VMTest uses. Then we can have two Tast stages on betty-release, one that runs informational tests with warn_only=True and another that runs non-informational tests with warn_only=False.
 
Actually, this is easier than that. I already made TastVMTestStage support ignoring failures in tests with a "flaky" attribute that I ended up not using. s/flaky/informational/ should fix this.
Project Member

Comment 2 by bugdroid1@chromium.org, Jul 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/c8f285131ec2d0a0de7b12b45632bb8b177136f0

commit c8f285131ec2d0a0de7b12b45632bb8b177136f0
Author: Daniel Erat <derat@chromium.org>
Date: Fri Jul 20 03:12:55 2018

cbuildbot/stages: Make TastVMTest support "informational".

Make TastVMTestStage disregard failures in tests with
"informational" attributes. This repurposes earlier unused
code that watched for "flaky" attributes.

BUG= chromium:865256 
TEST=updated unit tests pass

Change-Id: Ia5d6c6d5318fc274146b1d9f4c2804f198ddf511
Reviewed-on: https://chromium-review.googlesource.com/1142891
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[modify] https://crrev.com/c8f285131ec2d0a0de7b12b45632bb8b177136f0/cbuildbot/stages/tast_test_stages.py
[modify] https://crrev.com/c8f285131ec2d0a0de7b12b45632bb8b177136f0/cbuildbot/stages/tast_test_stages_unittest.py

Status: Fixed (was: Started)

Sign in to add a comment