New issue
Advanced search Search tips

Issue 895044 link

Starred by 2 users
Status: Fixed
Owner:
derat@chromium.org
Closed: Oct 15
Cc:
akes...@chromium.org
dgarr...@chromium.org
jamescook@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Make HWTest [chrome-informational] not fail for Tast test failures

Project Member Reported by derat@chromium.org, Oct 13 
Failures in Tast tests with "informational" attribute currently cause the "HWTest [chrome-informational]" stage to fail on *-tot-chrome-pfq-informational builders. See e.g. http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8934466642609516624.

While this doesn't cause actual Chrome PFQ runs to fail, it makes it harder for gardeners to spot upcoming failures in "important" tests caused by changes to ToT Chrome.

The tast.informational-chrome and tast.informational-android server tests run with ignore_test_failures=True, but that just controls outcome of the "tast" test. IIUC, individual Tast test failures will still make the stage (and builder) turn red.

James, I think that the fix here is to set warn_only on the chrome-informational suite in chromite. Then the tests will still run, but the stage will just be displayed as yellow (IIRC) if any tests fail. Any objections to me doing that?

Comment 1 by jamescook@chromium.org, Oct 13 

I didn't know we had warn_only. That seems fine to me.

Hopefully we can kill chrome-informational entirely once the Mash tests are important and in the Chrome CQ VMs. The only reason I added chrome-informational was to run the Mash autotests.

Comment 2 by derat@chromium.org, Oct 13

Status: Started (was: Assigned)
I've sent https://crrev.com/c/1279923 for review.
Project Member

Comment 3 by bugdroid1@chromium.org, Oct 15 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/34116c4c4cab91d8a7f6bdeb0206d56a31b1a89e

commit 34116c4c4cab91d8a7f6bdeb0206d56a31b1a89e
Author: Daniel Erat <derat@chromium.org>
Date: Mon Oct 15 17:58:45 2018

chromeos_config: Set warn_only for chrome-informational.

Set warn_only=True on the chrome-informational HWTest suite,
which is executed by *-tot-chrome-pfq-informational
builders.

This suite is used to run informational tests that aren't
necessarily expected to always pass. Making builds fail when
informational tests fail makes it hard for gardeners to
notice actual bugs that have been introduced in upstream
Chrome.

BUG= chromium:895044 
TEST=looked at config/config_dump.json

Change-Id: Ie76b859d77f11a4f849f3b282f29786fb9a24f0d
Reviewed-on: https://chromium-review.googlesource.com/1279923
Commit-Ready: Dan Erat <derat@chromium.org>
Tested-by: Dan Erat <derat@chromium.org>
Reviewed-by: Steven Bennetts <stevenjb@chromium.org>
Reviewed-by: James Cook <jamescook@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>

[modify] https://crrev.com/34116c4c4cab91d8a7f6bdeb0206d56a31b1a89e/config/chromeos_config.py
[modify] https://crrev.com/34116c4c4cab91d8a7f6bdeb0206d56a31b1a89e/config/config_dump.json

Comment 4 by derat@chromium.org, Oct 15

Status: Fixed (was: Started)
►

Sign in to add a comment