Add "experimental" option for tests in Chrome CQ |
||
Issue descriptionWhen we add new tests on Chrome CQ, we must be careful not to block innocent CLs. So, we want a way to add a test to CQ without blocking any CLs by its failure, so that we can make sure that the new test is not flaky in advance. This idea is similar to Tast's "informational" attribute. https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/test_attributes.md#manually_added-attributes --- The original context is crbug.com/852302#c52 I want to enable vivid for capture_unittests on Chrome CQ after making sure that the test is stably passed.
,
Oct 3
For example, the setting for capture_unittests on ChromeCQ is here. https://cs.chromium.org/chromium/src/testing/buildbot/test_suites.pyl?l=373&rcl=b7581ca4baf904b7fdddd28d95c8e1238f0f2271 I'd like a way to ignore its failure on CQ temporally.
,
Oct 3
The feature already exists for Chrome's CQ, courtesy of jbudorick@ See how you can mark a suite as "experimental" here: https://chromium.googlesource.com/chromium/src/+/931b6ae754be7eb495ecdbaf0a480c4368823894/testing/buildbot/test_suite_exceptions.pyl#1196 eg: the suite "telemetry_unittests" will run 100% of the time on the "Win10 Tests x64 (dbg)" builder, but if it fails it won't mark the build as failed or block CQ runs.
,
Oct 4
Oh, I didn't know this feature. Ben, Thanks a lot! |
||
►
Sign in to add a comment |
||
Comment 1 by keiichiw@chromium.org
, Oct 3