New issue
Advanced search Search tips

Issue 908973 link

Starred by 1 user

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Provide a way to disable CQ retries

Project Member Reported by fdegans@chromium.org, Nov 27

Issue description

A bit of background:
A bad roll of the Fuchsia SDK autoroller made it through the CQ because the tests ending up passing on retry:
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/fuchsia_x64/154584

This ended up being a bad roll, causing lots of flakiness across all Fuchsia test suites. We would like a way to prevent this from happening again. Our idea would be to disable CQ retries on Fuchsia bots for the Fuchsia autoroller CLs.
I am not familiar enough with Infra to know how we could best meet that goal, feel free to redirect or reformulate as you wish.
 
Cc: erikc...@chromium.org tandrii@chromium.org
Components: -Infra Infra>Platform>CQ Infra>Client>Chrome
+ some relevant components and people

So you're essentially looking for something like

  Cq-No-Retry: luci.chromium.try:fuchsia_arm64,luci.chromium.try:fuchsia_x64

as a gerrit footer which would exclude retries solely at the build level?
I think it's also reasonable to entertain possibility of annotating individual builders in cq.cfg with retry directives if e.g., fuchsia_arm64 builder is always required. Something like this:
..
builder {
  name: "fuchsia_arm64"
  retry_policy {
    failures: 0
    infra_failures: 1
  }
}
Status: Available (was: Untriaged)
I like the idea of #c2. IIUC, we are moving away from dynamically adding footers for CQ.
I suspect we'll still want a way to tweak the CQ on a per-CL basis, though (e.g., for autorollers). I'm not sure what our plan is for that beyond gerrit footers.

I know we replaced PRESUBMIT's addition of Cq-Include-Trybots; if there are plans to expand that to what the autorollers do, it's news to me.
There had been offline discussions about having CQ somehow understand DEPS.py and trigger different trybots depending on which file line gets changed.

However, given path_regexp feature, there is an easy but ugly workaround:
 * autorollers modifies not just DEPS file, but also some other whitespace-like file, say autoroll-fuchsia-trigger-bots.whitespace.
 * cq.cfg contains:
 builder {
   name: "fuchsia"
   path_regexp: "autoroll-fuchsia-trigger-bots.whitespace"
 }

That indeed is ugly.

Sign in to add a comment