New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 690090 link

Starred by 1 user

Issue metadata

Status: Verified
Owner: ----
Closed: Feb 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

recipes failure on multiple bots

Project Member Reported by roc...@chromium.org, Feb 8 2017

Issue description

Comment 1 by no...@chromium.org, Feb 8 2017

Components: Infra>Platform>Recipes
Owner: no...@chromium.org
Status: Started (was: Untriaged)

Comment 2 by no...@chromium.org, Feb 8 2017

Cc: martiniss@chromium.org
possibly culprit https://chromium-review.googlesource.com/c/437762/

Comment 3 by no...@chromium.org, Feb 8 2017

Cc: iannucci@chromium.org
probably not.

recipe_engine.stream.StreamEngineInvariants.StepStream.write_line breaks on strings that have a newline character.
https://cs.chromium.org/chromium/infra/recipes-py/recipe_engine/stream.py?l=296

but recipes.py calls write_line with failure human_reason without escaping
https://chromium.googlesource.com/external/github.com/luci/recipes-py/+blame/9456147555cc76adec014709e02928b620d7d8d9/recipes.py#107

Comment 4 by no...@chromium.org, Feb 8 2017

Status: Fixed (was: Started)
fix landed 23min ago https://codereview.chromium.org/2688563002/ and got rolled into build.git
https://chromium.googlesource.com/chromium/tools/build/+/master/infra/config/recipes.cfg#14

please verify
Status: Assigned (was: Fixed)
Well, the error message seems to have disappeared:

https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.linux%2Flinux_chromium_chromeos_rel_ng%2F361033%2F%2B%2Frecipes%2Fstderr

but there still seems to be a related failure here:

https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_chromeos_rel_ng/builds/361033

That build was started after comment #4 was posted and appears to have used build rev b3326718 which includes the recipes roll.

Unsure if it's actually the same bug. Thoughts?
Owner: ----
Status: Untriaged (was: Assigned)
This needs more attention. Still seeing failures for unclear reasons, on many CLs (not just mine). Another example:

https://build.chromium.org/p/tryserver.chromium.linux/builders/chromeos_amd64-generic_chromium_compile_only_ng/builds/278845
Cc: dpranke@chromium.org
Status: Verified (was: Untriaged)
Actually in all cases I think perhaps the new problem is:

"Failing build because ninja reported work to do."

So perhaps there's a separate bug in that errors are not being surfaced in any visible way, so they all look like the same generic "steps" failure and you have to be lucky enough to find the meaningful cause in thousands upon thousands of log lines.

Either way, sounds like that's a separate issue, and the original bug here has been fixed.
There's a real issue here with how the "Failing build because ninja reported work to do" problem is being handled.

I'll follow up on that in bog 690269.
 bug 690269  :)

Sign in to add a comment