Patch failures are resulting in uncaught recipe exceptions and infra failures |
||||||
Issue descriptionPatch failures are resulting in uncaught recipe exceptions and infra failures, e.g.: https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_asan_rel_ng/builds/373591 It seems likely that we've changed something in either the bot_update recipe module or the chromium_tests and chromium modules such that we're incorrectly swallowing an exception and trying to move forward in the build. Adding various people that might've been poking at the relevant code recently.
,
May 18 2017
Ah, no, this is different.
,
May 18 2017
This exception occurs since a long time. Previously it just stayed green, e.g.: https://build.chromium.org/p/tryserver.v8/builders/v8_presubmit/builds/41204 Now it got purple after a recent engine change, which is desired. The code around https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_checkout/api.py?l=87 is simply buggy as it tries to access the properties also in case bot_update failed, in which case there are no properties. Either way, there's no recovery after a patch failure. The patch failure itself was correct, so this isn't having such a high impact. The failure path in the code above should just be fixed, e.g. not setting properties on bot_update failure. I'm not so sure why we need to separate the two steps? E.g. have bot_update green despite error and report the patch failure in a separate step? Is this a pre-git relic? We could just report the patch failure at the bot_update step directly and then bail out.
,
May 18 2017
,
May 18 2017
We separate the two steps in order to make the failures clearer. Apart from failing to apply the patch, almost any other failure in bot_update needs to be addressed by the infra team, because nothing else should fail. On the other hand, a patch failure is almost certainly due to the patch in question and infra doesn't care, but the user does.
,
May 19 2017
Re 5: Thanks and acknowledged. Generally I'm a fan of purple vs. red to distinguish infra vs. user problem. E.g. we also do this in tests and many other steps. But maybe this here is different as, like you say, most of bot_update is infra and only the patch application is a user part. The fix needed her is orthogonal to this anyways. It doesn't really matter if there's one or two steps.
,
May 21 2018
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue. Sorry for the inconvenience if the bug really should have been left as Available. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
May 21 2018
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by iannucci@chromium.org
, May 18 2017