signer-pre-cq not reporting success correctly. |
||||||||
Issue descriptionThe Pre-CQ picked up this CL: https://chrome-internal-review.googlesource.com/c/340135/ Here's the build log, it passed: https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/pre_cq/24959 However the run was marked as failed and the CL updated with "The signer-pre-cq trybot for your change timed out after 240 minutes."
,
Mar 29 2017
This is happening again, https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/pre_cq/25100 completed successfully 2 hours ago but the pass result still hasn't been posted to the CL. It might be related to the fact that it's signer-pre-cq, apparently there were other problems with it before: https://bugs.chromium.org/p/chromium/issues/detail?id=673906 Raising the priority to P1 since this blocking the CL (the CL has dependencies under review, it's not a P0). Don, can I assign the bug to you since you fixed the previous issue?
,
Mar 29 2017
This is a different problem. Last time, it was because I created a broken PreCQ builder. This looks more like a buildbucket issue of some kind, so passing to our buildbucket expert.
,
Mar 29 2017
Another example of this issue: https://chrome-internal-review.googlesource.com/#/c/340218/
,
Mar 29 2017
If we can show it's a buildbucket problem, we should escalate to Chrome Ops.
,
Mar 29 2017
Does this happen on the other pre-cqs besides signer-pre-cq?
,
Mar 29 2017
Examining the example from #4... The builder started it's next build at "Mar 29 11:16", and the CL had the timeout reported at "2:06 PM". That shows that the builder really did complete long before the timeout.
,
Mar 29 2017
It looks to me the Pre-CQs didn't have the PreCQCompletion stage, like the issue what we saw months ago.
,
Mar 29 2017
I totally missed that they were both signer CLs. I'm an idgit.
,
Mar 29 2017
PS: For any signer CLs, if the tryjob passes, chumping is the right thing to do, until this is fixed.
,
Apr 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/081f2803753adf91d8a36640f81c1baa4925091b commit 081f2803753adf91d8a36640f81c1baa4925091b Author: Don Garrett <dgarrett@google.com> Date: Tue Apr 04 02:09:22 2017 SignerTestBuilder: Use PreCqBuilder base class. I did some refactoring a while back to make it easier to add new PreCQ builders. I didn't adjust the SignerTestBuilder to use it until the new mechanism was proven out. However, while creating the new mechanism I modified GenericBuilder in a way that broke SignerTestBuilder. BUG= chromium:706257 TEST=run_tests Change-Id: Ibf5b5bbd8f7a378e2a5e8b71f6ff01ec47f905bc Reviewed-on: https://chromium-review.googlesource.com/465353 Commit-Ready: Don Garrett <dgarrett@chromium.org> Tested-by: Don Garrett <dgarrett@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> [modify] https://crrev.com/081f2803753adf91d8a36640f81c1baa4925091b/cbuildbot/builders/test_builders.py
,
Apr 12 2017
I believe this is fixed. Please reopen, if that's not true.
,
Apr 12 2017
Yes, it's fixed. Confirmed with CL:*340135 |
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by norvez@chromium.org
, Mar 29 2017