CQ tests leak Git daemons. |
||||||||
Issue descriptionCQ tests leak Git daemons, until the system can't bind new ports. This ultimately causes builders to fail. We've paved over the problem for now by auto-rebooting buildslaves: https://chrome-internal-review.googlesource.com/c/422736 However, this is a bad bug and needs to be fixed, as similar leaks could happen on any system that runs these tests. Example builds: - https://luci-milo.appspot.com/buildbot/internal.infra/infra-internal-continuous-trusty-32/4250 - https://luci-milo.appspot.com/buildbot/internal.infra/infra-internal-continuous-trusty-32/4249 - https://luci-milo.appspot.com/buildbot/internal.infra/infra-internal-continuous-trusty-32/4248 - https://luci-milo.appspot.com/buildbot/internal.infra/infra-internal-continuous-trusty-32/4245
,
Aug 2 2017
I think P1 is appropriate because this test is run on developer machines as well, and having those lose gradually get all of their ports consumed seems bad to me.
,
Aug 2 2017
I think this code has started failing more frequently than before. I used to run CQ tests every time I flushed vim buffers, which means every ~10 seconds and never ran into this problem. I suspect new git version made this more likely. The tree got closed again just now for the same reason. I'll monitor this task for another few days and then perhaps just delete these tests after verifying that major projects don't use Rietveld CQ.
,
Aug 2 2017
The tree closing is not surprising because the waterfall wasn't restarted to pick up my fix. I tasked someone with that and it looks like they didn't do it :/
,
Aug 2 2017
Oh well, i'll restart it then: https://chrome-internal-review.googlesource.com/423369
,
Aug 2 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/master-manager/+/c5d4dbc369bfda81acf63e5734f0245e87c5cfd5 commit c5d4dbc369bfda81acf63e5734f0245e87c5cfd5 Author: Andrii Shyshkalov <tandrii@google.com> Date: Wed Aug 02 16:31:42 2017
,
Aug 17 2017
Major projects still use Rietveld CQ, though much less so. Downgrading to Pri2.
,
Aug 31 2017
,
Aug 31 2017
,
Oct 2 2017
,
Oct 2 2017
,
Oct 11 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal/+/2f06d5187315452ad90a14349c26fccd41630368 commit 2f06d5187315452ad90a14349c26fccd41630368 Author: Andrii Shyshkalov <tandrii@chromium.org> Date: Wed Oct 11 23:48:56 2017
,
Oct 12 2017
Tests are gone!
,
Oct 12 2017
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by tandrii@chromium.org
, Aug 2 2017Labels: -Pri-1 Pri-2
Status: Available (was: Untriaged)