New issue
Advanced search Search tips

Issue 599171 link

Starred by 3 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Mac
Pri: 2
Type: Bug-Regression



Sign in to add a comment

Fix flaky test in monitoring for go

Project Member Reported by tandrii@chromium.org, Mar 30 2016

Issue description

Example: https://build.chromium.org/p/chromium.infra/builders/infra-continuous-mac-10.9-64/builds/1753/steps/infra%20go%20tests/logs/stdio


--------------------------------------------------------------------------------
PACKAGE: infra/monitoring/proxy
--------------------------------------------------------------------------------
.....
5 total assertions

.........................................................
62 total assertions

.......................
85 total assertions

.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
939 total assertions

........................................................................................................................................................................................x
Failures:

  * /b/build/slave/infra-continuous/build/infra/go/src/infra/monitoring/proxy/main_test.go 
  Line 87:
  Error during mock: No registered mock matching send(context.Background.WithValue(0, (func() retry.Iterator)(0x9a200)).WithCancel.WithValue(1, logging.Fields{"worker":8}).WithValue(1, logging.Fields{"messageID":"msg-3", "worker":8, "size":1}), [3]).
  Error during mock: Result count (0) differs from bound args (1).
  Unused mock was registered: send(&{}, [3])


1125 total assertions

--- FAIL: TestMain (0.04s)
FAIL
coverage: 57.5% of statements
FAIL	infra/monitoring/proxy	0.073s
--------------------------------------------------------------------------------
step returned non-zero exit code: 1
 
Owner: sergeybe...@chromium.org
Status: Assigned (was: Untriaged)
Cc: d...@chromium.org
 Issue 599295  has been merged into this issue.
Labels: Pri-1
I receive 1 email daily from our CI about the flakes. I don't think this flake should be ignored any more:

https://build.chromium.org/p/chromium.infra/builders/infra-continuous-trusty-64/builds/1952/steps/infra%20go%20tests/logs/stdio

Comment 4 by d...@chromium.org, Apr 13 2016

sergeyberezin@ has wanted to update this script to use newer LUCI code for a while now. Most of the testing and complications will be deleted when this happens.

Comment 5 by d...@chromium.org, Apr 13 2016

Labels: -Pri-1 Pri-2
Also: Getting annoying e-mails doesn't make things P1.
Sure, but having builds failing randomly and thus preventing building cipd packages (and hence causing pain down the road) does.

Comment 7 by d...@chromium.org, Apr 13 2016

There is no intention of healing this component, since it is destined to be replaced by a better one in "luci-go". Therefore, let's kill it.

Change the test from Convey to SkipConvey.
Status: Fixed (was: Assigned)
The most annoying test is now disabled. There is still one flaky test remaining, but hopefully it's tolerable until we rewrite the proxy:

--------------------------------------------------------------------------------
PACKAGE: infra/monitoring/looper
--------------------------------------------------------------------------------
--- FAIL: TestLoopOverrunSome (0.00s)
	looper_test.go:114: Want 5 Overruns got 6

Comment 10 by d...@chromium.org, Apr 14 2016

That failure is not part of the proxy, and will not be affected by the monitoring proxy rewrite.
Status: Assigned (was: Fixed)
More reports about flaky tests in infra/monitoring/looper:
https://groups.google.com/a/google.com/forum/#!topic/chrome-infrastructure-team/xiI0Ny0bZuE
Yeah, it's the second most frequent one. I'll fix it.
Re #5: > Also: Getting annoying e-mails doesn't make things P1.

Yep. Sorry, I should have been clear: I receive email on every tree closing build failure as self-appointed-gardener, and i'd say 20% of time the person on the blamelist replies saying "nope, no my CL". So, it's annoying not to me, but to others who waste their time looking into these failurse.
Status: Started (was: Assigned)
Uploaded http://crrev.com/1896213002
Status: Fixed (was: Started)
This should now be fixed (again). Hopefully this time for good.
Components: Infra>Monitoring
Labels: -Infra-Monitoring

Sign in to add a comment