New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 675485 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

LogDog seems to have a memory leak.

Project Member Reported by d...@chromium.org, Dec 18 2016

Issue description

Instances eventually die with "soft memory limit exceeded". Bumping memory from 128MB to 256MB still incurs this error when the 256MB limit is exceeded.

This is observed frequently on the "services" module, which is notable because that module does not actually handle log data.
 
Project Member

Comment 2 by bugdroid1@chromium.org, Dec 20 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/fd0a102c24af05830dccbce7aeb7cf8d3295e90c

commit fd0a102c24af05830dccbce7aeb7cf8d3295e90c
Author: dnj <dnj@chromium.org>
Date: Tue Dec 20 01:06:45 2016

Don't allcoate a new math/rand every request.

Use a global math/rand instance, rather than re-allocating one for each
request.

BUG= chromium:675485 
TEST=None
R=vadimsh@chromium.org

Review-Url: https://codereview.chromium.org/2594463002

[modify] https://crrev.com/fd0a102c24af05830dccbce7aeb7cf8d3295e90c/appengine/gaemiddleware/context.go

Project Member

Comment 4 by bugdroid1@chromium.org, Dec 22 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/464d9ab6bc1614432afa076e4416b0033028cd7b

commit 464d9ab6bc1614432afa076e4416b0033028cd7b
Author: dnj <dnj@chromium.org>
Date: Thu Dec 22 01:43:29 2016

Create unbuffered Tumble entry point for LogDog.

Add RegisterInTransaction, an entry point for Tumble that initiates a
standard datastore transaction, performs an operation, and registers the
ensuing mutation against it. This is different from RunMutation in that
it doesn't require a Mutation as a starting point and also doesn't
create a transaction buffer.

Leverage this in LogDog's "registerStream" to reduce the overhead of
stream registration.

BUG= chromium:675485 
TEST=unit

Review-Url: https://codereview.chromium.org/2592753002

[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/logdog/appengine/coordinator/endpoints/services/registerStream.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/logdog/appengine/coordinator/endpoints/services/registerStream_test.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/logdog/appengine/coordinator/endpoints/services/terminateStream.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/logdog/appengine/coordinator/endpoints/services/terminateStream_test.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/logdog/appengine/coordinator/mutations/createArchiveTask.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/tumble/process.go
[modify] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/tumble/tumble.go
[add] https://crrev.com/464d9ab6bc1614432afa076e4416b0033028cd7b/tumble/tumble_test.go

Project Member

Comment 5 by bugdroid1@chromium.org, Dec 28 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/92236760134c91430cfd83d8d1e48d75703068b7

commit 92236760134c91430cfd83d8d1e48d75703068b7
Author: Dan Jacques <dnj@chromium.org>
Date: Wed Dec 28 20:44:26 2016

Update Go deps, add "luci/gtreap".

BUG= chromium:675485 
TEST=None

Change-Id: Icacba13206c426ad71d409ac50bacd24061cd02a
Reviewed-on: https://chromium-review.googlesource.com/424215
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/DEPS
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/deps.lock
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/deps.yaml
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/crimson/proto/crimson.pb.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/crimson/proto/pb.discovery.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/monorail/monorail.infra_testing
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/monorail/monorail.pb.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/monorail/pb.discovery.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/tools/cloudtail/chan.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/tools/cloudtail/chan_test.go
[modify] https://crrev.com/92236760134c91430cfd83d8d1e48d75703068b7/go/src/infra/tools/cloudtail/client.go
[delete] https://crrev.com/123f6d4cc0877027e799ddd15b22c9133a026291/go/src/infra/tricium/api/v1/pb.discovery.go

Project Member

Comment 6 by bugdroid1@chromium.org, Dec 29 2016

Project Member

Comment 7 by bugdroid1@chromium.org, Jan 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/a675724fae898cba8386a64186ad75b71730c393

commit a675724fae898cba8386a64186ad75b71730c393
Author: dnj <dnj@chromium.org>
Date: Sat Jan 07 02:39:07 2017

Fix race in treapstore test.

The treapstore test currently asserts that at least one read happens.
The first round, however, it is possible that no writes happen before
the read round finishes. Adjust the test to accommodate this.

The check itself is not very useful. The purpose of this test is to
expose a bunch of potential races and assert that they aren't actually
races.

TBR=vadimsh@chromium.org
BUG= chromium:675485 
TEST=local
  - for i in `seq 100`; do go test -cpu=1,2,4 ./common/data/treapstore;
    done

Review-Url: https://codereview.chromium.org/2613153007

[modify] https://crrev.com/a675724fae898cba8386a64186ad75b71730c393/common/data/treapstore/store_test.go

Project Member

Comment 8 by bugdroid1@chromium.org, Jan 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/647d8243b192c5a603b4befc3cf524f716003b95

commit 647d8243b192c5a603b4befc3cf524f716003b95
Author: dnj <dnj@chromium.org>
Date: Sun Jan 08 19:15:46 2017

treapstore: Add Collection.VisitItemsAscend.

This offers an optimal, but unfortunately callback-based, traversal of
the collection.

BUG= chromium:675485 
TEST=unit

Review-Url: https://codereview.chromium.org/2616303002

[modify] https://crrev.com/647d8243b192c5a603b4befc3cf524f716003b95/common/data/treapstore/store.go
[modify] https://crrev.com/647d8243b192c5a603b4befc3cf524f716003b95/common/data/treapstore/store_test.go

Project Member

Comment 9 by bugdroid1@chromium.org, Jan 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/e4594257cfc1171950f4e75b6ee70e5580eb0958

commit e4594257cfc1171950f4e75b6ee70e5580eb0958
Author: Dan Jacques <dnj@chromium.org>
Date: Sun Jan 08 20:18:44 2017

Roll luci-go to pick up treapstore change

infra/go/src/github.com/luci/luci-go:
647d8243 treapstore: Add Collection.VisitItemsAscend.
a675724f Fix race in treapstore test.
4600ce7d Use batch add for Tumble task queue tasks.

TBR=iannucci@chromium.org
BUG= chromium:675485 
TEST=None

Change-Id: I02e578020d09f2f4aff9d1cbc7745fc099699d94
Reviewed-on: https://chromium-review.googlesource.com/425843
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/e4594257cfc1171950f4e75b6ee70e5580eb0958/DEPS

Comment 10 by d...@chromium.org, Jan 8 2017

Owner: d...@chromium.org
Status: Fixed (was: Untriaged)

Sign in to add a comment