New issue
Advanced search Search tips

Issue 888665 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Mojo unneccessarily using 1.6GB of memory

Project Member Reported by erikc...@chromium.org, Sep 24

Issue description

See https://bugs.chromium.org/p/chromium/issues/detail?id=884705 for parent issue.

If I had to guess what's going on -- there were a large number of IPCs queued. This caused the underlying message buffer [vector?] to grow in size. Eventually the messages were dequeued but the buffer/vector stayed huge.
 
Screen Shot 2018-09-24 at 2.52.59 PM.png
179 KB View Download
trace_with_heap_dump2.json.gz
621 KB Download
Where is the evidence that these messages were dequeued? And dequeued from what? Looks like an incoming message queue if I understand the screenshot correctly.

The code which retains incoming messages is pretty trivial. Unless the vector impl itself is broken, the scenario you describe is not possible.

To me it looks more like a message pipe is being leaked (or at least held unbound indefinitely) and so it never has its messages read.
> Where is the evidence that these messages were dequeued? And dequeued from what? Looks like an incoming message queue if I understand the screenshot correctly.


In the parent bug, Chrome was unusable for several hours [while messages were being dequeued], and memory usage spiked to 80GB. It eventually quiesced down to 2GB. 

> The code which retains incoming messages is pretty trivial. Unless the vector impl itself is broken, the scenario you describe is not possible.

Vectors do not shrink by default. I just tested this locally with a test C++ program. pop_back() will never shrink the backing buffer. We need to call shrink_to_fit() to reduce the size of the backing allocation of std::vector.
Ah, thanks. Didn't realize that, but it does kind of make sense. Will fix!
Cc: ssid@chromium.org
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7fe00cd1270a395d3f1d66b14e41595a432d6ddc

commit 7fe00cd1270a395d3f1d66b14e41595a432d6ddc
Author: Ken Rockot <rockot@chromium.org>
Date: Tue Sep 25 21:18:20 2018

[mojo-core] Reduce lingering message queue costs

Incoming messages are stored in a std::vector which we never shrink when
popping off elements. This means its capacity may be allowed to grow
indefinitely, which is bad.

This CL adds a check when popping messages off the queue,
calling shrink_to_fit() if the operation results in the queue crossing
an arbitrary size threshold (multiples of 512 elements).

Bug: 888665
Change-Id: Ie2b89a2104bcd6d3eece29c339e297cdadbd837b
Reviewed-on: https://chromium-review.googlesource.com/1241561
Commit-Queue: Ken Rockot <rockot@chromium.org>
Reviewed-by: Reilly Grant <reillyg@chromium.org>
Cr-Commit-Position: refs/heads/master@{#594104}
[modify] https://crrev.com/7fe00cd1270a395d3f1d66b14e41595a432d6ddc/mojo/core/ports/message_queue.cc

Owner: rockot@google.com

Sign in to add a comment