New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 621837 link

Starred by 2 users

Issue metadata

Status: WontFix
Owner:
OOO until 2019-01-24
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 2
Type: Bug

Blocking:
issue 621829



Sign in to add a comment

Excessive memory consumption when sending messages to a worker

Reported by spaceca...@prezi.com, Jun 21 2016

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

Steps to reproduce the problem:
1. Download the 3 attached files.
2. Select any 'method' by editing 'main.js'
3. Open index.html, it will try to start a worker so needs a local webserver or use '--allow-file-access-from-files' 

No matter which method from the attached 4 is used for populating a ByteArray used in postMessage() the memory consumption periodically grows to ~2GB as reported by the Activity Monitor. Profiling tools (Timeline, Heap Snapshot) only reports ~1GB.

We would like to implement our background job system in terms of a worker using the least amount of dynamic memory allocation as possible to avoid GC stalls. The main thread calls the worker per frame. The application is implemented in C++ and compiled with Emscripten.

What is the expected behavior?
No excessive memory allocation using method #2 or #3

What went wrong?
Initial idea was to use method #0 but as reported in  issue 169705 , this is not possible, so we wanted to use a preallocated buffer (method #2) as a target buffer for message serialization but this also shows excessive memory consumption. Even tried copying byte-by-byte in method #3 to rule out the a possible optimization by sharing the underlying buffer of TypedArray.set().

Did this work before? N/A 

Chrome version: 51.0.2704.103  Channel: stable
OS Version: OS X 10.9.5
Flash Version: Shockwave Flash 22.0 r0
 
main.js
1.0 KB View Download
worker.js
119 bytes View Download
index.html
126 bytes View Download
Some additional information:

The issue doesn't happen if the result buffer is not posted to the worker.

Also attached the output of $ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --no-sandbox --disable-extensions --disable-plugins --js-flags="--trace_gc --trace_gc_verbose" http://localhost:6931/index.html as gc_trace.log

And tried forcing GC after postMessage() which completely eliminated the memory expansion, logs in force_gc.log. External memory is reported around ~2GB but Activity Monitor reports constant ~1GB, difference is because of non-committed pages?

I suspect this is connected to TypedArrays using external memory and scavenge vs. mark-and-sweep heuristics, but I'm not familiar with V8's GC.
gc_trace.log
38.8 KB View Download
force_gc.log
4.2 KB View Download

Comment 2 by kbr@chromium.org, Jun 22 2016

Cc: hpayer@chromium.org u...@chromium.org eisinger@chromium.org
Components: -Blink Blink>JavaScript
Labels: -OS-Mac OS-All
Owner: kbr@chromium.org
Status: WontFix (was: Unconfirmed)
This test case is allocating C-heap memory with each postMessage call, because the typed array in the message is being copied. This memory is only reclaimed when garbage collection occurs on the worker's JavaScript context.

The correct way to implement a producer-consumer queue with a web worker that uses a bounded amount of memory is to transfer an ArrayBuffer from the main thread to the worker (using the postMessage(message, [transferables]) form -- see https://developer.mozilla.org/en-US/docs/Web/API/Worker/postMessage ), and transfer it back from the worker to the main thread for refilling using another postMessage call. A double-buffering technique can be used so that the main thread always has a buffer ready to fill.

I'm closing this as WontFix. The primitives exist in the platform to avoid unbounded memory consumption in cases like this.

Sure, the mentioned technique would be the best way, and that's what I wanted to use originally, but IE doesn't have Transferable and if possible we wanted to avoid platform specifics in our architecture if possible. Also Firefox doesn't show this behaviour.

I expected the C-heap memory to be deallocated much earlier as it's not referenced in the worker after the onmessage callback finishes, so it's not purely unbounded memory consumption. How come that the memory grows so much and no collection runs?

Comment 4 by jochen@chromium.org, Jun 22 2016

We're currently reworking the way c memory is handled, see issue 621829

But in general, I agree with Ken: there's an API for that
Yeah I found this issue, will keep an eye on it, thanks.

In theory I do agree as well, in practice it's not just black and white; as said we have to support every major browser and they behave differently (as expected?).

We will implement specific paths for using Transferables, but hope this issue will be fixed in the future with the mentioned rework.

Comment 6 by kbr@chromium.org, Jun 22 2016

Blocking: 621829

Sign in to add a comment