New issue
Advanced search Search tips
Starred by 41 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
HW: ----
NextAction: ----
OS: All
Priority: 2
Type: Bug

Blocked on:
issue 7025


Show other hotlists

Hotlists containing this issue:
Hotlist-1


Sign in to add a comment
link

Issue 2869: Substring of huge string retains huge string in memory

Reported by adam.hoo...@gmail.com, Sep 3 2013

Issue description

Steps to reproduce:

  var s = "a huge, huge, huge string...";

  s = s.substring(0, 5);

Expected results: s takes five bytes of memory, plus some overhead.

Actual results: s takes a huge, huge, huge amount of memory.

Unfortunately, most String functions use substring() or no-ops internally: concatenating with empty string, trim(), slice(), match(), search(), replace() with no match, split(), substr(), substring(), toString(), trim(), valueOf().

My workaround is:

function unleakString(s) { return (' ' + s).substr(1); }

But it's not satisfying, because it breaks an abstraction and forces me to think about memory allocation.

Perhaps there should be some special logic for handling substring() when the output string length is less than, say, 0.125 times the input string length? that's not perfect, but in my opinion, the pros of this solution outweigh the cons.

This crops up constantly when scraping HTML.
 

Comment 1 by svenpanne@chromium.org, Sep 4 2013

Another approach not needing any kind of ad-hoc copy heuristic is to teach the garbage collector about projection functions. This is a well-known technique described e.g. in http://homepages.inf.ed.ac.uk/wadler/papers/leak/leak.ps.

Comment 2 by yangguo@chromium.org, Sep 4 2013

I briefly glanced the paper. It's not as easy:
- replacing a sliced string by a sequential string with same content usually uses up more memory
- in return, releasing the backing store of the sliced string frees up memory

It's really hard to tell whether flattening the sliced string is worthwhile or not, especially since we don't have a good way to track what strings refer to the same backing store. We had an attempt to flatten sliced strings when GC happens, two years ago, but was rejected due to the fact that it could cause an explosion on GC.

Comment 3 by svenpanne@chromium.org, Sep 5 2013

The point here is: On one hand, our sliced strings are an optimization for one kind of use cases (probably chosen from one of the "great" benchmarks we care about), on the other hand, sliced strings in their current naive form are a source of unlimited(!) space leaks for other use cases. So the question is: Can we mitigate the latter problem at least a bit? While a general solution might involve quite some work, solving easier cases might be relatively straightforward, e.g. when a single sliced string is the only reason for keeping another string alive.

We are not the first ones encountering this kind of problem, so there should be a variety of solutions already out there. Let's keep this issue open.

Comment 4 by dslomov@chromium.org, Nov 4 2014

Labels: Type-FeatureRequest Priority-Low
Owner: svenpanne@chromium.org
Status: Accepted
Sven, assigning to you since you suggested to keep the issue open :)

Comment 5 by svenpanne@chromium.org, Nov 20 2014

Cc: svenpanne@chromium.org
Owner: yangguo@chromium.org

Comment 6 by habl...@google.com, Apr 29 2015

Status: Assigned

Comment 7 by bit@google.com, Jul 30 2015

Comment 8 by ricardo....@gmail.com, Sep 13 2016

Guys, people are starting to propose hacks as workaround for this bug :(

https://github.com/mrdoob/three.js/issues/9679
https://github.com/mrdoob/three.js/pull/9680/files

Comment 9 by kbr@chromium.org, Sep 13 2016

Cc: -svenpanne@chromium.org kbr@chromium.org hpayer@chromium.org
Labels: -Priority-Low -Type-FeatureRequest OS-All Priority-Medium Type-Bug
This is affecting the most popular JavaScript library for drawing 3D graphics on the web. Per the above links, could this please be given some more thought? Thanks.

Comment 10 by kbr@chromium.org, Sep 13 2016

Components: Runtime Language

Comment 11 by adamk@chromium.org, Sep 13 2016

Components: -Language

Comment 12 by hablich@chromium.org, Mar 23 2017

Labels: Priority-2

Comment 13 by bmeu...@chromium.org, Nov 2 2017

Blockedon: 7025

Comment 14 by jkummerow@chromium.org, Mar 29 2018

Cc: yukishiino@chromium.org jkummerow@chromium.org adamk@chromium.org ishell@chromium.org vamshi.kommuri@chromium.org
 Issue chromium:822249  has been merged into this issue.

Comment 15 by bro...@gmail.com, Apr 2 2018

Add https://github.com/kelektiv/node-uuid/pull/267 to the list projects being asked to hack around this.

Comment 16 by zac.spit...@gmail.com, Apr 12 2018

another post about a workaround for this https://www.just-bi.nl/a-tale-of-a-javascript-memory-leak/

Comment 17 by david.vu...@gmail.com, Dec 17

Can someone please shed some light on whether there is a plan for how to deal with this issue?

This is not an academic matter, it is unexpected behaviour that leads to real bugs and requires creative workarounds. These serve only as temporary measures as it would be really easy to accidentally trigger such leaks again by adding a single line of code somewhere.

On my end, our use case is Node.js - latest LTS - and I could live with --no-string-slices and take whatever performance hit that might imply - however from what I can tell this option is either ignored or not working, depending on what version of V8 is used. It looks like this is a V8 issue, not a Node.js issue per se.

I'm not sure to what extent that approach would solve things for other people, but it would be a start.

Thank you.

Comment 18 by yangguo@chromium.org, Dec 17

--no-string-slices is a flag you need to set at build time. At runtime it becomes read-only.

Comment 19 by david.vu...@gmail.com, Dec 17

Would it be possible in some way to provide a runtime flag for toggling string slices? I'd like to know the situation as best I can before I talk to the Node.js folks.

Sign in to add a comment