New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 675911 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Allow snapshots of large heaps in DevTools

Project Member Reported by u...@chromium.org, Dec 20 2016

Issue description

Currently taking a snapshot of a large heap (close max heap capacity) results in an OOM crash.

There are multiple sources of OOM:
1. DevTools front-end uses the same max heap limit as an ordinary renderer process. We can fix it (at least on x64) by increasing the heap limit for V8 isolate in DevTools.

2. The heap snapshot generator in V8 stores edges in vector-like array (List<T>), which is limited to max 2GB size due to tcmalloc security constraint for chrome.

3. The heap snapshot generator in V8 uses 32-bit ints for indices and ids, which is unsafe for heap snapshots > 2GB. We can fix it by using size_t.

I can look into 1 and maybe 2.

alph@, pfeldman@, how hard would it be to fix 3?
 

Comment 1 by u...@chromium.org, Dec 20 2016

Cc: kozyatinskiy@chromium.org
Project Member

Comment 3 by bugdroid1@chromium.org, Dec 30 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/b00fc8be8a2ff527d8462bb9c0fff163bfce850f

commit b00fc8be8a2ff527d8462bb9c0fff163bfce850f
Author: ulan <ulan@chromium.org>
Date: Fri Dec 30 16:27:15 2016

Use std::deque for storing edges and children in heap snapshot.

This patch fixes OOM crash that happens for large heap where
the total size of edges exceeds 2GB, which is the hard limit
for v8::internal::List allocated using tcmalloc.

BUG= chromium:675911 

Review-Url: https://codereview.chromium.org/2595003002
Cr-Commit-Position: refs/heads/master@{#42004}

[modify] https://crrev.com/b00fc8be8a2ff527d8462bb9c0fff163bfce850f/src/api.cc
[modify] https://crrev.com/b00fc8be8a2ff527d8462bb9c0fff163bfce850f/src/profiler/heap-snapshot-generator-inl.h
[modify] https://crrev.com/b00fc8be8a2ff527d8462bb9c0fff163bfce850f/src/profiler/heap-snapshot-generator.cc
[modify] https://crrev.com/b00fc8be8a2ff527d8462bb9c0fff163bfce850f/src/profiler/heap-snapshot-generator.h
[modify] https://crrev.com/b00fc8be8a2ff527d8462bb9c0fff163bfce850f/test/cctest/test-heap-profiler.cc

Project Member

Comment 4 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/0959983c1afadeedfd9dad432a1eeccbde26dec3

commit 0959983c1afadeedfd9dad432a1eeccbde26dec3
Author: ulan <ulan@chromium.org>
Date: Wed Jan 11 13:20:53 2017

[heap, debugger] Introduce out-of-memory listener for debugger.

This API will allow DevTools to intercept out-of-memory condition,
increase the heap limit and schedule heap snapshot.

BUG= chromium:675911 

Review-Url: https://codereview.chromium.org/2621873003
Cr-Commit-Position: refs/heads/master@{#42225}

[modify] https://crrev.com/0959983c1afadeedfd9dad432a1eeccbde26dec3/src/api.cc
[modify] https://crrev.com/0959983c1afadeedfd9dad432a1eeccbde26dec3/src/debug/debug-interface.h
[modify] https://crrev.com/0959983c1afadeedfd9dad432a1eeccbde26dec3/src/heap/heap.cc
[modify] https://crrev.com/0959983c1afadeedfd9dad432a1eeccbde26dec3/src/heap/heap.h
[modify] https://crrev.com/0959983c1afadeedfd9dad432a1eeccbde26dec3/test/cctest/test-debug.cc

Project Member

Comment 5 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/18104fac86ed56ab21843e3e628f33aab92b1c4a

commit 18104fac86ed56ab21843e3e628f33aab92b1c4a
Author: ulan <ulan@chromium.org>
Date: Wed Jan 11 13:46:27 2017

[heap] Add API function for checking if the heap limit was increased
for debugging. This function is needed to pass increased heap limit
from the main DevTools isolate to the worker isolates it spawns.

BUG= chromium:675911 

Review-Url: https://codereview.chromium.org/2624973003
Cr-Commit-Position: refs/heads/master@{#42228}

[modify] https://crrev.com/18104fac86ed56ab21843e3e628f33aab92b1c4a/include/v8.h
[modify] https://crrev.com/18104fac86ed56ab21843e3e628f33aab92b1c4a/src/api.cc
[modify] https://crrev.com/18104fac86ed56ab21843e3e628f33aab92b1c4a/src/heap/heap.h
[modify] https://crrev.com/18104fac86ed56ab21843e3e628f33aab92b1c4a/test/cctest/test-api.cc

Comment 6 by alph@chromium.org, Jan 12 2017

 Issue 680789  has been merged into this issue.

Comment 7 by alph@chromium.org, Jan 12 2017

Copying info from 680789:

The snapshot is supposed to be stored in typed arrays. However while retrieving snapshot
the chucks go through main isolate before they get processed and discarded by the snapshot worker thread.

When the backend push rate for chunks is much higher than the processing rate, the chunks are accumulated in the main isolate heap and can cause OOM.




Screen Shot 2017-01-12 at 3.39.48 PM.png
700 KB View Download

Comment 8 by alph@chromium.org, Jan 13 2017

Regarding item #3 in the original report. I think using int32 is quite safe, as it's an index. provided that the minimum object size in the heap is 24 bytes, it's enough to represent objects in a heap of at least 48GB.

Switching to 64-bit indexes seems to be an overkill in terms of memory usage for the snapshot. 
Project Member

Comment 9 by bugdroid1@chromium.org, Jan 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/843960f748b26fbd707e1bf6e3a3f51bc201db6a

commit 843960f748b26fbd707e1bf6e3a3f51bc201db6a
Author: ulan <ulan@chromium.org>
Date: Fri Jan 13 18:29:22 2017

Make workers inherit the large heap limit of the main isolate.

This is mainly needed for DevTools, where the main isolate runs with
an increased heap limit. Workers spawned by DevTools should also run
with increased heap limits, otherwise they will OOM while inspecting
large heaps.

This patch replaces the V8CacheOptions field of WorkerThreadStartupData
with more generic WorkerV8Settings and propagates the heap limit state
from the main isolate to worker isolates.

BUG= 675911 

Review-Url: https://codereview.chromium.org/2627953005
Cr-Commit-Position: refs/heads/master@{#443619}

[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/bindings/bindings.gni
[add] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/bindings/core/v8/WorkerV8Settings.h
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/DedicatedWorkerTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/InProcessWorkerMessagingProxy.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/ThreadedWorkletMessagingProxy.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/ThreadedWorkletTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/WorkerThread.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/WorkerThreadStartupData.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/WorkerThreadStartupData.h
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/WorkerThreadTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/core/workers/WorkerThreadTestHelper.h
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/modules/compositorworker/AnimationWorkletThreadTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/modules/compositorworker/CompositorWorkerThreadTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/modules/serviceworkers/ServiceWorkerGlobalScope.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/modules/webaudio/AudioWorkletThreadTest.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/web/WebEmbeddedWorkerImpl.cpp
[modify] https://crrev.com/843960f748b26fbd707e1bf6e3a3f51bc201db6a/third_party/WebKit/Source/web/WebSharedWorkerImpl.cpp

Project Member

Comment 10 by bugdroid1@chromium.org, Jan 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1a56e240a23b3d01c57a9bf39f9296a45eced88c

commit 1a56e240a23b3d01c57a9bf39f9296a45eced88c
Author: ulan <ulan@chromium.org>
Date: Fri Jan 13 21:16:09 2017

Increase heap limit for DevTools isolate.

BUG= 675911 

Review-Url: https://codereview.chromium.org/2630453004
Cr-Commit-Position: refs/heads/master@{#443670}

[modify] https://crrev.com/1a56e240a23b3d01c57a9bf39f9296a45eced88c/third_party/WebKit/Source/web/WebDevToolsFrontendImpl.cpp

Cc: -u...@chromium.org
Owner: u...@chromium.org
Status: Fixed (was: Available)

Sign in to add a comment