Issue metadata
Sign in to add a comment
|
Fuzzer produces lots of gc output |
||||||||||||||||||||||
Issue descriptionRunning the wasm-compile-fuzzer locally, I get lots of this output: [...] [119221:0x62d000000400] 410949 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410956 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410964 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410972 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410980 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410990 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 410998 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411007 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411016 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411023 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411030 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411038 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [119221:0x62d000000400] 411048 ms: [Finished reentrant Mark-Compact during Mark-sweep.] [...] This hides other interesting output and slows dows fuzzing significantly, because my terminal eats a lot of CPU time :) Can we hide this output behind some flag?
,
Jul 13
In the WebAssembly fuzzers we do not trigger GCs explicitly. It seems like this behavior has been introduced by a recent change. Has there been a change in the GC which could have introduced it?
,
Jul 13
Nothing that I know of. There seem to be non-trivial workloads running from GC callsbacks which is not something the GC controls.
,
Jul 16
We hold some C++ data structures in Managed objects, so their constructor will be called during phantom handle disposal. If this frees a NativeModule, this will call {AdjustAmountOfExternallyAllocatedMemory}, hence can trigger another GC.
Can this be the cause of these messages?
If yes, I am still wondering why we need these messages.
,
Jul 16
The explanation makes sense. I didn't see that it was not behind any flag at all, so I will move it behind a GC tracing flag. The behavior you are seeing can be problematic though if NativeModules can be large and trigger synchronous GCs in the wild.
,
Jul 16
Thanks for taking care of this!
NativeModules can indeed be huge (several 100MBs). But {AdjustAmountOfExternalAllocatedMemory} should only trigger synchronous GC if we are under memory pressure, right?
,
Jul 16
There's a hard limit where we trigger a synchronous full GC, independently of whether memory pressure mode is set. Currently it is 50% of the old generation maximum size. That's why I am bringing this up. (The hard limit has been 256M for years until we moved to something different.) One intern project currently is building infrastructure for "trusted" external memory, i.e., external memory that is only reachable from within V8 where we know the exact graph. E.g., backing stores for arrays or strings where we know that there's only a single reference from within v8. We plan to integrate such memory in the v8 internal strategy which better adjusts to application behavior. If NativeModule fits the characteristics we can think of integrating that too.
,
Jul 16
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/0fb4f6a2aee8b020036bf6d142a408abc83cc30c commit 0fb4f6a2aee8b020036bf6d142a408abc83cc30c Author: Michael Lippautz <mlippautz@chromium.org> Date: Mon Jul 16 12:10:08 2018 [heap] Put print behind flag Bug: chromium:863362 Change-Id: I88896d7477d893f1b7fae08f6dfd5709748a6edd Reviewed-on: https://chromium-review.googlesource.com/1138080 Reviewed-by: Ulan Degenbaev <ulan@chromium.org> Commit-Queue: Michael Lippautz <mlippautz@chromium.org> Cr-Commit-Position: refs/heads/master@{#54461} [modify] https://crrev.com/0fb4f6a2aee8b020036bf6d142a408abc83cc30c/src/heap/gc-tracer.cc
,
Jul 17
This bug is fixed but the general behavior around this is still an open issue. Should we track this somehow?
,
Jul 17
We should definitly talk about this once you have a better infrastructure to track external memory. We hold the NativeModule in a Managed object, which knows an estimate of the memory it keeps alive, and calls {AdjustAmountOfExternalAllocatedMemory} on allocation and deallocation (in the second-pass phantom callback).
If there is a better way of doing it, please open a bug!
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by mlippautz@chromium.org
, Jul 13