Figure out mismatch with allocated_objects/effective_size on Windows |
||
Issue descriptionWe are observing that effective_size metric is reporting more objects than it supposed too. On Windows, the malloc dumper is computing the allocated memory by using a HeapWalk on the main process Heap. From that metric, telemetry is subtracting the memory used by other allocators and tracing. We prototyped an other "internal" metric which iterate through tracked objects and we are observing 25% - 33% memory space that is not tracked with MemoryInfra. We would like to know the blind spot MemoryInfra is having and figure out if this is fixable.
,
Apr 27 2017
I realized that the AllocationContextTracker is not keeping track of allocated objects within a scope gated with HEAP_PROFILER_SCOPED_IGNORE. These allocations are ignored by MallocDumpProvider::InsertAllocation MallocDumpProvider::RemoveAllocation The total number of allocated objects on Windows is calculated using the HeapWalk API (see WinHeapMemoryDumpImpl). This means the total number of allocated objects also includes the objects ignored by the profiler. Which explains the difference. The total number objects tracked by the malloc dump allocator (via shim): 104713 The total number of objects ignored by the malloc dump allocator due to HEAP_PROFILER_SCOPED_IGNORE: 43661 By adding these two numbers, we expect to found ~148374 objects allocated in the Heap. The HeapWalk is counting: 151921 objects. There is a gab of 3547 objects, which sounds reasonable.
,
Apr 28 2017
It'll be hard to estimate the number of objects allocated by tracing. We have "tracing" dump provider which subtracts the memory used by memory-infra from malloc. So, this does not show up on effective_size. But, we do not count the number of objects allocated by tracing to be able to subtract this number. Without heap profiling being enabled it will be very hard to calculate the actual number of allocated objects without tracing overhead.
,
Apr 28 2017
ssid #3, This is exactly what I observed and came to the conclusion: The |effective_size| (Malloc-only) has some noise in it because the memory dumper is allocating objects during the dump which are not taken into account when subtracting the tracing cost. Precision of the tracked objects/size metric is much more precise and stable but it required the SHIM to be activated. Which is not the case when shipping in the field. But could be a viable solution for unitest tests/regression/performance.
,
Sep 7 2017
|
||
►
Sign in to add a comment |
||
Comment 1 by etienneb@chromium.org
, Apr 24 2017