New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 757747 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 770203
Owner:
Xoogler
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android
Pri: 2
Type: Feature
Hotlist-MemoryInfra

Blocked on:
issue 764357



Sign in to add a comment

Heap profiler performance testing

Project Member Reported by kraynov@chromium.org, Aug 22 2017

Issue description

TL;DR: We're making heap profiler performance telemetry benchmark. In order to compare performance the same page set has to be runned twice with and without heap profiler.

Proposed benchmark below.

What to do:
- Open about:blank (2-3 times to warm up the browser) and take a memory dump to the end
- Turn the heap profiler on (needs devtools plumbing https://chromium-review.googlesource.com/623813)
- Open about:blank and take a memory dump at the end

What to measure:
- CPU and wall time that takes to open about:blank (use existing trace events)
- Total memory (including heap profiler overhead) after having opened it

What to output:
- Relative (cpu-time%, wall-time%) of time to load about:blank (heap profiler on vs. off)
- Absolute memory overhead (heap profiler on vs. off)
- Time it takes to serialise the heap profiler dump (there're TRACE_EVENTs for that)
- Time that takes to import the trace
- Size of the events in the trace

More details:
[1] https://docs.google.com/document/d/1CMSaLtMc5NPqibmrSiTT9EaQmbu6bXxePAOmlAvslU8 (for Googlers)
 
Components: Internals>Instrumentation>Memory
First step:
Enable heap profiler in runtime.
[2] https://chromium-review.googlesource.com/623813

In order to utilise heap profiler in Telemetry test (Telemetry doesn't support derivative metrics and shares the state between pageset runs) we need a switch to enable it in runtime. It can be achieved by adding DevTools handler and Mojo dispatcher (within existing memory instrumentation service). Draft CL with DevTools handler (contains some unnecessary code, not intended for landing before [2]).

Comment 2 Deleted

Forgot the link to DevTools handler CL
[3] https://chromium-review.googlesource.com/613141
Cc: erikc...@chromium.org ajwong@chromium.org etienneb@chromium.org
+erikchen, +ajwong and +etienneb: I don't know what are the plans on the heap profiler, but maybe they might be interested in a similar API for other purposes (heap profiler from the field and/or controlling HP via an extension).
If this is not the case (they don't need anything like this), stop reading here.
If this is the case, let's make sure that this API fits our use cases.
From my side, the comments I have right now with my visibility of things are:
- Naming wise, as the memory instrumentation stuff gets more decoupled with tracing I'd probably put this API under the Memory namespace (i.e. next to Memory.setPressureNotificationsSuppressed) and not Tracing.
- Functionality wise let's just factor in the fact that in the near future we might want to selectively enable it: 
  - Only in one mode (pseudo stack, native stack, no-stack-only-tasks)
  - Only for a given process.

What else?

Awesome, glad to see someone working on this problem!

Note that out-of-process heap-profiling is O(weeks) away from being functional, at which point we will begin discussions to deprecate/turn down the existing heap profiler. That being said, the interface/behavior are quite comparable, so all the work should carry over. The only open question is whether you want to start working directly with the OOP HP, or whether you want to continue to work with HP for now.
we had an offline chat in LON.
Given that this has already slipped off the dealine and the fact that turning the heap profiler midway through seems to introduce extra complexity, the new plan is to simplify this in the following way:
have independent stories (think to ref builds) that test the same user story with and without the heap profiler (from the beginning, using cmdline). That will remove the need of introducing a devtools api and simplify the metric.
Downside: this will make the "with-heap-profiler" time-serie susceptible to any sort of change into chrome, so the way we'd be make a use of this data is:
1) Having only a very high threshold on regressions
2) using the dual-timeline overlay (again, think to ref builds, e.g. [1]) for a-posteriori human inspection.

This will still get the ability to bisect after the fact.


[1] https://chromeperf.appspot.com/report?sid=9fba0571188acb7b4a494a8c3d8c32812561a576cf9ba5d6a370c9ce807b5446&start_rev=486073&end_rev=497440
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c554a7cf816824b363cb9ee8d1b6f2476495632b

commit c554a7cf816824b363cb9ee8d1b6f2476495632b
Author: Greg Kraynov <kraynov@chromium.org>
Date: Fri Sep 01 21:27:20 2017

[memory-infra] MDM's EnableHeapProfiling returns result.

EnableHeapProfiling now returns a result and MemoryDumpManager
now exposes GetHeapProfilingMode. Some refactofing also made.
This change is the result of splitting that CL:
https://chromium-review.googlesource.com/623813

Bug:  757747 
Change-Id: I9d0b01b65c64373ea8af48b4485865161b63ac5f
Reviewed-on: https://chromium-review.googlesource.com/643389
Commit-Queue: Grigoriy Kraynov <kraynov@chromium.org>
Reviewed-by: Siddhartha S <ssid@chromium.org>
Cr-Commit-Position: refs/heads/master@{#499305}
[modify] https://crrev.com/c554a7cf816824b363cb9ee8d1b6f2476495632b/base/trace_event/memory_dump_manager.cc
[modify] https://crrev.com/c554a7cf816824b363cb9ee8d1b6f2476495632b/base/trace_event/memory_dump_manager.h
[modify] https://crrev.com/c554a7cf816824b363cb9ee8d1b6f2476495632b/base/trace_event/memory_dump_manager_unittest.cc

Project Member

Comment 8 by bugdroid1@chromium.org, Sep 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1e94989bc10b79261d67ccfeaf0e7e6308733747

commit 1e94989bc10b79261d67ccfeaf0e7e6308733747
Author: Siddhartha <ssid@chromium.org>
Date: Wed Sep 06 23:52:15 2017

Memory-infra: Heap profiler can be enabled when tracing

This is needed because new process starting when tracing and heap
profiler are enabled can get heap profiling notification later than
tracing enabled notification.
The heap profiler state should be setup when both tracing and heap
profiler are enabled. Do not call EnableHeapprofilingIfNeeded() from
constructor since the pseudo stack profiling needs to enable filtering
on TraceLog and MDM could be constructed by TraceLog in case of startup
tracing. This does not miss allocations since the heap profiling is
enabled by EnableStartupTracingIfNeeded very early (ContentMainRunner on
desktop and library loader on Android).

BUG= 757747 , 700245, 625170

Change-Id: I6fc593838dc1fa1c8450addefec782dd37ae9e5f
Reviewed-on: https://chromium-review.googlesource.com/651908
Commit-Queue: Siddhartha S <ssid@chromium.org>
Reviewed-by: Primiano Tucci <primiano@chromium.org>
Cr-Commit-Position: refs/heads/master@{#500138}
[modify] https://crrev.com/1e94989bc10b79261d67ccfeaf0e7e6308733747/base/trace_event/heap_profiler_serialization_state.h
[modify] https://crrev.com/1e94989bc10b79261d67ccfeaf0e7e6308733747/base/trace_event/memory_dump_manager.cc
[modify] https://crrev.com/1e94989bc10b79261d67ccfeaf0e7e6308733747/base/trace_event/memory_dump_manager.h
[modify] https://crrev.com/1e94989bc10b79261d67ccfeaf0e7e6308733747/base/trace_event/memory_dump_manager_unittest.cc
[modify] https://crrev.com/1e94989bc10b79261d67ccfeaf0e7e6308733747/base/trace_event/process_memory_dump.cc

Cc: kraynov@chromium.org picksi@chromium.org dskiba@chromium.org
 Issue 725174  has been merged into this issue.
Blockedon: 764357
Mergedinto: 770203
Status: Duplicate (was: Started)
Project Member

Comment 12 by bugdroid1@chromium.org, Oct 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3da60b74a4a5019c4eb4486ced60d671f0ace19a

commit 3da60b74a4a5019c4eb4486ced60d671f0ace19a
Author: Greg Kraynov <kraynov@chromium.org>
Date: Mon Oct 09 21:45:25 2017

[memory-infra] Enable heap profiling in runtime.

Introduce EnableHeapProfiling method in ClientProcess
interface of memory instrumentation service and type traits.

Bug:  757747 

Change-Id: I51c9b333996e9fcb9e7715769cc73c8349a7c3f9
Reviewed-on: https://chromium-review.googlesource.com/623813
Commit-Queue: Grigoriy Kraynov <kraynov@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Hector Dearman <hjd@chromium.org>
Reviewed-by: Siddhartha S <ssid@chromium.org>
Cr-Commit-Position: refs/heads/master@{#507499}
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/memory_instrumentation/coordinator_impl_unittest.cc
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/cpp/memory_instrumentation/client_process_impl.cc
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/cpp/memory_instrumentation/client_process_impl.h
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/cpp/memory_instrumentation/memory_instrumentation.typemap
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/cpp/memory_instrumentation/memory_instrumentation_struct_traits.cc
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/cpp/memory_instrumentation/memory_instrumentation_struct_traits.h
[modify] https://crrev.com/3da60b74a4a5019c4eb4486ced60d671f0ace19a/services/resource_coordinator/public/interfaces/memory_instrumentation/memory_instrumentation.mojom

Sign in to add a comment