Issue metadata
Sign in to add a comment
|
Taking memory dump with chrome://tracing (with native profiling) is really slow |
||||||||||||||||||||||||
Issue descriptionI'm having hard time to grab a single memory dump by using chrome://tracing and activating only memory-infra. The UI is stuck for about 6-8 seconds before I can hit the stop button and stop tracing. It was introduced by https://crrev.com/2650863003. I looked a little bit in the code and the culprit seems to be: StackFrameDeduplicator::StackFrameDeduplicator For a browser with a single empty blank tab, there is about 467k unique stackframes and there is about 30k unique strings(StringDeduplicator::StringDeduplicator). My initial though was that the string duplicator is too huge, but it seems reasonable. The stackframe deduplicator is the one causing the latency and consuming a lot of memory. There is about 7 secondes between the first call to StackFrameDeduplicator::Insert and StackFrameDeduplicator::SerializeIncrementally which means it's taking 7 seconds to build the trie. The memory peak was observable on the speed waterfall. https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgjrDm8ggM https://chromeperf.appspot.com/group_report?keys=agxzfmNocm9tZXBlcmZyFAsSB0Fub21hbHkYgIDgjvfAugsM The same memory can be observed on the UI side (only loading a trace): https://bugs.chromium.org/p/chromium/issues/detail?id=736714 The trade-off for the memory dump V2 was to compress the format at the cost of a memory cost when generating or reading the trace. I'm curious about some trade-off. Can we keep a cache of stackframes (not being unique, but having a part of the compression gain)? Can we disable the stackframe deduplicator and keep the others deduplicator? We can disable it only for native heap profiling?
,
Jul 5 2017
I can repro on canary, not on ToT. Any clues?
,
Jul 5 2017
Windows 10, 64-bit
,
Jul 5 2017
Issue 735751 has been merged into this issue.
,
Jul 5 2017
This is also responsible for regressing total run time of memory.desktop benchmarks. From issue 735751 : " Total run time of memory.desktop benchmarks are doubled of many platforms (20 minutes --> 40 minutes). Bot(s) for this bug's original alert(s): chromium-rel-mac-retina chromium-rel-mac11 chromium-rel-mac11-air chromium-rel-mac11-pro chromium-rel-mac12 chromium-rel-mac12-mini-8gb chromium-rel-win10 chromium-rel-win7-dual chromium-rel-win7-gpu-ati chromium-rel-win7-gpu-intel chromium-rel-win7-gpu-nvidia chromium-rel-win7-x64-dual chromium-rel-win8-dual win-high-dpi "
,
Jul 5 2017
etienneb@ can you share unsymbolized traces from both Canary and ToT? I'm thinking maybe on Canary we have many more distinct stack traces, for some reason?
,
Jul 5 2017
I don't think this is related. I was able to repro last week with a ToT (not canary) and the ~600k stackframes was on chromium (not canary). I can still repro the behavior on canary, which it telling me we may fix something. Are we pruning the stackframe graph?
,
Jul 5 2017
Interesting. I'll check on macOS. We can't prune frames unfortunately, because until we symbolize we don't know if different addresses point to the same function.
,
Jul 6 2017
Huh, I never would have guessed this root cause. Nice investigation, dskiba and etienneb!
,
Jul 7 2017
OK, so I added some instrumentation and on macOS got the following:
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 25.3ms, stack deduplicator reports: insertions: 7506 (63% hits), new frames: 23339
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 25.717ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 357.769ms, stack deduplicator reports: insertions: 62887 (85% hits), new frames: 148690
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 359.378ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 95.945ms, stack deduplicator reports: insertions: 59109 (89% hits), new frames: 67433
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 97.23ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 742.685ms, stack deduplicator reports: insertions: 346669 (95% hits), new frames: 148825
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 751.206ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 326.535ms, stack deduplicator reports: insertions: 80297 (64% hits), new frames: 294529
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 331.922ms
[...trace_event_argument.cc(466)] TracedValue::AppendAsTraceFormat took 98.911ms, json string size: 1361382, output string size: 1365695
[...trace_event_argument.cc(466)] TracedValue::AppendAsTraceFormat took 1999.31ms, json string size: 25504152, output string size: 25506684
[...trace_event_argument.cc(466)] TracedValue::AppendAsTraceFormat took 3818.96ms, json string size: 20071594, output string size: 20074577
[...trace_event_argument.cc(466)] TracedValue::AppendAsTraceFormat took 1351.61ms, json string size: 8083385, output string size: 8088092
Two points here:
1. Stack deduplicator hits (i.e. when Insert() doesn't end up inserting anything) are common - I'm going to add additional map {backtrace -> frame_index} and see how it helps.
2. TracedValue::AppendAsTraceFormat() is really inefficient. See also issue 664350 where I found that ConvertTraceEventsToTraceFormat() (which indirectly calls AppendAsTraceFormat()) does ~2M allocations and recycles tens of megabytes of memory.
,
Jul 7 2017
> 2 TracedValue::AppendAsTraceFormat() is really inefficient. Sure, that mostly depends on the current tracing design and is going to be hard to change without changin that design. I don't think it can really do any better with the current JSON stuff. My only thought here is: TracedValue::AppendAsTraceFormat has always been in this state, but wasn't a problem for us before. Which to me suggests that a good portion of the problem is that we are trying to emit way too many objects compared to the past. Can we trim the heap dump within chrome (applying some threshold to drop branches < 4k or similar)? I think that will really help.
,
Jul 7 2017
Yes of course we now dump more stuff, and all inefficiencies start to show more.
And I think we can do better in TracedValue::AppendAsTraceFormat(): I found out about TraceEvent::AppendValueAsJSON() method and TRACE_VALUE_TYPE_XXX values, which mirror kTypeXXX values TracedValue uses. I.e. instead of first converting to base::Value and then to JSON, TraceEvent::AppendAsTraceFormat() could iterate over items and convert directly to JSON utilizing TraceEvent::AppendValueAsJSON() (adding {} and [] as necessary).
I think that we should address these inefficiencies first, and then consider re-adding thresholding on the Chrome side, because it'll kill two advantages of the new format: (1) simplicity on the Chrome side and (2) dumping of full information. Although I'm thinking of ways to implement thresholding, in parallel with optimization efforts.
,
Jul 7 2017
I optimized both stack deduplicator and AppendAsTraceFormat(), and the numbers went down ~4x:
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 210.988ms, stack deduplicator reports: {insertions=41030 (0% hits, 0% lookup hits), new frames=399169, lookups=72479 (0% collisions), lookup table size=41030, hashing time=22.07ms, matching time=25.47ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 228.212ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 13.169ms, stack deduplicator reports: {insertions=2770 (0% hits, 0% lookup hits), new frames=25008, lookups=4374 (0% collisions), lookup table size=2770, hashing time=1.77ms, matching time=1.78ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 13.705ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 117.105ms, stack deduplicator reports: {insertions=9219 (0% hits, 0% lookup hits), new frames=166731, lookups=58610 (0% collisions), lookup table size=9219, hashing time=14.10ms, matching time=30.14ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 120.356ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 80.987ms, stack deduplicator reports: {insertions=6248 (0% hits, 0% lookup hits), new frames=77546, lookups=52119 (0% collisions), lookup table size=15467, hashing time=12.04ms, matching time=22.29ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 82.562ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 402.235ms, stack deduplicator reports: {insertions=14681 (0% hits, 0% lookup hits), new frames=148455, lookups=336529 (0% collisions), lookup table size=55711, hashing time=67.43ms, matching time=141.46ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 414.164ms
[...heap_profiler_event_writer.cc(76)] VisitAllocations took 211.704ms, stack deduplicator reports: {insertions=30137 (0% hits, 0% lookup hits), new frames=329993, lookups=66398 (0% collisions), lookup table size=45604, hashing time=20.16ms, matching time=39.06ms}
[...heap_profiler_event_writer.cc(104)] SerializeHeapDump took 223.642ms
[...trace_event_argument.cc(565)] TracedValue::AppendAsTraceFormat took 29.483ms, json string size: 1434095, output string size: 1437535
[...trace_event_argument.cc(565)] TracedValue::AppendAsTraceFormat took 578.565ms, json string size: 28652613, output string size: 28654938
[...trace_event_argument.cc(565)] TracedValue::AppendAsTraceFormat took 742.019ms, json string size: 19775399, output string size: 19778396
[...trace_event_argument.cc(565)] TracedValue::AppendAsTraceFormat took 305.204ms, json string size: 8042281, output string size: 8045658
I.e. VisitAllocations + AppendAsTraceFormat time went down from 8817ms to 2480ms, almost 4x.
There is one more thing that I want to do, but first I'm going to cleanup and submit these two.
,
Jul 11 2017
I am suspecting that this is causing flaky failure of system_health memory benchmark in getting trace of the multi-tab case: https://chromium-swarm.appspot.com/task?id=3748e8f47efdd010&refresh=10&show_raw=1 RunBenchmark at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:397 expectations=expectations, metadata=benchmark.GetMetadata()) Run at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:260 _RunStoryAndProcessErrorIfNeeded(story, results, state, test) _RunStoryAndProcessErrorIfNeeded at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/story_runner.py:102 state.RunStory(results) traced_function at /b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:75 return func(*args, **kwargs) RunStory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/page/shared_page_state.py:314 self._current_page.Run(self) Run at /b/s/w/ir/third_party/catapult/telemetry/telemetry/page/__init__.py:112 self.RunPageInteractions(action_runner) RunPageInteractions at /b/s/w/ir/tools/perf/page_sets/system_health/system_health_story.py:143 self._Measure(action_runner) _Measure at /b/s/w/ir/tools/perf/page_sets/system_health/system_health_story.py:126 action_runner.MeasureMemory(deterministic_mode=True) traced_function at /b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:75 return func(*args, **kwargs) MeasureMemory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py:155 dump_id = self.tab.browser.DumpMemory() DumpMemory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/browser/browser.py:342 return self._browser_backend.DumpMemory(timeout=timeout) traced_function at /b/s/w/ir/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py:75 return func(*args, **kwargs) DumpMemory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:279 return self.devtools_client.DumpMemory(timeout=timeout) DumpMemory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/devtools_client_backend.py:402 return self._tracing_backend.DumpMemory(timeout=timeout) DumpMemory at /b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py:213 'request:\n' + traceback.format_exc()) TracingTimeoutException: Exception raised while sending a Tracing.requestMemoryDump request: Traceback (most recent call last): File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/tracing_backend.py", line 209, in DumpMemory response = self._inspector_websocket.SyncRequest(request, timeout) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 110, in SyncRequest res = self._Receive(timeout) File "/b/s/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_websocket.py", line 149, in _Receive data = self._socket.recv() File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 293, in recv opcode, data = self.recv_data() File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 310, in recv_data opcode, frame = self.recv_data_frame(control_frame) File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 323, in recv_data_frame frame = self.recv_frame() File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 357, in recv_frame return self.frame_buffer.recv_frame() File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 336, in recv_frame self.recv_header() File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 286, in recv_header header = self.recv_strict(2) File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_abnf.py", line 371, in recv_strict bytes_ = self.recv(min(16384, shortage)) File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_core.py", line 427, in _recv return recv(self.sock, bufsize) File "/b/s/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_socket.py", line 83, in recv raise WebSocketTimeoutException(message) WebSocketTimeoutException: timed out Locals: request : {'method': 'Tracing.requestMemoryDump', 'id': 0} timeout : 1200
,
Jul 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f43d398eac5c9d4c2bccda6d2630d108bbed1c99 commit f43d398eac5c9d4c2bccda6d2630d108bbed1c99 Author: dskiba <dskiba@chromium.org> Date: Thu Jul 13 23:44:19 2017 [tracing] Add heap profiler perftests. This CL creates a generic place for heap profiler perftests, and adds couple of stack frame deduplicator related perftests. BUG= 739378 Review-Url: https://codereview.chromium.org/2976843002 Cr-Commit-Position: refs/heads/master@{#486544} [modify] https://crrev.com/f43d398eac5c9d4c2bccda6d2630d108bbed1c99/components/tracing/BUILD.gn [add] https://crrev.com/f43d398eac5c9d4c2bccda6d2630d108bbed1c99/components/tracing/test/heap_profiler_perftest.cc
,
Jul 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/19cbc1d36c775115307c12cc99a67770ee108eff commit 19cbc1d36c775115307c12cc99a67770ee108eff Author: mmenke <mmenke@chromium.org> Date: Fri Jul 14 00:12:27 2017 Revert of [tracing] Add heap profiler perftests. (patchset #3 id:40001 of https://codereview.chromium.org/2976843002/ ) Reason for revert: This broke Win 64: https://build.chromium.org/p/chromium.win/buildstatus?builder=Win%20x64%20Builder%20%28dbg%29&number=56316 Original issue's description: > [tracing] Add heap profiler perftests. > > This CL creates a generic place for heap profiler perftests, and adds > couple of stack frame deduplicator related perftests. > > BUG= 739378 > > Review-Url: https://codereview.chromium.org/2976843002 > Cr-Commit-Position: refs/heads/master@{#486544} > Committed: https://chromium.googlesource.com/chromium/src/+/f43d398eac5c9d4c2bccda6d2630d108bbed1c99 TBR=primiano@chromium.org,dskiba@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 739378 Review-Url: https://codereview.chromium.org/2981833002 Cr-Commit-Position: refs/heads/master@{#486555} [modify] https://crrev.com/19cbc1d36c775115307c12cc99a67770ee108eff/components/tracing/BUILD.gn [delete] https://crrev.com/4920ad848a95e41761e9c0dd6a266c8510007c3d/components/tracing/test/heap_profiler_perftest.cc
,
Jul 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/484350a921288d2f41c025e7c114924d85007951 commit 484350a921288d2f41c025e7c114924d85007951 Author: dskiba <dskiba@chromium.org> Date: Fri Jul 14 12:32:37 2017 Reland of [tracing] Add heap profiler perftests. > Reason for revert: > This broke Win 64: > https://build.chromium.org/p/chromium.win/buildstatus?builder=Win%20x64%20Builder%20%28dbg%29&number=56316 > > Original issue's description: > > [tracing] Add heap profiler perftests. > > > > This CL creates a generic place for heap profiler perftests, and adds > > couple of stack frame deduplicator related perftests. > > > > BUG= 739378 > > > > Review-Url: https://codereview.chromium.org/2976843002 > > Cr-Commit-Position: refs/heads/master@{#486544} > > Committed: https://chromium.googlesource.com/chromium/src/+/f43d398eac5c9d4c2bccda6d2630d108bbed1c99 > > TBR=primiano@chromium.org,dskiba@chromium.org > # Skipping CQ checks because original CL landed less than 1 days ago. > NOPRESUBMIT=true > NOTREECHECKS=true > NOTRY=true > BUG= 739378 > > Review-Url: https://codereview.chromium.org/2981833002 > Cr-Commit-Position: refs/heads/master@{#486555} TBR=primiano@chromium.org BUG= 739378 Review-Url: https://codereview.chromium.org/2975133004 Cr-Commit-Position: refs/heads/master@{#486744} [modify] https://crrev.com/484350a921288d2f41c025e7c114924d85007951/components/tracing/BUILD.gn [add] https://crrev.com/484350a921288d2f41c025e7c114924d85007951/components/tracing/test/heap_profiler_perftest.cc
,
Jul 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/4136d111362c226d05204abb711b45bdaed48852 commit 4136d111362c226d05204abb711b45bdaed48852 Author: dskiba <dskiba@chromium.org> Date: Fri Jul 14 20:38:46 2017 [tracing] Add trace events to key heap profiler functions. This CL adds TRACE_EVENTs to several heap profiler-related functions to surface their timings. BUG= 739378 Review-Url: https://codereview.chromium.org/2980083002 Cr-Commit-Position: refs/heads/master@{#486874} [modify] https://crrev.com/4136d111362c226d05204abb711b45bdaed48852/base/trace_event/heap_profiler_event_writer.cc [modify] https://crrev.com/4136d111362c226d05204abb711b45bdaed48852/base/trace_event/heap_profiler_stack_frame_deduplicator.cc [modify] https://crrev.com/4136d111362c226d05204abb711b45bdaed48852/base/trace_event/heap_profiler_string_deduplicator.cc [modify] https://crrev.com/4136d111362c226d05204abb711b45bdaed48852/base/trace_event/heap_profiler_type_name_deduplicator.cc
,
Jul 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/3de608576703311817c69a8a64bdbc4feb87519b commit 3de608576703311817c69a8a64bdbc4feb87519b Author: dskiba <dskiba@chromium.org> Date: Sun Jul 16 17:46:06 2017 [tracing] Optimize StackFrameDeduplicator. Profiling StackFrameDeduplicator with new heap format revealed that: 1. At least 60% of all insertions are hits, i.e. they insert backtraces that already exist. 2. There can be as many as 500K frames. 3. The average child count at each level is ~1.2. This CL optimizes StackFrameDeduplicator based on that data: 1. Adds a hash-based backtrace lookup to cut down number of lookups in FrameNode::children maps. 2. Changes FrameNode storage from std::vector to std::deque, avoiding costly vector reallocations. 3. Changes FrameNode::children storage from std::map to std::flat_map. These changes improve HeapProfilerPerfTest.DeduplicateStackFrames perftest ~1.6x on macOS (1040ms -> 620ms), but real-world impact should be higher because the perftest is pessimistic. BUG= 739378 Review-Url: https://codereview.chromium.org/2977783002 Cr-Commit-Position: refs/heads/master@{#487015} [modify] https://crrev.com/3de608576703311817c69a8a64bdbc4feb87519b/base/trace_event/heap_profiler_stack_frame_deduplicator.cc [modify] https://crrev.com/3de608576703311817c69a8a64bdbc4feb87519b/base/trace_event/heap_profiler_stack_frame_deduplicator.h [modify] https://crrev.com/3de608576703311817c69a8a64bdbc4feb87519b/base/trace_event/heap_profiler_stack_frame_deduplicator_unittest.cc [modify] https://crrev.com/3de608576703311817c69a8a64bdbc4feb87519b/base/trace_event/memory_usage_estimator.h
,
Jul 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/131e21f45f5d7b04874b636c1ffa642c13944f1e commit 131e21f45f5d7b04874b636c1ffa642c13944f1e Author: dskiba <dskiba@chromium.org> Date: Sun Jul 16 19:41:39 2017 [tracing] Optimize TracedValue::AppendAsTraceFormat(). This CL optimizes TracedValue::AppendAsTraceFormat() by removing intermediate base::Value conversion. HeapProfilerPerfTest.AppendStackFramesAsTraceFormat shows ~1.9x improvement on macOS (1730ms -> 920ms). TBR=jbauman@chromium.org BUG= 739378 , 664350 Review-Url: https://codereview.chromium.org/2975033002 Cr-Commit-Position: refs/heads/master@{#487019} [modify] https://crrev.com/131e21f45f5d7b04874b636c1ffa642c13944f1e/base/trace_event/trace_event_argument.cc [modify] https://crrev.com/131e21f45f5d7b04874b636c1ffa642c13944f1e/base/trace_event/trace_event_argument_unittest.cc [modify] https://crrev.com/131e21f45f5d7b04874b636c1ffa642c13944f1e/cc/base/filter_operations_unittest.cc [modify] https://crrev.com/131e21f45f5d7b04874b636c1ffa642c13944f1e/components/tracing/common/graphics_memory_dump_provider_android_unittest.cc
,
Jul 17 2017
With the two optimizations above I still don't see meaningful drop in benchmark_duration in memory.desktop benchmarks. I'm going to look at either introducing collapsing on the Chrome side, or reverting new heap format CL.
,
Jul 18 2017
I found that I can't complete tracing of 5 tabs on macOS with the new format. This is crazy, so I'm reverting the new format for now: https://chromium-review.googlesource.com/c/575682 This will buy me some time to add (optional) coalescing of small backtrace branches to the new format.
,
Jul 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/dd1180754539805af312a7f2ac52f758d543226f commit dd1180754539805af312a7f2ac52f758d543226f Author: Dmitry Skiba <dskiba@chromium.org> Date: Thu Jul 20 10:12:37 2017 Revert "[tracing] Switch to new heap dump format." This reverts commit d4a5e98235da0cc8c4fb1ae2f67826b0b480ce4c. Reason for revert: regressed performance ( crbug.com/739378 , crbug.com/736714 ). The main cause is that new heap format produces a lot more stack frames in an attempt to dump all information in contrast with the old format, where small stack branches are coalesced together. I'll that functionality to the new format and reland it. > [tracing] Switch to new heap dump format. > > This CL switches tracing to a new heap dump format, that offers the > following advantages: > > 1. Dumps include all the information collected by Chrome's heap profiler. > > 2. The format is simpler and more compact. > > 3. The format can easily be extended to include additional per-entry data. > > 4. The format allows for post-processing (see recategorization from > crrev.com/2906413002 as an example). > > BUG= 708930 > > Review-Url: https://codereview.chromium.org/2650863003 > Cr-Commit-Position: refs/heads/master@{#480580} TBR=mark@chromium.org Bug: 708930 , 739378 , 736714 Change-Id: Ifc2b647aa1f57f4decae6e593ad032d72208b304 Reviewed-on: https://chromium-review.googlesource.com/575682 Commit-Queue: Dmitry Skiba <dskiba@chromium.org> Reviewed-by: Primiano Tucci <primiano@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Commit-Position: refs/heads/master@{#488186} [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/BUILD.gn [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_event_writer.cc [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_event_writer.h [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_event_writer_unittest.cc [add] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_heap_dump_writer.cc [add] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_heap_dump_writer.h [add] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_heap_dump_writer_unittest.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_serialization_state.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_serialization_state.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_stack_frame_deduplicator.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_stack_frame_deduplicator.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_stack_frame_deduplicator_unittest.cc [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_string_deduplicator.cc [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_string_deduplicator.h [delete] https://crrev.com/e773fb28d92d781058b58897b4f8638166fc5a45/base/trace_event/heap_profiler_string_deduplicator_unittest.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_type_name_deduplicator.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_type_name_deduplicator.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/heap_profiler_type_name_deduplicator_unittest.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/malloc_dump_provider.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/memory_dump_manager.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/process_memory_dump.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/process_memory_dump.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/process_memory_dump_unittest.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/sharded_allocation_register.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/sharded_allocation_register.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/trace_event_memory_overhead.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/base/trace_event/trace_event_memory_overhead.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/components/tracing/test/heap_profiler_perftest.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/third_party/WebKit/Source/platform/PartitionAllocMemoryDumpProvider.cpp [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/third_party/WebKit/Source/platform/heap/BlinkGCMemoryDumpProvider.cpp [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/third_party/WebKit/Source/platform/instrumentation/tracing/web_process_memory_dump.cc [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/third_party/WebKit/Source/platform/instrumentation/tracing/web_process_memory_dump.h [modify] https://crrev.com/dd1180754539805af312a7f2ac52f758d543226f/tools/gn/bootstrap/bootstrap.py
,
May 21 2018
Obsolete, as Chrome switched to memlog for heap profiling. |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by dskiba@chromium.org
, Jul 5 2017