[memory-infra] Get OS memory information on Windows |
|||||||||||
Issue descriptionMemory-Infra on Windows doesn't use memory usage reported by the OS (private memory, shared memory, etc.) yet. As a result, the corresponding charts are flat: https://chromeperf.appspot.com/report?sid=cd105b396af1f05e3153586a5241ba4b7505bf3b2aa44dcd017ef7a364e27722. Starting point: https://cs.chromium.org/chromium/src/components/tracing/common/process_metrics_memory_dump_provider.cc?l=252
,
Jul 20 2016
,
Aug 11 2016
,
Aug 22 2016
* PSS is not a commonly used Windows concept, but I have written code to calculate it before. The sharing count is not perfect - the maximum count for number-of-sharers is seven - but in practice it works quite well. * Private dirty is something I'm not sure how to get on Windows, but I will investigate. * Total resident is easy to get * Swapped is a value that I would love to get but I have never found a way to do it. So, PSS and Total resident are easy and should be identical or very similar in meaning to Linux. Private dirty and swapped may not be possible. It should be fairly straightforward to fill in the base::trace_event::ProcessMemoryMaps structure, as long as the interpretations of the fields are well documented.
,
Aug 31 2016
I'll take this.
,
Aug 31 2016
Thanks for the thorough explanation Bruce. Yeah I remember that 7 (it's the ShareCount of PSAPI_WORKING_SET_EX_BLOCK, right?) , at some point I was looking at attempting the PSS calc on windows but eventually never got time to finish it. But looks like you have that code already :) Thanks a lot for the help stanisc@. The impl for Linux/Android is in components/tracing/common/process_metrics_memory_dump_provider.cc, happy to give help for the plumbing / reviews any memory-infra related question.
,
Oct 5 2016
,
Oct 5 2016
,
Nov 22 2016
,
Nov 25 2016
Issue 530236 has been merged into this issue.
,
Nov 30 2016
,
Dec 9 2016
,
Dec 12 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b172892773fe15a10508f733ca0bc831240a7ec6 commit b172892773fe15a10508f733ca0bc831240a7ec6 Author: chengx <chengx@chromium.org> Date: Mon Dec 12 06:55:56 2016 Add function to compute proportional set size for OS_WIN Memory-Infra on Windows doesn't use memory usage reported by the OS (proportional set size, private dirty, etc.) yet. This CL added functionality to calculate proportional set size for a process. BUG= 623499 Review-Url: https://codereview.chromium.org/2549803003 Cr-Commit-Position: refs/heads/master@{#437827} [modify] https://crrev.com/b172892773fe15a10508f733ca0bc831240a7ec6/base/process/process_metrics.h [modify] https://crrev.com/b172892773fe15a10508f733ca0bc831240a7ec6/base/process/process_metrics_win.cc [modify] https://crrev.com/b172892773fe15a10508f733ca0bc831240a7ec6/components/tracing/common/process_metrics_memory_dump_provider.cc
,
Dec 13 2016
The CL as stated in comment #12 calculates Proportional Set Size (PSS) for a process, rather than generating full mmaps. It is enough as far as the bots and chromeperf are concerned. The extra value of having the full mmaps is when developers pull the trace from the bots to investigate a regression and they happen to have more details. This can be a possible follow up for the future. I have checked the Chrome Performance Dashboard. The PSS numbers begin to show up and look reasonable. So I will mark this bug as fixed for now.
,
Jan 4 2017
|
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by primiano@chromium.org
, Jul 20 2016Catching up very slowly. Bruce, here's the deal: in memory-infra we have two categories of probes: 1. Probes that come from codebase instrumentation (v8 heaps, partition alloc stats, skia stats, etc etc). We call this "memory reported by Chrome" in chromeperf These work on all OS right now, including Windows. PRO: they are pretty stable (non-noisy) metrics and allow us to catch or investigate bugs. CON: as they come from manual instrumentations, they don't give us full coverage of the codebase (we might miss instrumentation). As a partial mitigation we have the "malloc" catch-all bucket ("winheap" in windows) which capture all the remaining heap stuff. Still, that is not a fully inclusive metric as one can use memory without using the heap (at least on Linux/Android) 2. What we call this "memory reported by OS". Think to this as a sort of "control group" which gives us feedback on the coverage we get by 1. Today we are missing this in Windows. This bug is about this. On Linux/Android specifically we track: - PSS (Proportional Set Size) which is the sum of (private pages) + (shared_pages / number of sharers). That accounts any kind of memory (anonymous and not, dirty and not) and is one of our "grand totals" to have a overall summary. Has the nice property that can be added sanely cross processes. - Private dirty: just the dirty pages (mostly == to anonymous memory + file mmaped and written) - Total resident [not a big deal]: similar to PSS, without the "/number of sharers part". The only reason being of this is that the kernel gives us also the peak value, which is useful to spot where critical situations happen. - Swapped [not a big deal]: the amount of memory moved which is not resident anymore and moved to the swap file. The question now is: what are the closest translations of these for Windows? How should we track them (code-wise)? From what I remember from your presentation (Life of a Windows memory page) there isn't a 1:1 mapping of those in windows land, because the memory model is not identical. It is fine. What we want to ultimately achieve here is having one (or a bunch) of known total counters that Windows folks are familiar with and that we are confident catches most of the memory usage.