yunlian, llozano: WDYT?

semenzato, erikchen: FYI

- it is good you are seeing that memory being swapped out because I think your concern was that, because it was in hugepages, it would never be swapped out. So, this is good. 
- It is not possible that that memory is never touched again. It is part of the executable instructions and the chrome code layout is not as good as to only touch 10 MB of memory. 
- There have been some recent discussions to add support on kernel for this optimization. Another large company contacted Google to see if there was interest in adding this to the kernel. So, as far as I know there are not new versions of the kernel that support this. Not sure if there will something coming out of those discussions (I don't drive them). 
- We have a couple of ideas to reduce the 30MB and we will try them next quarter. We try this before but caused problems with breakpad. Compiler is different now and linker will be different so we can revisit.

To be fair, I haven't confirmed that the swapped-out memory is the
executable; it may be some anonymous memory leak in the Zygote process.

Re "it is not possible that that memory is never touched again"; the Chrome
binary is considerably larger than 30MiB, and contains both the
Renderer-process code such as Blink, and Browser-side code such as the Aura
Shell and Views framework - if the 30MiB we copy to anonymous happens to
consist largely of browser-only code then we might well only touch part of
it in Renderers, surely? Really we're at the mercy of how the linker
chooses to lay out the binary, aren't we?

Can you clarify what you mean by "reduce the 30MB"? Do you mean having the
toolchain emit separate hot and cold sections and only copying the hot
section to hugepages, for example?

Re-describing this issue, since we don't have information to confirm that it's really huge-pages causing it.

+rsesek: Do we actually benefit from the zygote process under ChromeOS? Should we experiment with launching via exec_spawn or similar instead?

regarding #4

- from what I remember in the heatmaps, most of the 30MB of current layout were being exercised by the benchmarks (including page cyclers). So, I really doubt that those 20 MB are never used again.
- It is not totally true that we are at the mercy of the linker. With AFDO the compiler marks code as hot and linker will layout according to that. 

also about #4
forgot to answer last question
To reduce the 30 MB we have the following ideas
- revisit the splitting of cold and hot segments that we tried before (and broke breakpad). We are using a new compiler now and we will use a different linker soon too.
- Do some adjustments to the layout.. 
- improve layout of routines according to AFDO counts based on work from FB and Intel.

> +rsesek: Do we actually benefit from the zygote process under ChromeOS? Should we experiment with launching via exec_spawn or similar instead?

The zygote has two primary functions on Linux: one is to orchestrate sandboxing and the other is to have a safe place to fork(), since once Chrome is multithreaded that is not a safe way to start new child processes. I'm not familiar with exec_spawn, did you mean posix_spawn? If so, that could definitely be used for the second function, and it's what we use on Mac now. I'm less certain about the sandboxing aspect, since I'm not sure how expensive it would be for each child to jail itself.

Have you looked at where that 30MB is going? The zygote does preload various things that would be blocked by the sandbox after it's sealed. I'm wondering if it could be related to that? https://cs.chromium.org/chromium/src/content/zygote/zygote_main_linux.cc?q=zygote_main&sq=package:chromium&dr=C&l=388

Re #9: I'm observing this on a Production-mode ChromeOS devices, so I don't think I have access to tools like vmmap to look at the details of the process' memory footprint.  I'll try navigating a New Tab to chrome://crash to see if the resulting report provides any useful detail.

However, now that I'm looking at this on my device with two days of uptime and lots of process comings and goings over those two days, I notice that a New Tab process now starts with 34MB swapped-out, which suggests that this may be a gradual leak in the Zygote, that renderers then inherit.

Perhaps we should list the Zygote stats in Task Manager :)