To put my money where my mouth is, I propose that I start experimenting with this.

sebmarchand@ is actively working on a more realistic memory-pressure measure to use across platforms, with the intention that we can use it to drive mitigations of this sort - I'd suggest that we focus our efforts on that, and on mitigations based upon it, rather than trying to do something specific to ChromeOS, since this is a general problem across all swap-capable platforms.

Good, because I am not totally convinced my approach works :)

If Sebastian has any available info on his proposal, it would be good to see it so we can bikeshed it :)

Yes, if there's a doc I'd also be interested in seeing it.

That said, I still think we should look at the kernel notifier a little more, in particuler it probably won't hurt to trigger it more often. It's a feature that's specific to Chrome OS, so we can improve it separately of any platform-agnostic work.

FYI, I use the following memory pressure benchmark
Chrome perf - tab switching benchmark ( crbug.com/689388 )
 - Create multiple tabs and measure the tab switching time
 - Cross platform

I totally agree that the trigger from the kernel should be tuned per platform, and once Chrome received the low-memory notifier we'd like the reaction to be as platform independent as possible.

In addition, the ChromeOS swap size tuning should be per board too. Vovo has some preliminary data from Cyan (quad-core Celeron vs quad-core i5 on Caroline) and the tab switching time slowdown is more than 200% when both swap and discard are happening. we'd like to come up with a heuristic to schmoo the swap size using a benchmark and determine the optimal value per board.

My work will initially consist in understanding the relation between memory pressure (especially swap-thrashing) and performance (via UMA, UKM and Slow reports). The high-level goal is to "Create a high-fidelity thrashing signal that can be used to drive user-visible performance optimizations".

I'm initially focusing on the Windows platform but I want this to stay generic enough that it'll then be used on all platforms, I've a WIP document here (@google.com only): https://docs.google.com/a/google.com/document/d/1Y1_GwrPF_2XFTfw03zb3lXhbGmXwgY2sh01T55c_X2Y/edit?usp=sharing

I've opened crbug.com/771478 to track my work

The following steps can reproduce the persistent kswapd ~10%:

1. On caroline R63.9988.0
2. Open a lot of tabs, ex: open 144 tabs with the following command
  # chromium/src/tools/perf/run_benchmark --browser=cros-chrome --remote=DUT_IP cros_tab_switching.typical_24 --tabset-repeat=6 --pause-after-creation=100000
3. Manually switch tabs a few times
4. kswapd persistent occupy ~10% cpu

On step 4. 
a. the free memory is around 400MB
  - < 505MB(low memory watermark), so kswapd would wake up.
  - > 200MB (low memory notification threshold, /sys/kernel/mm/chromeos-low_mem/margin), so no low mem notification
b. kswapd can't free any memory
  - swap free: 0, can't swap
  - cache: ~100MB, (min filelist: 100MB), can't drop file cache

Since free > /sys/kernel/mm/chromeos-low_mem/margin, force sending low memory notification via /dev/chromeos-low-mem would not discard tabs. May need to raise the low mem margin to discard tabs.

The tab discard logic can also just follow the low-mem notification instead of doing its own calculations.

re #12 -- I'm not sure that's a good idea, there's a lot of cases observed in feedback where the notification fires, but by the time Chrome looks at the available memory level, the kernel has swapped enough out to bring free memory back above the threshold.  So, I worry that in those cases, we might just blindly discard and end up hurting users who once fit just fine in memory but now see more discards.

In any case, we need the metric to tell us how often this is occurring before we design something to deal with it.