The amount of a free memory isn't calculated properly on all OSes except Win
Reported by
mko...@yandex-team.ru,
Dec 9 2016
|
|||||
Issue descriptionUserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/602.2.14 (KHTML, like Gecko) Version/10.0.1 Safari/602.2.14 Steps to reproduce the problem: Linux, Mac: 1. Trash OS fs cache. Ex.: sudo grep -a smthing / 2. Check base::AmountOfAvailablePhysicalMemory() result and compare it with free/vmstat/vm_stat/Active Monitor output; 3. The function result is too small (near zero value). What is the expected behavior? What went wrong? Do the same on Win. The function result is Ok. Did this work before? No Chrome version: 57.0.2946.0 Channel: canary OS Version: Flash Version: Currently, there are 2 problems with free memory calculation: 1. It is calculated in different ways (It means different things under different OSes); 2. That numbers are almost useless on all OSes except Win. A bit detailed explanation. Modern OSes tend to use all free memory for disk/fs caches/buffers even if that data hasn't already been required by any app or kernel, or even if the data hasn't been requested yet (ex.: speculative reads on Mac). That's why the amount of a `pure` free memory tends to reach near zero value. And that's why base::AmountOfAvailablePhysicalMemory() returns near zero value in some minutes or hours after booting (depending on a workload pattern). ... On all OSes except Win. You can check it by trashing OS fs cache: grep -a smthing /. The amount of a disk/fs cache/buffer memory is divided into an active and inactive part. The last one isn't used by any app or kernel recently, that data can be easily re-read at any time on demand. That's why it is also free memory. And this is how base::AmountOfAvailablePhysicalMemory() works on Win. (I call it `soft` free memory). Allocating this amount of memory will not make a significant impact to the system. And even the active part of a file-backed memory can be treated as free memory: it can be freed at any time and can be re-read again from disk at any time if necessary. And allocating that whole amount of a free memory will not lead to a swapping at the moment of the allocation. But may lead to a swapping in the future. I call it `hard` free memory. Thus there are 3 meanings of the "free memory" term: `pure`, `soft` and `hard.` (I have doubts about this naming). Returning to the first statements: 1. On Win AmountOfAvailablePhysicalMemory() returns `soft` free memory, on other OSes - `pure` free. This isn't good; 2. Almost always we need to know `soft` free memory size. (`pure` can be used for some histograms/statistics, `hard` - for leak detection on ChromeOS ex.) I've made a CL. PTAL. I'm not sure you (the chromium team) are interested in all 3 implementations. If you aren't I'll rework the CL: the original AmountOfAvailablePhysicalMemory() will return `soft` free memory only on all OSes).
,
Dec 9 2016
,
Dec 12 2016
mkolom@: Could you please respond to the CL under review. Cc'ing reviewers and marking this as Untriaged to change the status accordingly once the CL is reviewed.
,
Mar 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a4258cad75d8cc63170555cc74bd584b5bfca965 commit a4258cad75d8cc63170555cc74bd584b5bfca965 Author: mkolom <mkolom@yandex-team.ru> Date: Tue Mar 21 03:13:26 2017 Fix free memory calculation. The term "free memory" can mean different things: it can mean 1) absolutely/pure free physical memory (not consumed by anything) or 2) it can include the inactive part(s) of OS (file) cache(s) or 3) it can mean the maximum amount of physical memory that can be allocated quickly without swapping and thus can also include the active part(s) of OS cache(s). This CL fixes base::AmountOfAvailablePhysicalMemory() to function according to the 2nd meaning on all platforms where it is possible because this is what everyone needs most of the time. (Win platform has already had such implementation but others haven't). BUG= 672745 Review-Url: https://codereview.chromium.org/2558043007 Cr-Commit-Position: refs/heads/master@{#458291} [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics.h [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics_ios.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics_linux.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics_mac.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics_unittest.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/process/process_metrics_win.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info.h [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_ios.mm [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_linux.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_mac.mm [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_openbsd.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_unittest.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/base/sys_info_win.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/chrome/browser/ui/webui/about_ui.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/components/memory_pressure/direct_memory_pressure_calculator_win.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/components/memory_pressure/direct_memory_pressure_calculator_win_unittest.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/content/browser/memory/memory_monitor_win.cc [modify] https://crrev.com/a4258cad75d8cc63170555cc74bd584b5bfca965/content/browser/memory/memory_monitor_win_unittest.cc
,
Mar 21 2017
,
Mar 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/ad039f65398a94f4a28751c303df388cff4bbca3 commit ad039f65398a94f4a28751c303df388cff4bbca3 Author: horo <horo@chromium.org> Date: Tue Mar 21 07:02:56 2017 Revert of Fix free memory calculation. (patchset #27 id:520001 of https://codereview.chromium.org/2558043007/ ) Reason for revert: SysInfoTest.AmountOfAvailablePhysicalMemory is failing BUG= 703522 Original issue's description: > Fix free memory calculation. > > The term "free memory" can mean different things: it can mean > 1) absolutely/pure free physical memory (not consumed by anything) or > 2) it can include the inactive part(s) of OS (file) cache(s) or > 3) it can mean the maximum amount of physical memory that can be > allocated quickly without swapping and thus can also include the > active part(s) of OS cache(s). > > This CL fixes base::AmountOfAvailablePhysicalMemory() to function > according to the 2nd meaning on all platforms where it is possible > because this is what everyone needs most of the time. (Win platform > has already had such implementation but others haven't). > > BUG= 672745 > > Review-Url: https://codereview.chromium.org/2558043007 > Cr-Commit-Position: refs/heads/master@{#458291} > Committed: https://chromium.googlesource.com/chromium/src/+/a4258cad75d8cc63170555cc74bd584b5bfca965 TBR=primiano@chromium.org,danakj@chromium.org,brucedawson@chromium.org,xiyuan@chromium.org,chrisha@chromium.org,haraken@chromium.org,mkolom@yandex-team.ru # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 672745 Review-Url: https://codereview.chromium.org/2766623002 Cr-Commit-Position: refs/heads/master@{#458333} [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics.h [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics_ios.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics_linux.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics_mac.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics_unittest.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/process/process_metrics_win.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info.h [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_ios.mm [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_linux.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_mac.mm [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_openbsd.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_unittest.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/base/sys_info_win.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/chrome/browser/ui/webui/about_ui.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/components/memory_pressure/direct_memory_pressure_calculator_win.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/components/memory_pressure/direct_memory_pressure_calculator_win_unittest.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/content/browser/memory/memory_monitor_win.cc [modify] https://crrev.com/ad039f65398a94f4a28751c303df388cff4bbca3/content/browser/memory/memory_monitor_win_unittest.cc
,
Mar 21 2017
It was better to disable the test.
,
Mar 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/01ac10b7938701bc9f9f2c05faf18229f9573b4c commit 01ac10b7938701bc9f9f2c05faf18229f9573b4c Author: mkolom <mkolom@yandex-team.ru> Date: Wed Mar 22 01:47:29 2017 Fix free memory calculation. The term "free memory" can mean different things: it can mean 1) absolutely/pure free physical memory (not consumed by anything) or 2) it can include the inactive part(s) of OS (file) cache(s) or 3) it can mean the maximum amount of physical memory that can be allocated quickly without swapping and thus can also include the active part(s) of OS cache(s). This CL fixes base::AmountOfAvailablePhysicalMemory() to function according to the 2nd meaning on all platforms where it is possible because this is what everyone needs most of the time. (Win platform has already had such implementation but others haven't). BUG= 672745 Review-Url: https://codereview.chromium.org/2558043007 Cr-Original-Commit-Position: refs/heads/master@{#458291} Committed: https://chromium.googlesource.com/chromium/src/+/a4258cad75d8cc63170555cc74bd584b5bfca965 Review-Url: https://codereview.chromium.org/2558043007 Cr-Commit-Position: refs/heads/master@{#458615} [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics.h [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics_ios.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics_linux.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics_mac.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics_unittest.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/process/process_metrics_win.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info.h [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_ios.mm [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_linux.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_mac.mm [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_openbsd.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_unittest.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/base/sys_info_win.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/chrome/browser/ui/webui/about_ui.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/components/memory_pressure/direct_memory_pressure_calculator_win.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/components/memory_pressure/direct_memory_pressure_calculator_win_unittest.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/content/browser/memory/memory_monitor_win.cc [modify] https://crrev.com/01ac10b7938701bc9f9f2c05faf18229f9573b4c/content/browser/memory/memory_monitor_win_unittest.cc
,
Apr 18 2018
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by mko...@yandex-team.ru
, Dec 9 2016