New issue
Advanced search Search tips

Issue 823915 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Bug



Sign in to add a comment

Pair calls to MADV_FREE_REUSABLE with MADV_FREE_REUSE

Project Member Reported by erikc...@chromium.org, Mar 20 2018

Issue description

We switched from MADV_FREE to MADV_FREE_REUSABLE to fix accounting for memory regions that are being "discarded", or "decommitted".

https://bugs.chromium.org/p/chromium/issues/detail?id=708797#c29

Before reusing the regions, we should first call madvise(MADV_FREE_REUSE). Failing to do so creates a short-term accounting error:

If the region is immediately touched after the call to madvise(MADV_FREE_REUSABLE), e.g.
"""
madvise(a, size, MADV_FREE_REUSABLE);
 for (int i = 0; i < size; ++i)                                              
   a[i] = 'b';         
"""

then the accounting is not updated correctly, and the memory still shows up as "reusable". This will eventually be corrected by a special case in the page-out scanner, see VM_PAGEOUT_SCAN_HANDLE_REUSABLE_PAGE and xnu-4570.41.2/osfmk/vm/vm_pageout.c:3064.

Note: If the page is allowed to be discarded, and is then faulted, then all the accounting will work correctly.
Note: MADV_FREE_REUSE forwards to vm_object_reuse_pages(), whose effects are purely for accounting updates.

Note to self: Here's what happens under the hood for MADV_FREE_REUSABLE.
1) vm_object_deactivate_pages updates some accounting, and sets vm_page.dirty and vm_page.precious to FALSE. Clears VM_MEM_MODIFIED [eventually] using delayed work mechanism.
2) osfmk/vm/vm_pageout.c:3331 [pageout] will eventually notice the clean page and reclaim the page.

3) If the page is dirtied before it is paged out, then osfmk/vm/vm_pageout.c:3019 picks up VM_MEM_MODIFIED from the pmap and sets vm_page.dirty = TRUE, thus preventing the page from being reclaimed.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Mar 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/cf1b5ae213001436b8dd0923e6b8d5358babba33

commit cf1b5ae213001436b8dd0923e6b8d5358babba33
Author: Erik Chen <erikchen@chromium.org>
Date: Fri Mar 23 22:46:15 2018

Call MADV_FREE_REUSE in implementation of discardable shared memory.

This is required to correctly perform accounting of shared memory on macOS.

Bug:  823915 
Change-Id: I75a3bc4951e38bc24cc8c2b7867cfdd952fb860d
Reviewed-on: https://chromium-review.googlesource.com/974443
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Commit-Queue: Erik Chen <erikchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#545602}
[modify] https://crrev.com/cf1b5ae213001436b8dd0923e6b8d5358babba33/base/memory/discardable_shared_memory.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Apr 2 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f3acd3959823427070da1aff7f19f1a567ad1597

commit f3acd3959823427070da1aff7f19f1a567ad1597
Author: erikchen <erikchen@chromium.org>
Date: Mon Apr 02 22:22:05 2018

Update accounting on macOS for recommitted pages in partition alloc.

The syscall madvise(MADV_FREE_REUSE) is used to update accounting for
memory-usage, and is intended to be paired with madvise(MADV_FREE_REUSABLE).

Bug:  823915 
Change-Id: Iedbeebf58c1c8dd85ce25615d0ddcfc5eb1d4afb
Reviewed-on: https://chromium-review.googlesource.com/988194
Reviewed-by: Chris Palmer <palmer@chromium.org>
Commit-Queue: Erik Chen <erikchen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#547538}
[modify] https://crrev.com/f3acd3959823427070da1aff7f19f1a567ad1597/base/allocator/partition_allocator/page_allocator_internals_posix.h

Project Member

Comment 3 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/9a9fdbff04953ad19d32991fcf2a9e840f9a2d05

commit 9a9fdbff04953ad19d32991fcf2a9e840f9a2d05
Author: Erik Chen <erikchen@chromium.org>
Date: Wed Apr 18 15:02:47 2018

Fix accounting for reused memory for v8 on macOS.

Calls to madvise(MADV_FREE_REUSABLE) [when discarding/decommitting memory]
should be paired with calls to madvise(MADV_FREE_REUSE) [when reusing/committing
memory]. The latter is purely for accounting purposes.

Bug:  chromium:823915 
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
Change-Id: Ib1758fd72c5ad4dfe731f5d9a6dbaf75b1e0e14b
Reviewed-on: https://chromium-review.googlesource.com/988193
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52674}
[modify] https://crrev.com/9a9fdbff04953ad19d32991fcf2a9e840f9a2d05/src/base/platform/platform-posix.cc

Status: Fixed (was: Assigned)

Sign in to add a comment