New issue
Advanced search Search tips

Issue 732540 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

Allocate physical disk blocks for files written to by memory-mapped I/O on macOS

Project Member Reported by mark@chromium.org, Jun 12 2017

Issue description

On POSIX, MemoryMappedFile attempts to physically allocate disk blocks for the underlying file when arranging for writeable memory-mapped regions backed by files on disk. This is because, unlike traditional write()-based operations that fail with an error code on write error, failed writes to memory-mapped files result in a signal (SIGBUS) which is difficult to cope with. The easiest way for this to happen is when using memory-mapped I/O to write to a sparse file residing on a sparse filesystem. If a write would force new block to be physically allocated and there aren’t any free blocks, the operation will raise a signal.

https://chromium.googlesource.com/chromium/src/+/0413fd9f7c660ba7504dade7bb838862f138348e/base/files/memory_mapped_file_posix.cc#104

This was done in bug 715523, https://codereview.chromium.org/2860943005, e41b5ce8a877875575ad0c370bd4bdc8593113b2. macOS was excluded because its default (and only truly viable) filesystem, HFS+, doesn’t support sparse files. All files always have all blocks physically allocated.

In 10.13, Apple is declaring their new filesystem, APFS, ready for desktop use, and is making it the default. APFS does support sparse files.

The idea that sparse files can’t be encountered on macOS is no longer true. The POSIX MemoryMappedFile now needs to take this into account.

Unfortunately, the best way to implement this, posix_fallocate(), doesn’t exist on macOS. I filed https://openradar.appspot.com/32720223 requesting it.

In the mean time, we should enable the “need to worry about sparse files, but there’s no posix_fallocate()” fallback that we currently use for Android with API < 21, and for that matter, that we fall back to when posix_fallocate() fails. This mechanism is racy and could cause data loss, but shouldn’t be a practical problem in its usage in MemoryMappedFile. It’s also potentially slower than a posix_fallocate() system call would be, but if the system doesn’t provide it, then our hands are tied.
 
There's no realization of the file under Windows, either.  It hasn't generally been a problem on Desktop systems which typically have plenty of free disk space.

Using the "manual" extension method of writing a byte to every disk page might be okay as the main user of this (persistent metrics) now creates a "spare" file as a delayed background job to be used during the *next* run.  The file creation no longer affects startup time when that has happened.

Comment 2 by tapted@chromium.org, Jun 16 2017

Status: Available (was: Untriaged)
[mac triage] Taking out of triage queue. (mark/bcwhite: feel free to assign an owner).
It's really easy to do because the "manual" code is already there.  It just needs an ifdef to set a flag to "true" on whatever platform needs it.

The bigger question is...  Given the unlikelihood of a full disk on a mac, should we?

Comment 4 by mark@chromium.org, Jun 16 2017

Status: WontFix (was: Available)
Since we don’t do this on Windows either, we can skip this on macOS until we find ourselves blaming crashes on it.
If there comes a time when fallocate() becomes available, we should add that.

Sign in to add a comment