Linux sandbox memory limits are overly restrictive |
|||||||||||||||||
Issue descriptionSee also: https://crbug.com/752185 (Google-only). The Linux sandbox restricts allocations [0]; on 64-bit builds, address_space_limit to 16GB and new_data_segment_max_size to 8GB. These limits can easily be reached when web workers are involved: each worker creates a new Isolate, which reserves 512M of memory for code space. Sometime before creating the 32nd isolate / web worker, we hit the limit and crash with OOM. Real web apps have been running into this and crash reports are starting to trickle in ( https://crbug.com/792711 , https://crbug.com/798404 ). [0] https://chromium-review.googlesource.com/c/chromium/src/+/641992/3/content/common/sandbox_linux/sandbox_linux.cc
,
Jan 9 2018
FYI: palmer@ The only possible solution for ToT is to avoid enforcing virtual address space limits. AFAIK we only do that on Linux, which (personally) seems overly restrictive. The last time we evaluated that limit we only had other components (e.g. wasm) reserving address space in mind, so keeping the limit seemed fine. With the web worker use case in the mix this might change, or not? In future, V8 might impose a limit on JITed code which would allow us to reserve less space for the CodeRange (512M contiguous reservation needed for relative calls) which would also help in this case.
,
Jan 10 2018
,
Feb 12 2018
,
Feb 12 2018
,
Feb 12 2018
Why not just limit the number of webworkers to something like, 20. I think 20 is what firefox defaults to...?
,
Mar 6 2018
Issue 817441 has been merged into this issue.
,
Mar 6 2018
,
Mar 7 2018
This would be the same issue with issue 782982 .
,
Mar 8 2018
,
Mar 13 2018
,
Mar 13 2018
+ahaas, titzer as this also seems to be blocking successfully running the wasm spec tests on Linux.
,
Mar 14 2018
Agree with c#2. AFAIU Linux is the only platform on which we impose address space restrictions. Given how little of the userbase this impacts, do we still believe this is an effective security measure?
,
Mar 15 2018
+jschuh Ping palmer@
,
Mar 15 2018
I had expected this to be the case on all platforms, not just Linux. That might be a regression. +all the various OS experts from Platform Security: what do you think? The address_space_limit is 4 GiB, not 16 (the code lives in services/service_manager/sandbox/linux/sandbox_linux.cc now): ``` // Limit the address space to 4GB. // This is in the hope of making some kernel exploits more complex and less // reliable. It also limits sprays a little on 64 bits. rlim_t address_space_limit = std::numeric_limits<uint32_t>::max(); rlim_t address_space_limit_max = std::numeric_limits<uint32_t>::max(); ``` The comment explains why we do this, and as far as I know the reasoning and context behind haven't changed. (+jorgelo and tsepez) Might Site Isolation help here, because fewer origins will be creating workers/isolates in the same renderer process? Imposing a limit on JITted code would also be an excellent idea (and/or moving compilation into a separate process, but that's obviously a longer-term thing). 512 MiB "seems like a lot" (but maybe it's not). Anyway, we (Platform Security) will have a think. Assigning to myself to make sure it doesn't get lost.
,
Mar 15 2018
So, that's not actually right:
if (sizeof(rlim_t) == 8) {
// On 64 bits, V8 and possibly others will reserve massive memory ranges and
// rely on on-demand paging for allocation. Unfortunately, even
// MADV_DONTNEED ranges count towards RLIMIT_AS so this is not an option.
// See crbug.com/169327 for a discussion.
// On the GPU process, irrespective of V8, we can exhaust a 4GB address
// space under normal usage, see crbug.com/271119 .
// For now, increase limit to 16GB for renderer, worker, and GPU processes
// to accomodate.
...
address_space_limit = 1ULL << 34;
I still agree we need to harmonize the settings across platforms.
,
Mar 15 2018
On Mac we don't have the ability to limit the size of the address space via rlimit (issue 435269).
,
Mar 16 2018
We do limit also on Windows. It's done via the job object restrictions. https://cs.chromium.org/chromium/src/services/service_manager/sandbox/win/sandbox_win.cc?q=Sandbox_win&sq=package:chromium&l=561
,
Apr 16 2018
Issue v8:7635 has been merged into this issue.
,
May 3 2018
Issue 827627 has been merged into this issue.
,
May 4 2018
The following revision refers to this bug: https://chromium.googlesource.com/v8/v8.git/+/6b1457cfd6baa332ffa968b74fff705bac72e762 commit 6b1457cfd6baa332ffa968b74fff705bac72e762 Author: Hannes Payer <hpayer@chromium.org> Date: Fri May 04 13:47:09 2018 Reduce maximal code range size to 128M on Linux, Windows, and OSX for x64 and ia32. Bug: chromium:800348 , chromium:827627 , chromium:839750 Change-Id: I112e20b83eb1937476ebb4f30cf5679113759c0c Reviewed-on: https://chromium-review.googlesource.com/1044195 Reviewed-by: Michael Starzinger <mstarzinger@chromium.org> Commit-Queue: Hannes Payer <hpayer@chromium.org> Cr-Commit-Position: refs/heads/master@{#52996} [modify] https://crrev.com/6b1457cfd6baa332ffa968b74fff705bac72e762/src/globals.h
,
May 8 2018
,
May 17 2018
,
May 18 2018
Are there any plans for relaxing this restriction? It looks like current limits block pointer compression in V8.
,
May 18 2018
,
May 18 2018
re: #24 which specific restriction(s) are you talking about, and on which platforms? My understanding was that the intent behind pointer compression was to *reduce* memory usage (the design doc [1] says "35% of V8 heap reduction") so I am slightly confused why the memory limits would have to be increased? [1] https://docs.google.com/document/d/1TJTIYHNyG8KCCLQTu46VW3enRNtnC425eEobUp0eUQA/edit
,
May 18 2018
I'm sorry, I was not precise. IIUC this issue is about the fact that the sandbox may restrict the amount of reserved (and not yet committed) virtual space. Pointer compression aims at reducing the amount of committed memory and it requires the ability to reserve a contiguous 4Gbyte region of address space for each V8 isolate. Chrome creates one V8 isolate for the main thread and one V8 isolate per Web Worker instance. Not to mention Wasm which in addition reserves 8Gb of virtual address space for Wasm memory. #16 mentions 16Gb limit which doesn't seem enough for this pointer compression idea to fly.
,
May 31 2018
To what extent does the fix in #21 relieve the pressure? Given #27, it seems like essentially no limit would fly — 4 GiB isolates; and although currently there is no bound on the number of isolates, even 20 (as proposed in #6) would mean we'd need an 80 GiB limit at least. And we very much want the memory savings from pointer compression, especially since we're shipping and relying so much on Site Isolation (-> more memory usage) now. The code comment // This is in the hope of making some kernel exploits more complex and less // reliable. It also limits sprays a little on 64 bits. doesn't go far enough in my mind to justify the limit — we probably should have written real documentation for it when the limit was created. I don't know what particular kernel exploits that would protect against; and does it still matter in a post-KPTI world? And speaking of post-Spectre: heap spraying is a way to work around ASLR, but in a post-Spectre world (https://chromium.googlesource.com/chromium/src/+/master/docs/security/side-channel-threat-model.md) we're assuming an attacker has a sufficient infoleak to beat ASLR (in-process) anyway, even without any actual infoleak bugs. (And of course there are infoleak bugs.)
,
May 31 2018
The limit dates back to https://chromium.googlesource.com/chromium/src/+/a540e60eaaa7866abf0901b873ec6814d9cc81a9. Based on the referenced bugs ( issue 169327 and issue 157177 ), this may have been done for two reasons: 1) "a good defence to a know WebKit issue with ref count overflowing" 2) "mitigate 32 bits sizes on 64 bits architecture types of bug, we should prevent contiguous mappings of more than 4GB" Given that Linux is now 64-bit only, those reasons don't seem very relevant. Maybe jln@ can chime in?
,
Jun 1 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/683115b5f36ba1b80a1fbcd589a6b2a3ee075a88 commit 683115b5f36ba1b80a1fbcd589a6b2a3ee075a88 Author: Chris Palmer <palmer@chromium.org> Date: Fri Jun 01 16:50:31 2018 Document the sandbox memory limits marginally more. Future maintainers will need to know why and to what extent we think this stops exploits, and where the numbers came from. Bug: 800348 Change-Id: I8da01a89f290304aa99117676e1d9eb599a2f570 Reviewed-on: https://chromium-review.googlesource.com/1081391 Reviewed-by: Robert Sesek <rsesek@chromium.org> Commit-Queue: Chris Palmer <palmer@chromium.org> Cr-Commit-Position: refs/heads/master@{#563678} [modify] https://crrev.com/683115b5f36ba1b80a1fbcd589a6b2a3ee075a88/services/service_manager/sandbox/linux/sandbox_linux.cc
,
Jun 4 2018
Per rsesek, macOS honors RLIMIT_DATA, so we can and (I believe) should harmonize the policy across Linux and macOS in that way. We should define the limit in a single place so that if it changes (up or down), the platforms stay harmonized.
,
Jun 12 2018
,
Jun 13 2018
Issue 851626 has been merged into this issue.
,
Jun 18 2018
One reason for keeping a virtual address space limit is that it guards against exploiting the rowhammer bug via spraying page table entries (as described in https://googleprojectzero.blogspot.com/2015/03/exploiting-dram-rowhammer-bug-to-gain.html).
,
Jun 18 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b558ba514e82ec7624ab29733ca11cbc208b3d00 commit b558ba514e82ec7624ab29733ca11cbc208b3d00 Author: Chris Palmer <palmer@chromium.org> Date: Mon Jun 18 18:10:40 2018 [sandbox] Remove address space limits on Linux. Unfortunately, it does not appear possible to set RLIMIT_AS such that it will both (a) be high enough to support V8's and WebAssembly's address space requirements while also (b) being low enough to mitigate exploits using integer overflows that require large allocations, heap spray, or other memory-hungry attack modes. Bug: 800348 TBR: jln Change-Id: I4ec234709a4e49abde1c22f9c63a87bac107b8a0 Reviewed-on: https://chromium-review.googlesource.com/1097553 Commit-Queue: Chris Palmer <palmer@chromium.org> Reviewed-by: Robert Sesek <rsesek@chromium.org> Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Cr-Commit-Position: refs/heads/master@{#568073} [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/components/nacl/loader/sandbox_linux/nacl_sandbox_linux.cc [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/sandbox/linux/services/resource_limits.cc [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/sandbox/linux/services/resource_limits.h [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/sandbox/linux/services/resource_limits_unittests.cc [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/services/service_manager/sandbox/linux/sandbox_linux.cc [modify] https://crrev.com/b558ba514e82ec7624ab29733ca11cbc208b3d00/services/service_manager/sandbox/linux/sandbox_linux.h
,
Jun 18 2018
#34: Noted, thank you. Fundamentally, I don't think it's reasonable for probabilistic mitigations for probabilistic attacks to block legitimate application functionality, as was the case here. Especially when the root cause of the problem is several layers lower in the stack. Additionally, the limit before the CL landed in #35 was already higher than it sounds like the precondition was (reading the section "Kernel privilege escalation"), and Rowhammer still has other preconditions for success. I realize that Rowhammer research has advanced since that post, and there may now be fewer and/or less difficult preconditions for attack, but as far as I understand Rowhammer still probabilistic, highly variable, and ultimately fixed by ECC RAM. FWIW, we do intend to continue not using huge pages (https://bugs.chromium.org/p/chromium/issues/detail?id=806725).
,
Jul 5
Issue 782982 has been merged into this issue. |
|||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||
Comment 1 by jgruber@chromium.org
, Jan 9 2018