New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 730789 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Task
Hotlist-MemoryInfra



Sign in to add a comment

[cc] GpuImageDecodeCache::OnMemoryDump should exclude unlocked discardable memory from effective_size

Project Member Reported by trchen@chromium.org, Jun 7 2017

Issue description

This is a spin-off from https://crbug.com/721808 .

Had offline discussion with a few people who are familiar with discardable memory. We all agree that unlocked discardable memory is as good as freed from the kernel's perspective and it makes more sense to exclude it from measurements.
 
Cc: haraken@chromium.org ericrk@chromium.org
Owner: ----
I'd like to discuss a bit more widely.

primiano@, haraken@, do you have any thoughts on this? Might need to loop in Android / other experts. While I believe we've confirmed that unlocked discardable in a FG process will not contribute to killing that FG process, there are some less clear points:
- Does unlocked discardable in a FG proc contribute to killing BG procs?
- Does unlocked discardable in low-memory-Android contribute to pushing an app into zram?
- Is unlocked discardable on non-Android systems as "free" as it is on Android?

Maybe for regression / reporting purposes, it does make sense to switch discardable to only count locked-discardable in effective size. If this is the case, we should make the change globally, here:

https://cs.chromium.org/chromium/src/components/discardable_memory/client/client_discardable_shared_memory_manager.cc?rcl=e67fe485752ab3b2e21eeeaec2a0c155ad9b497d&l=73

WDYT?
So the thing is tricky here, because devil hides in details.
Discardable memory doens't map 1:1 with system primitives, there is a (chrome) layer int he middle. THe root problem is that, on Android at least, each discardable segment == 1 file descriptor, and file descriptors are a scarce resource.
So the tradeoff has been introducing a DiscardableMemoryManager which:
- Gets large (~2 MB IIRC) chunks of discardable memory from the OS
- Handles them to the various processes. The renderers handle a freelist
- deals with OS interaction. Eventually this also emulates the discardable part on OS where we don't have ephemeral semantics primitives.

Now, the details are very subtle. IIRC the chrome discardable manager marks a segment as unlocked only if all the elements in its freelist are unlocked. Until then it tries to manage memory at its best (IIRC doing madvise if there are large holes). So there is a layer of complexity involved on teh chrome side that makes reasoning on this extremely subtle.

At a 1st level of approximation, if we consider the chrome implementation fully trusted, I would agree with the statement that "switch discardable to only count locked-discardable". However that is something that we want to keep an eye on. If something goes wrong, that will inflate the PSS of the process and cause actual problems.

Now, going back to your question, assuming an ideal chrome implementation which maps 1:1 with android ashmem (discardable memory)
 
> - Does unlocked discardable in a FG proc contribute to killing BG procs?
IIRC bg process killing is based on a stack rank of processes prioritized by:
 1. OOM bucket (fg, bg, visible, perceivable, system, prev_activity)
 2. RSS modulated with some LRU heuristics 

So I am mostly (yet not 100%) confident that if ashmem is not dropped the system will not put too much effort into looking into how much RSS is due to ashmem.

Ideally before getting there (to the point of killing a process) the system should run the other vmscan shrinkers, which in turn would discard unlocked ashmem. But whether this is tuned  (and system tuning is something that each different OEM has the power to tweak) properly is something I never looked into and I wouldn't surprised if there are macroscopic failures in that logic.  
 
> - Does unlocked discardable in low-memory-Android contribute to pushing an app into zram?
Interesting question. pushing into zram should be handled entirely by the kernel. the ashem shrinker should kick in before. But again, if that is badly tuned and doesn't discard aggressively when it should, the other shrinker will kick in and move stuff to zram.

> - Is unlocked discardable on non-Android systems as "free" as it is on Android?
I don't know what backs discardable these days. I am mostly sure that on Linux is fully emulated. I remember that there were thoughts years ago to use new primitives for Windows, but I don't know if we ever used them. I have no idea about the status on mac os.

> Maybe for regression / reporting purposes, it does make sense to switch discardable to only count locked-discardable in effective size. 
Yeah my current though is that we should not count unlocked discardable within the context of individual CLs but we should keep an eye on discardable itself to make sure we it doesn't grow to suspicious values (that would indicate some regression in our discardable subsystem).
 
 
Primiano should be more familiar with the details, but overall I'm fine with excluding unlocked discardable from the effective size. We can keep track of the unlocked discardable size separately.

Status: Available (was: Untriaged)
Marking as available since this is discussion about a task.
Project Member

Comment 5 by sheriffbot@chromium.org, Jun 18 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
ericrk@, do you think there is follow up work to do here? Is this something still worth looking into?
Owner: primiano@chromium.org
Status: Assigned (was: Untriaged)
primiano: It sounds like you proposed some work here.  I'm assigning this to you to triage/prioritize not counting discardable memory in effective size.

Sign in to add a comment