Umm, does renderers get memory pressure signals from the browser, or directly from android trim APIs? For some reason, I thought it was the former (to allow a "unified" signal for all of chrome)

(1) and 2() aren't really solvable afaik? there is no way to query the current "trim" level?
(3) was definitely a design decision, so debatable. But I wouldn't be surprised if there are things that should not be released when chrome is in the foreground that are tied to CRITICAL

The signal comes directly to the renderer from Trim APIs. The unified signal used to be implemented for desktop, but has been removed last year in favor of not sending the renderers any signals at all based on the fact that it seemed to thrash the system worse due to paging.

Also looks like there's ActivityManager#getMyMemoryState() as of API level 16 that returns interesting information like the last trim level dispatched and the lru ordering within a particular importance category. We should take a look at whether we can use this data.

Yup, I checked getMyMemoryState(), and it throws SecurityException "Isolated process not allowed to call getMyMemoryState" for renderers.

But we can simply remember last reported level ChromeApplication.

So the idea here is to report last seen level to all newly added callbacks? I think it makes sense. 

> So the idea here is to report last seen level to all newly added callbacks? I think it makes sense.

Not necessarily.. there isn't really any way to know if that last value is out of date or not. And that's a problem with getMyMemoryState as well.

Trim isn't a great API :/

No argument about the API :/

However, I do think we could cache received signals that happen before native is initialized and then send them as soon as that happens. While we don't know the info is still accurate, we can probably assume on a critical memory pressure system, not much would change....

We would want to keep a timestamp of course and ensure it's close enough (for all those cases where the application is randomly started by Android in background or services are being used, etc.)

Yeah, having a limit on the delay should be fine.

Then moving onto next problem/question. getMyMemoryState doesn't work in renderer. So is it ok for renderer to be using the browser trim level?

In my testing I see browser/renderer notified consequently. For example:

  683 I ActivityManager: Start proc 11854:com.google.android.apps.chrome/u0a82 for activity com.google.android.apps.chrome/.Main
11854 W MY-DBG  : ChromeApplication.onTrimMemory: 15
  683 I ActivityManager: Start proc 11888:com.google.android.apps.chrome:sandboxed_process0/u0i7 for service com.google.android.apps.chrome/org.chromium.content.app.SandboxedProcessService0
11854 W MY-DBG  : MemoryPressureListener.registerSystemCallback, lastTrimLevel=15
11888 W MY-DBG  : ChromeApplication.onTrimMemory: 15
11888 W MY-DBG  : MemoryPressureListener.registerSystemCallback, lastTrimLevel=N/A
...
11888 W MY-DBG  : ChromeApplication.onLowMemory
11888 W MY-DBG  : MemoryPressureListener.onLowMemory
11854 W MY-DBG  : ChromeApplication.onLowMemory
11854 W MY-DBG  : MemoryPressureListener.onLowMemory


So I think it's fine to send browser's trim levels to the foreground renderer.

Interesting, looks like 'lastTrimLevel' from getMyMemoryState() reflects current memory pressure. Here is the log of repeatedly calling getting it (21885 is the browser process):

02-20 23:27:06.854   683  5727 I ActivityManager: Start proc 21885:com.google.android.apps.chrome:sandboxed_process0/u0i19 for service com.google.android.apps.chrome/org.chromium.content.app.SandboxedProcessService0
02-20 23:27:06.936 21838 21838 W MY-DBG  : MemoryPressureListener.registerSystemCallback, lastTrimLevel=10
02-20 23:27:07.025 21885 21885 W MY-DBG  : ChromeApplication.onTrimMemory: 10
02-20 23:27:07.726 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:08.057 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:09.102 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:09.218 21885 21901 W MY-DBG  : MemoryPressureListener.registerSystemCallback, lastTrimLevel=N/A
02-20 23:27:09.976 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:10.126   683  5727 I ActivityManager: Start proc 21955:com.google.android.apps.chrome:sandboxed_process1/u0i20 for service com.google.android.apps.chrome/org.chromium.content.app.SandboxedProcessService1
02-20 23:27:10.292 21955 21955 W MY-DBG  : ChromeApplication.onTrimMemory: 10
02-20 23:27:10.325 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:10.627 21955 21971 W MY-DBG  : MemoryPressureListener.registerSystemCallback, lastTrimLevel=N/A
02-20 23:27:10.658 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
...
02-20 23:27:15.502 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:15.763   683  5727 I ActivityManager: Process com.google.android.apps.chrome:sandboxed_process1 (pid 21955) has died: vis  TOP 
02-20 23:27:15.788 21838 21838 W MY-DBG  : ChromeApplication.onTrimMemory: 15
02-20 23:27:15.788 21838 21838 W MY-DBG  : MemoryPressureListener.onTrimMemory: 15
02-20 23:27:15.811 21885 21885 W MY-DBG  : ChromeApplication.onTrimMemory: 15
02-20 23:27:15.812 21885 21885 W MY-DBG  : MemoryPressureListener.onTrimMemory: 15
02-20 23:27:15.839 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
...
02-20 23:27:25.508 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:25.587   683  4502 I ActivityManager: Start proc 22039:com.google.android.gms.ui/u0a16 for service com.google.android.gms/.chimera.UiIntentOperationService
02-20 23:27:25.811 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:25.961   683  4502 I ActivityManager: Start proc 22060:com.google.android.apps.searchlite/u0a22 for broadcast com.google.android.apps.searchlite/com.google.apps.tiktok.experiments.phenotype.ConfigurationUpdatedReceiver_Receiver
02-20 23:27:26.113 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
...
02-20 23:27:27.924 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:28.066   683  6063 I ActivityManager: Start proc 22109:com.google.android.apps.assistant/u0a4 for broadcast com.google.android.apps.assistant/.go.config.PhenotypeBroadcastReceiver
02-20 23:27:28.225 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:28.528 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:28.818   683  5727 I ActivityManager: Start proc 22135:com.google.android.apps.youtube.mango/u0a74 for broadcast com.google.android.apps.youtube.mango/com.google.android.libraries.internal.growth.growthkit.internal.experiments.impl.PhenotypeBroadcastReceiver
02-20 23:27:28.830 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
...
02-20 23:27:31.544 21838 21838 W MY-DBG  : Current lastTrimLevel: 15
02-20 23:27:31.830   683  3494 I ActivityManager: Start proc 22228:com.android.vending:instant_app_installer/u0a33 for broadcast com.android.vending/com.google.android.finsky.instantapps.PhenotypeUpdateReceiver
02-20 23:27:31.846 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
...
02-20 23:27:33.052 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:33.109   683  3494 I ActivityManager: Start proc 22279:com.google.android.calendar/u0a49 for broadcast com.google.android.calendar/com.google.android.apps.calendar.config.phenotypesupport.PhenotypeBroadcastReceiver
02-20 23:27:33.353 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:33.655 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:33.957 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:34.030   683  5727 I ActivityManager: Start proc 22312:com.android.providers.calendar/u0a5 for content provider com.android.providers.calendar/.CalendarProvider2
02-20 23:27:34.259 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:34.561 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:34.863 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
02-20 23:27:35.165 21838 21838 W MY-DBG  : Current lastTrimLevel: 10
...
02-20 23:27:44.229 21838 21838 W MY-DBG  : Current lastTrimLevel: 5


Note that lower values were not delivered to the app though. Chrome saw change from 10->15, but not 15->10->5.

that's... not what the docs say!! But if we can rely on that behavior, that makes things a lot easier

> So I think it's fine to send browser's trim levels to the foreground renderer.

what about background renderers though?

OK, so I dug through code, and here is what is happening:

* ActivityManager.getMyMemoryState() doesn't have any logic, it simply calls ActivityManagerService.getMyMemoryState() via IPC.

* ActivityManagerService calls enforceNotIsolatedCaller() to throw SecurityException if the request comes from an isolated process.

* Then ActivityManagerService copies ProcessRecord.trimMemoryLevel to RunningAppProcessInfo.lastTrimLevel.

* ProcessRecord.trimMemoryLevel is set by ActivityManagerService.updateOomAdjLocked(), which always changes trimMemoryLevel, but sends it to the app only when it goes up:

if (app.trimMemoryLevel < curLevel && app.thread != null) {
    app.thread.scheduleTrimMemory(curLevel);
    ...
}
app.trimMemoryLevel = curLevel;

And the code has been this way since JellyBean times.

> what about background renderers though?

I need to check, but on 512MiB Go devices there are no BG renderers. They are killed immediately as Android is almost always under critical memory pressure.

+haraken@, I remember your team was interested in getting current memory pressure on Android.

> > what about background renderers though?

> I need to check, but on 512MiB Go devices there are no BG renderers. They are killed immediately as Android is almost always under critical memory pressure.

Actually s/background renderer/renderer without IMPORTANT binding/, since you are still pushing for that change as well

Regarding TRIM_MEMORY_RUNNING_CRITICAL not being treated as CRITICAL:

* Initially (2013) the code only checked for TRIM_MEMORY_COMPLETE and reported everything else as MODERATE: https://chromiumcodereview.appspot.com/15995014/patch/60001/61003

* That was fixed a month later, and both >=TRIM_MEMORY_BACKGROUND && TRIM_MEMORY_RUNNING_CRITICAL were routed to MODERATE: https://chromiumcodereview.appspot.com/18908002/patch/1/2

rmcilroy@ do you remember why you routed TRIM_MEMORY_RUNNING_CRITICAL trim level to MODERATE memory pressure?


Note that onLowMemory() is treated as CRITICAL, and docs say that onLowMemory() is equivalent to onTrimMemory(TRIM_MEMORY_COMPLETE).

However, I can definitely see onLowMemory() being delivered to foreground browser/renderer processes on 512MiB Go device (sample log attached). Turns out the condition is more complex. As ActivityManagerService.doLowMemReportIfNeededLocked() explains:

// If there are no longer any background processes running,
// and the app that died was not running instrumentation,
// then tell everyone we are now low on memory.

doLowMemReportIfNeededLocked() was added in KitKat, and it still looks the same in Oreo.


So, given that CRITICAL memory pressure is already reported when Chrome is in the foreground I see no harm in treating TRIM_MEMORY_RUNNING_CRITICAL as CRITICAL too.

Deleted: onLowMemory.log 7.5 KB

onLowMemory.log 7.5 KB View Download

We also have MemoryAndroid.NotificationForeground histogram which includes both TRIM_MEMORY_RUNNING_CRITICAL and onLowMemory(), and TRIM_MEMORY_RUNNING_CRITICAL is far more frequent:

TrimMemoryRunningModerate       08.95%
TrimMemoryRunningLow            19.99%
TrimMemoryRunningCritical       60.20%
LowMemory                       10.86%

Which makes me think that in most cases we're killed before conditions for onLowMemory() are met.

(why can't the code just do what the docs say it does...)

Let me raise the priority of this bug -- it's critical to make sure that critical memory signals work on low-end Android properly.

> rmcilroy@ do you remember why you routed TRIM_MEMORY_RUNNING_CRITICAL trim level to MODERATE memory pressure?

That was a long time ago :). I don't remember off-hand, but if I remember correctly that CL was more about avoiding triggering memory pressure signals on TRIM_MEMORY_UI_HIDDEN, which was causing massive jank at the time. I think the reason TRIM_MEMORY_RUNNING_CRITICAL was routed to MODERATE was just to keep the behavior the same as it had been before the change.  I believe we also used to do more memory saving work on MODERATE than we do now.

FWIW, I'm in favor of routing TRIM_MEMORY_RUNNING_CRITICAL to a CRITICAL memory pressure signal.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c7edeb5488edd5f07f7104336f10a1b645c8b8de

commit c7edeb5488edd5f07f7104336f10a1b645c8b8de
Author: Dmitry Skiba <dskiba@chromium.org>
Date: Mon Feb 26 20:08:57 2018

Treat TRIM_MEMORY_RUNNING_CRITICAL as CRITICAL memory pressure.

When Chrome is in the foreground, it treats onLowMemory() as
CRITICAL pressure and ignores TRIM_MEMORY_RUNNING_CRITICAL
trim level.

Turned out this is an oversight, and in fact according to UMA
RUNNING_CRITICAL is delivered 6 times more frequently than
onLowMemory(). See the bug for more details.

This CL treats RUNNING_CRITICAL trim level as CRITICAL pressure,
and adds tests.

Bug: 813909
Change-Id: I7e6d650881ed6e67b171f2cb29d593fc1b4acd3b
Reviewed-on: https://chromium-review.googlesource.com/935669
Reviewed-by: Richard Coles <torne@chromium.org>
Commit-Queue: Dmitry Skiba <dskiba@chromium.org>
Cr-Commit-Position: refs/heads/master@{#539247}
[modify] https://crrev.com/c7edeb5488edd5f07f7104336f10a1b645c8b8de/android_webview/glue/java/src/com/android/webview/chromium/SharedStatics.java
[modify] https://crrev.com/c7edeb5488edd5f07f7104336f10a1b645c8b8de/base/BUILD.gn
[modify] https://crrev.com/c7edeb5488edd5f07f7104336f10a1b645c8b8de/base/android/java/src/org/chromium/base/MemoryPressureListener.java
[add] https://crrev.com/c7edeb5488edd5f07f7104336f10a1b645c8b8de/base/android/javatests/src/org/chromium/base/MemoryPressureListenerTest.java

I've documented all places that use MemoryPressureListeners here: https://docs.google.com/spreadsheets/d/1tAJGcp2T6RFyag4zFNAOnpmuztTrz0WRNES5pBS9GTs/edit?usp=sharing

Some observations:

* Most places are stateless.

* Most places react differently on MODERATE / CRITICAL levels.

* There are 2 places which are stateful, i.e. last reported memory pressure affects future behavior.


My takeaways:

* It doesn't make sense to report pressure frequently, especially CRITICAL level. We won't get much memory back (since the last signal was not so long ago), and we can affect performance negatively (if we clear cached data that is being used).

* We need to report CRITICAL -> MODERATE transitions, because that affects stateful listeners.

I also took a closer look on what trim levels are reported. Observations:

* onTrimMemory() can sort of go back and report lower values. I was playing with backgrounding / foregrounding Chrome and got the following:
  TRIM_MEMORY_RUNNING_CRITICAL (right after start)
  -> TRIM_MEMORY_UI_HIDDEN (backgrounded)
  -> TRIM_MEMORY_COMPLETE (after being in bg for some time)
  (foregrounded, but no pressure was reported)
  -> TRIM_MEMORY_UI_HIDDEN (backgrounded)
  -> TRIM_MEMORY_COMPLETE (several times while backgrounded)
  (still in bg, getMyMemoryState().lastTrimLevel went TRIM_MEMORY_COMPLETE -> TRIM_MEMORY_RUNNING_CRITICAL -> TRIM_MEMORY_RUNNING_LOW)
  (foregrounded, started a renderer, it got TRIM_MEMORY_RUNNING_LOW, but not the browser)
  -> TRIM_MEMORY_RUNNING_CRITICAL (delivered to both renderer and browser)

* getMyMemoryState().lastTrimLevel never reports TRIM_MEMORY_UI_HIDDEN.

* onLowMemory() is sometimes called for the browser in the foreground, but not for its only renderer.

* Renderer gets onTrimMemory(TRIM_MEMORY_COMPLETE) every time I close its tab. Probably because we're dropping some binding.


My takeaway is that Android memory pressure reporting is a mess, and while it somewhat works, we can't rely on it.

Thanks Dmitry for looking into this!

The current situation is a kind of mess... People interpret MODERATE / CRITICAL in different meanings. CRITICAL pressures are not working as expected in the first place.

I'm even fine with dropping MODERATE states for now (i.e., just use NONE / CRITICAL). Our top priority would be to get the CRITICAL signals to work correctly.

You can implement a signal for NONE -> CRITICAL and CRITICAL -> NONE.

Re #27: Yeah, due to the complexity, I'd prefer focusing on delivering CRITICAL signals at a right timing (as much as we can). We can worry about more details (e.g., MODERATE) later :)

> My takeaway is that Android memory pressure reporting is a mess, and while it somewhat works, we can't rely on it.

This was my observation too when I was implementing memory coordinator / oom intervention. That's why I stopped using memory pressure signals from the OS.

FWIW, onTrimMemory()/onLowMemory() would be working well for most other apps. Chrome is unique as it has two high-priority processes (browser and renderer). That makes the OS difficult to deliver memory pressure signals accordingly.

I'm almost done with a CL that does the following:

* MemoryPressureListener polls (and reports) (using getMyMemoryState) pressure once when it's initialized (registerSystemCallback is called from native).

* Polling continues as long as memory pressure remains CRITICAL. The interval for polling is 60 seconds.

* Once polling gets MODERATE (or NONE) pressure, it stops (reporting last MODERATE).

* Once CRITICAL is reported by Android, we start polling again.

* We stop polling when we go to the background.

* We poll once when we go to the foreground. (Polling will continue if we're under CRITICAL pressure).

* Additionally, reports to native are throttled with the same 60 second interval. I.e. no matter how many times pressure is received (via polling / directly from Android) it'll be reported at most once per 60 seconds to native.


There are two things I don't like about all this:

1. It's pretty complex.

2. It relies on Android reporting us CRITICAL pressure to start polling.


I'm wondering if we should just stop listening to onTrimMemory() / onLowMemory() signals and exclusively rely on polling:

* When MemoryPressureListener is initialized, it starts polling.

* Polling stops when we go to the background. (We can still deliver single CRITICAL pressure here.)

* Polling starts when we go to the foreground.

* CRITICAL signals are always reported.

* MODERATE signals are reported only on NONE->MODERATE, CRITICAL->MODERATE transitions.

* Browser distributes pressure signals to renderers (which can't call getMyMemoryState because they are sandboxed).

I have two concerns wrt polling only:
1) We may have to poll quite often in foreground in order to get memory signals in time for them to be useful. How expensive is it to call getMyMemoryState?
2) I am not sure that getMyMemoryState works the same way on other OEM devices (since the docs say something quite different from reality)

Yeah, as maria mentioned, if we want to rely on polling only, we need to increase the interval (e.g., 2 seconds). Would it be acceptable?

If yes, I'm fine with it. If no, we'll need to use onTrimMemory/onLowMemory.

I'm doubtful that the OS memory signals are useful:

The OS signals are not actually based on amount of RAM available to the foreground app, but instead use the number of alive background processes as a proxy. If there are a many small background apps, then we won't get the signals will be good even when we're almost out of memory. If there is one large background app near the top of the LRU, we would get critical signals even before it gets killed are releases a large amount of memory.

Having Chrome always look at isLowMemoryDevice() has been fairly successful so far (I think?). 

If we truly want to try signals, maybe we should just look at chrome_usage / total_ram?

Although... it just occurred to me that on the high end, e.g. Samsung Dex, where many apps can be VISIBLE at the same time, it may actually make sense to use the "how many background apps are running" signal. 

Perhaps use these only for high-end devices? Or for when UiMode != phone?

The Tokyo team did an investigation into using other system signals for estimating critical memory pressure and the TL;DR; is that what the platform supplies is bad, but everything else we can compute at runtime is not really any better. And at the very least it tells us interesting things because the system acts based on these signals. E.g. we don't know how close we are to oom, but we know if we get onLowMemory we're about to be in a world of hurt because no background apps are left.

The CL is up: https://chromium-review.googlesource.com/c/chromium/src/+/953166

Comments are welcome!

BTW, the CL above only handles browser process, there will one more CL to pipe pressure polled by the browser process into the foreground renderer.

In the CL polling stops when Chrome goes to the background. The idea was that there is no point in reducing memory usage in that case (since Android will likely simply kill us if it needs memory). So I used ApplicationStatus to listen for app state changes.

But I learned that despite being in base/ ApplicationStatus is not supposed to be used outside Chrome because it doesn't make sense for WebView (which makes sense). There is an issue 470582 to remove it from base.

So I proposed to have several modes of MemoryListener operation, so that WebView continues to receive pressure signals exactly how it does now (via ComponentCallbacks2, without throttling or polling).

But Bo noted that this means that memory listeners would work differently in WebView than they are in Chrome. They (1) won't be called periodically, and (2) won't notify listeners about subsided pressure.


I think the only way out here is to poll always, regardless of app state. This simplifies the algorithm and it becomes:

1. When initialized, check pressure once using getMyMemoryState().

2. Keep checking while pressure is CRITICAL. (This also means that we'll start checking if we get CRITICAL from ComponentCallbacks2.)

3. Throttle reports to native (once per minute), but make sure CRITICAL->MODERATE and CRITICAL->NONE transitions are delivered.

4. (Next CL) Distribute pressure signals from browser / WebView to all their renderers (because isolated processes can't call getMyMemoryState()).


Per process type:

* Browser / WebView: listens to ComponentCallbacks2, throttles, polls getMyMemoryState(), distributes pressure to renderers

* GPU: listens to ComponentCallbacks2, throttles, polls getMyMemoryState()

* Renderer: listens to ComponentCallbacks2, throttles, receives pressure signals from the browser process

* WebView renderer: listens to ComponentCallbacks2, throttles, receives pressure signals from WebView process


What do you think?

> I think the only way out here is to poll always, regardless of app state.

I think we should not poll forever.

We can probably make webview work in with the original proposal (of chrome using ApplicationStatus). At minimum, can use AwContentsLifecycleNotifier and stop polling when all webviews are gone: https://cs.chromium.org/chromium/src/android_webview/java/src/org/chromium/android_webview/AwContentsLifecycleNotifier.java?rcl=8ec2a06321af1ec8b42cecd1df8f29d254b5b52b&l=24

We can be more sophisticated and stop polling when all webviews are "invisible", for some definition of invisible that were discussed offline.

In the CL Bo raised the following point:

Previously when Android reported MODERATE and then CRITICAL these signals were delivered to the native side right when they were reported. My CL adds 60 second delay between them (throttling interval). The concern is that by delaying cleanups that happen on CRITICAL signal we increase likelihood of Chrome being killed (or simply make Android work harder to recover memory from elsewhere).

So the idea is to report worsening pressure right away, but throttle in all other cases.

I can see Bo's point, but I think that it's easier to reason about pressure listeners when they are called at some fixed interval. I.e. right now the interval is 60 seconds, and that number is completely arbitrary. We'll need to think of a way to figure out the "right" interval, and it'll be helpful to know that there is no skew because of MODERATE->CRITICAL transitions.

What do folks think about this?

During codereview boliu@ raised a point that special handling of NONE complicates implementation, since NONE is not supposed to be reported. I filed issue 828057 to change that.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5baa1f510464d09419514f8e41b15482a46078db

commit 5baa1f510464d09419514f8e41b15482a46078db
Author: Dmitry Skiba <dskiba@chromium.org>
Date: Wed Apr 11 08:08:44 2018

Improve memory pressure reporting on Android.

This CL overhauls the way memory pressure is sensed and reported for
the browser process. Main changes are:

* ActivityManager.getMyMemoryStat() is used to poll pressure.

* Pressure signals are throttled.

* CRITICAL->MODERATE pressure changes are always reported.

See MemoryPressureMonitor.java comments for details.

Bug: 813909
Change-Id: I6fb5395b175cf8ca4ac40fac021125dd9d7fbc9f
Reviewed-on: https://chromium-review.googlesource.com/953166
Commit-Queue: Dmitry Skiba <dskiba@chromium.org>
Reviewed-by: Alexei Svitkine <asvitkine@chromium.org>
Reviewed-by: Yaron Friedman <yfriedman@chromium.org>
Reviewed-by: Brian White <bcwhite@chromium.org>
Reviewed-by: Bo <boliu@chromium.org>
Reviewed-by: Richard Coles <torne@chromium.org>
Reviewed-by: Scott Violet <sky@chromium.org>
Cr-Commit-Position: refs/heads/master@{#549818}
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/android_webview/browser/aw_browser_main_parts.cc
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/android_webview/glue/java/src/com/android/webview/chromium/SharedStatics.java
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/android_webview/java/src/org/chromium/android_webview/AwBrowserContext.java
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/BUILD.gn
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/java/src/org/chromium/base/MemoryPressureListener.java
[add] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/java/src/org/chromium/base/memory/MemoryPressureCallback.java
[add] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/java/src/org/chromium/base/memory/MemoryPressureMonitor.java
[delete] https://crrev.com/ff4cbba00a2f6f40c62e3c64526e860d995cd2e1/base/android/javatests/src/org/chromium/base/MemoryPressureListenerTest.java
[add] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/junit/src/org/chromium/base/memory/MemoryPressureMonitorTest.java
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/memory_pressure_listener_android.cc
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/base/android/memory_pressure_listener_android.h
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/chrome/android/java/src/org/chromium/chrome/browser/ChromeApplication.java
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/chrome/browser/browser_process_platform_part_android.cc
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/content/app/android/content_child_process_service_delegate.cc
[modify] https://crrev.com/5baa1f510464d09419514f8e41b15482a46078db/tools/metrics/histograms/histograms.xml

Now that main CL has landed, let's discuss the second part: propagating pressure signals to renderers.

Right now services listen to ComponentCallbacks2 and don't get any additional signals from the browser. That was added in the context of  issue 251723  back in 2013.

Now the browser polls on CRITICAL pressure, and I want to propagate these signals to renderers to reduce their memory usage. In some cases this constraints renderer memory growth and greatly reduces chances of an OOM-kill. For an example see attached screenshots of "Unknown" renderer memory (v8 + PartitionAlloc) when playing long Youtube video.

In addition to renderers it would also be nice to propagate pressure signal to the GPU process too.

Plus propagation needs to happen in Java to benefit from throttling.

Given all this I propose the following:

1. We propagate pressure signals to all services. This makes implementation simple, but also raises a question of a conflicting signals in case of background services. The idea is that Android might be more aggressive with background services, so it might send CRITICAL to a background service, but MODERATE to the browser process. Unfortunately there is no UMA on memory pressure sent to services (browser is covered by MemoryAndroid.Notification{Foreground,Background}).

2. To mitigate conflicting signals we'll only take propagated pressure into account when it's not better than the last one that was reported to a service. I.e. browser can drive pressure higher, but pressure will only go down when Android sends a signal.

Additionally I propose we add UMAs to understand distribution of propagated signals vs reported by Android. Maybe we don't need to listen for Android signals at all, and can rely solely on signals from the browser.

Deleted: ChromeBeta.png 48.0 KB

	ChromeBeta.png 48.0 KB View Download

Deleted: ToTWithPropagation.png 68.8 KB

	ToTWithPropagation.png 68.8 KB View Download

> Now the browser polls on CRITICAL pressure, and I want to propagate these signals to renderers to reduce their memory usage. In some cases this constraints renderer memory growth and greatly reduces chances of an OOM-kill.

How does that work when polling can only send the critical -> non-critical transition?

> 2. To mitigate conflicting signals we'll only take propagated pressure into account when it's not better than the last one that was reported to a service. I.e. browser can drive pressure higher, but pressure will only go down when Android sends a signal.

what's the combining function here exactly? effective pressure = max(pressure from browser, pressure from android)?

Here's my understanding of when signals are sent:

App in Foreground:
* onTrimMemory(moderate) == fewer than 13 cch processes
* onTrimMemory(low) == fewer than 6 cch processes
* onTrimMemory(critical) == fewer than 4 cch processes
* onLowMemory() == 0 cch processes

App in Background:
* onTrimMemory(backgrounded) == First 33% of LRU
* onTrimMemory(moderate) == Next 33% of LRU
* onTrimMemory(complete) == Bottom 33% of LRU

These are sent only when:
* foreground app is under memory pressure (any level) *and* an app was recently killed.
  * More simply: Only when < 13 background apps.


For non-foreground renderers, I think they should behave as if memory pressure is critical all the time (or if they don't, we should aim for that to be true - maybe have them just fake a critical memory signal?)

For foreground renderers - I believe the OS sends them all the same signal. Given what you've implemented, I think it would be least complex to have the services listen only for the events propagated from the browser, and have them ignore any direct memory pressure events.

boliu@:

>what's the combining function here exactly? effective pressure = max(pressure from browser, pressure from android)?

Yes, essentially. But we ignore browser's pressure if it's less than the last reported one in the renderer, so the code is more like:

if pressure_from_browser >= pressure_monitor.last_reported_pressure:
  pressure_monitor.report(pressure_from_browser)


agrieve@:

Those rules look pretty simple, thanks! Did you extract them from ActivityManagerService.updateOomAdjLocked()?

> For non-foreground renderers, I think they should behave as if memory pressure is critical all the time (or if they don't, we should aim for that to be true - maybe have them just fake a critical memory signal?)

There are two issues:

1. V8 memory pressure listener is stateful, so we can't just send CRITICAL to free memory - we also need to send NONE to switch V8 back to non-CRITICAL mode. But NONE is not supposed to be sent, see issue 828057.

2. How easy it's to hook "renderer background / foreground" events? And is there access to ChildProcessConnection at that point? Treating all services equally really simplifies the implementation (see below).


CL: https://chromium-review.googlesource.com/c/chromium/src/+/992865

It was over a year ago when I looked into how the signals worked. I don't remember exactly where I gleaned it, but it was from reading source, and also from creating a demo app to validate the model. At the time N was the latest Android, so it may actually have changed for O...

Sending the same signals to all processes sgtm. Presumably background renderers should be using some other "isBackgrounded" logic to tighten their memory usage. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923

commit 9bd1a50ca7a132d6eb582f0fa2674b98b57f9923
Author: Dmitry Skiba <dskiba@chromium.org>
Date: Sat Apr 14 01:30:33 2018

Propagate memory pressure to child processes.

MemoryPressureMonitor (added in https://crrev.com/c/953166) polls
memory pressure when it gets CRITICAL to (1) lower memory usage by
repeatedly invoking pressure listeners and (2) sense (and notify
listeners) when pressure subsides.

However the polling only happens in the browser / WebView process,
because (1) ActivityManager.getMyMemoryState() can't be called from
isolated services, and (2) we want to poll only when Chrome is in
the foreground / there are WebView instances around.

This CL propagates pressure signals from the polling process to all
its services. That way in addition to getting pressure signals from
Android via ComponentCallbacks2, services also get signals from the
their main process.

Bug: 813909
Change-Id: Icef3b31106dcf432e6cdbdb0e1cdd84539dd690b
Reviewed-on: https://chromium-review.googlesource.com/992865
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Bo <boliu@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: Dmitry Skiba <dskiba@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550850}
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/MemoryPressureListener.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/memory/MemoryPressureMonitor.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/ChildProcessConnection.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/ChildProcessService.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/IChildProcessService.aidl

Requesting merge of commit 9bd1a50ca7a132d6eb582f0fa2674b98b57f9923 from #50, which narrowly missed M67.

Your change meets the bar and is auto-approved for M67. Please go ahead and merge the CL to branch 3396 manually. Please contact milestone owner if you have questions.
Owners: cmasso@(Android), cmasso@(iOS), kbleicher@(ChromeOS), govind@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923

commit 9bd1a50ca7a132d6eb582f0fa2674b98b57f9923
Author: Dmitry Skiba <dskiba@chromium.org>
Date: Sat Apr 14 01:30:33 2018

Propagate memory pressure to child processes.

MemoryPressureMonitor (added in https://crrev.com/c/953166) polls
memory pressure when it gets CRITICAL to (1) lower memory usage by
repeatedly invoking pressure listeners and (2) sense (and notify
listeners) when pressure subsides.

However the polling only happens in the browser / WebView process,
because (1) ActivityManager.getMyMemoryState() can't be called from
isolated services, and (2) we want to poll only when Chrome is in
the foreground / there are WebView instances around.

This CL propagates pressure signals from the polling process to all
its services. That way in addition to getting pressure signals from
Android via ComponentCallbacks2, services also get signals from the
their main process.

Bug: 813909
Change-Id: Icef3b31106dcf432e6cdbdb0e1cdd84539dd690b
Reviewed-on: https://chromium-review.googlesource.com/992865
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Bo <boliu@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: Dmitry Skiba <dskiba@chromium.org>
Cr-Commit-Position: refs/heads/master@{#550850}
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/MemoryPressureListener.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/memory/MemoryPressureMonitor.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/ChildProcessConnection.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/ChildProcessService.java
[modify] https://crrev.com/9bd1a50ca7a132d6eb582f0fa2674b98b57f9923/base/android/java/src/org/chromium/base/process_launcher/IChildProcessService.aidl

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e148251412671288598a2299ff2523039bab7b0b

commit e148251412671288598a2299ff2523039bab7b0b
Author: Dmitry Skiba <dskiba@chromium.org>
Date: Wed Apr 18 04:48:45 2018

[Merge to M67] Propagate memory pressure to child processes.

MemoryPressureMonitor (added in https://crrev.com/c/953166) polls
memory pressure when it gets CRITICAL to (1) lower memory usage by
repeatedly invoking pressure listeners and (2) sense (and notify
listeners) when pressure subsides.

However the polling only happens in the browser / WebView process,
because (1) ActivityManager.getMyMemoryState() can't be called from
isolated services, and (2) we want to poll only when Chrome is in
the foreground / there are WebView instances around.

This CL propagates pressure signals from the polling process to all
its services. That way in addition to getting pressure signals from
Android via ComponentCallbacks2, services also get signals from the
their main process.

(cherry picked from commit 9bd1a50ca7a132d6eb582f0fa2674b98b57f9923)

Bug: 813909
Change-Id: Icef3b31106dcf432e6cdbdb0e1cdd84539dd690b
Reviewed-on: https://chromium-review.googlesource.com/992865
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Bo <boliu@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: Dmitry Skiba <dskiba@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#550850}
Reviewed-on: https://chromium-review.googlesource.com/1016065
Reviewed-by: Dmitry Skiba <dskiba@chromium.org>
Cr-Commit-Position: refs/branch-heads/3396@{#74}
Cr-Branched-From: 9ef2aa869bc7bc0c089e255d698cca6e47d6b038-refs/heads/master@{#550428}
[modify] https://crrev.com/e148251412671288598a2299ff2523039bab7b0b/base/android/java/src/org/chromium/base/MemoryPressureListener.java
[modify] https://crrev.com/e148251412671288598a2299ff2523039bab7b0b/base/android/java/src/org/chromium/base/memory/MemoryPressureMonitor.java
[modify] https://crrev.com/e148251412671288598a2299ff2523039bab7b0b/base/android/java/src/org/chromium/base/process_launcher/ChildProcessConnection.java
[modify] https://crrev.com/e148251412671288598a2299ff2523039bab7b0b/base/android/java/src/org/chromium/base/process_launcher/ChildProcessService.java
[modify] https://crrev.com/e148251412671288598a2299ff2523039bab7b0b/base/android/java/src/org/chromium/base/process_launcher/IChildProcessService.aidl

Things to do here:

1. Always report throttled pressure at the end of the interval. Right now we're reporting only if there was a change. I.e. if we receive CRITICAL and then MODERATE, we report CRITICAL, start throttling interval, and report MODERATE once throttling interval ends. But if we receive MODERATE and then MODERATE again, the second MODERATE is ignored. For the renderer this means that even though browser polls on CRITICAL pressure, renderer might ignore polled pressure from the browser if it receives CRITICAL through ComponentCallbacks2 before. This can increase intervals between pressure levels seen by the native side, and might cause spikes in memory usage as seen on ToTWithPropagation.png from #45.

2. Actually we might just stop listening to ComponentCallbacks2 in services and always rely on the signals from the browser. For that one needs to analyze Android.MemoryPressureNotification.* histograms.

One other (potential) thing to do: map TRIM_MEMORY_RUNNING_MODERATE to MODERATE memory pressure. Here is a CL that does that: https://chromium-review.googlesource.com/c/chromium/src/+/1014630 (for the actual change change Patchset2-Patchset1 difference).

Available, but no owner or component? Please find a component, as no one will ever find this without one.