New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 850545 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 4
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Tab discarder is kicked in long after a low memory notification is fired

Project Member Reported by cylee@chromium.org, Jun 7 2018

Issue description

When a low memory notification comes in, tab discarder should start discarding tab promptly to ease memory pressure. However we observed that sometimes on ChromeOS the gap is significant. It could be several seconds or even tens of seconds before tab discarder start working. The problem is that before the real tab discarder logic is run, a log line which dumps tab details and their memory usages could took lots of time:

memory::OomMemoryDetails::Log(title, callback); 
|callback| is PurgeMemoryAndDiscardTab() where the real discarding logic happens.
https://cs.chromium.org/chromium/src/chrome/browser/resource_coordinator/tab_manager.cc?rcl=d8cd8cb89ff6e2ca428a54a697166237b2f50d5d&l=312

OomMemoryDetails::Log() collects tab statistics asynchronously and calls the callback when all needed information is collected and printed. In certain situations, it could take a long time to collect all needed data.

In system_logs, lines like this report the time taken (in the parenthesis) to collect tabs information:
[1110:1110:0502/214952.638171:WARNING:oom_memory_details.cc(46)] Tab Discards Memory details (769 ms):

I collected 173 feedback report which contains OOM kill and here's the distribution of time taken:

time(seconds), occurrence, cumulative percentage
0 11278 62.47%
1 1824 72.58%
2 1199 79.22%
3 852 83.94%
4 686 87.74%
5 494 90.47%
6 490 93.19%
7 377 95.28%
8 190 96.33%
9 140 97.10%
10 70 97.49%
11 67 97.86%
12 59 98.19%
13 31 98.36%
14 21 98.48%
15 23 98.60%
16 23 98.73%
17 12 98.80%
18 23 98.93%
19 19 99.03%
20 11 99.09%
21 15 99.17%
22 13 99.25%
23 8 99.29%
24 6 99.32%
25 3 99.34%
26 9 99.39%
27 2 99.40%
28 4 99.42%
29 4 99.45%
30 7 99.48%
31 3 99.50%
32 7 99.54%
33 5 99.57%
34 5 99.60%
35 3 99.61%
36 7 99.65%
37 9 99.70%
38 2 99.71%
39 3 99.73%
40 2 99.74%
41 4 99.76%
42 4 99.78%
43 3 99.80%
44 1 99.81%
45 1 99.81%
46 2 99.82%
47 1 99.83%
48 2 99.84%
49 1 99.84%
50 1 99.85%
51 1 99.86%
52 1 99.86%
53 1 99.87%
54 1 99.87%
55 1 99.88%
56 2 99.89%
57 2 99.90%
58 2 99.91%
59 1 99.92%
60 2 99.93%
62 2 99.94%
63 2 99.95%
65 2 99.96%
66 1 99.97%
67 1 99.97%
68 1 99.98%
69 1 99.98%
70 2 99.99%
73 1 100.00%
Total 18053

That is, more than 37.5% of the tab info dump takes > 1s. 10% even takes > 5s. It's not acceptable.

The logging line should not block tab discarding from happening since we need immediate action when low memory signal comes in. I'm working on a CL to make the log line non-blocking. It should not be a ChromeOS only problem but apply to all platforms.

chrisha, fdoray
 Any comments?

 
Wow that is really slow, good find.
This seems consistent with some of the graphs generated by memd under heavy pressure.

Comment 3 by vovoy@chromium.org, Jun 11 2018

Cc: vovoy@chromium.org
Project Member

Comment 5 by bugdroid1@chromium.org, Jun 16 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9f5206473f137a0ea67c645b5d9798059393c08f

commit 9f5206473f137a0ea67c645b5d9798059393c08f
Author: Cheng-Yu Lee <cylee@chromium.org>
Date: Sat Jun 16 15:50:08 2018

Discard tab immediately without waiting for LogMemory().

LogMemory() calls memory::OomMemoryDetails::Log(), which is an async
call and may take a long time (only 60% returns < 1s, 10% returns > 5s).
When we're running out of memory, we need to discard tab faster.

BUG= 850545 

Change-Id: Ia652b2a80f968b0e437904bb6bc2c7c131c1958d
Reviewed-on: https://chromium-review.googlesource.com/1097038
Reviewed-by: Cheng-Yu Lee <cylee@chromium.org>
Reviewed-by: Trent Apted <tapted@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Reviewed-by: François Doray <fdoray@chromium.org>
Commit-Queue: Cheng-Yu Lee <cylee@chromium.org>
Cr-Commit-Position: refs/heads/master@{#567891}
[modify] https://crrev.com/9f5206473f137a0ea67c645b5d9798059393c08f/chrome/browser/memory/oom_memory_details.cc
[modify] https://crrev.com/9f5206473f137a0ea67c645b5d9798059393c08f/chrome/browser/memory/oom_memory_details.h
[modify] https://crrev.com/9f5206473f137a0ea67c645b5d9798059393c08f/chrome/browser/resource_coordinator/tab_manager.cc
[modify] https://crrev.com/9f5206473f137a0ea67c645b5d9798059393c08f/chrome/browser/resource_coordinator/tab_manager.h
[modify] https://crrev.com/9f5206473f137a0ea67c645b5d9798059393c08f/chrome/browser/ui/sad_tab.cc

Cc: conradlo@chromium.org

Comment 7 by cylee@chromium.org, Jun 28 2018

Labels: Merge-Request-68
Request mergeback to M68 since this is a significant bug.
The react time of tab discarder would be improved a lot.
Project Member

Comment 8 by sheriffbot@chromium.org, Jun 28 2018

Labels: -Merge-Request-68 Hotlist-Merge-Review Merge-Review-68
This bug requires manual review: M68 has already been promoted to the beta branch, so this requires manual review
Please contact the milestone owner if you have questions.
Owners: cmasso@(Android), kariahda@(iOS), bhthompson@(ChromeOS), abdulsyed@(Desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 9 by cylee@chromium.org, Jun 28 2018

Cc: bhthompson@chromium.org
+bhthompson@ for mergeback request
Labels: -Hotlist-Merge-Review -Merge-Review-68 Merge-Approved-68 OS-Chrome
Project Member

Comment 11 by bugdroid1@chromium.org, Jun 29 2018

Labels: -merge-approved-68 merge-merged-3440
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa89643c4b629a17bd568bfd3b8423a03a90c1c8

commit aa89643c4b629a17bd568bfd3b8423a03a90c1c8
Author: Cheng-Yu Lee <cylee@chromium.org>
Date: Fri Jun 29 21:20:27 2018

Discard tab immediately without waiting for LogMemory().

LogMemory() calls memory::OomMemoryDetails::Log(), which is an async
call and may take a long time (only 60% returns < 1s, 10% returns > 5s).
When we're running out of memory, we need to discard tab faster.

BUG= 850545 

Change-Id: Ia652b2a80f968b0e437904bb6bc2c7c131c1958d
Reviewed-on: https://chromium-review.googlesource.com/1097038
Reviewed-by: Cheng-Yu Lee <cylee@chromium.org>
Reviewed-by: Trent Apted <tapted@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Reviewed-by: François Doray <fdoray@chromium.org>
Commit-Queue: Cheng-Yu Lee <cylee@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#567891}(cherry picked from commit 9f5206473f137a0ea67c645b5d9798059393c08f)
Reviewed-on: https://chromium-review.googlesource.com/1121176
Cr-Commit-Position: refs/branch-heads/3440@{#573}
Cr-Branched-From: 010ddcfda246975d194964ccf20038ebbdec6084-refs/heads/master@{#561733}
[modify] https://crrev.com/aa89643c4b629a17bd568bfd3b8423a03a90c1c8/chrome/browser/memory/oom_memory_details.cc
[modify] https://crrev.com/aa89643c4b629a17bd568bfd3b8423a03a90c1c8/chrome/browser/memory/oom_memory_details.h
[modify] https://crrev.com/aa89643c4b629a17bd568bfd3b8423a03a90c1c8/chrome/browser/resource_coordinator/tab_manager.cc
[modify] https://crrev.com/aa89643c4b629a17bd568bfd3b8423a03a90c1c8/chrome/browser/resource_coordinator/tab_manager.h
[modify] https://crrev.com/aa89643c4b629a17bd568bfd3b8423a03a90c1c8/chrome/browser/ui/sad_tab.cc

Components: OS>Performance>Memory
Status: Assigned (was: Untriaged)
This bug has an owner, thus, it's been triaged. Changing status to "assigned".
Status: Fixed (was: Assigned)

Sign in to add a comment