New issue
Advanced search Search tips

Issue 704099 link

Starred by 3 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

A tab can consume up to 2GB of memory from JavaScript on 64bit Linux

Reported by jf.pamb...@gmail.com, Mar 22 2017

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36

Steps to reproduce the problem:
1. Open multiple memory intensive tabs from the same site-instance
2. Witness 'Aw snap' crash in all tabs.

What is the expected behavior?
No crash, especially because of other neighbors I don't have control over.

What went wrong?
This is a multifaceted problem.

1) The memory limitation per process is low. Using [0] we can observe that the memory limit per process is:
  * 1.8 GB on Linux
  * 4 GB on Windows
  * > 8 GB (?) on MacOS

2) The site-instance process makes it impossible to know with how many tabs this limited resource is shared. For instance, all links opened from Gmail share the same process and thus memory pool[1] even though they are unrelated. If the pool gets close to 1.8 GB in Linux ALL tabs in the shared process will be terminated as shown in [2] when using [0] opened from an email.

We are building a memory intensive application and this behavior makes Chrome almost impossible to support, especially on Linux.

What makes this worst it that a) we can't know the limitation because of the share nature of the memory; b) when don't get allocation errors is JS, but complete 'Aw Snap' crashes. 

Also, with the devtools opened in any of the tabs sharing a common process, you have barely and headroom left.

Finally, Firefox will happily let me deplete all available RAM which is the expected behavior. 

[0] https://jsfiddle.net/jasonklotzer/kvnouyz7/
[1] http://dl.dropbox.com/u/168338/screenshots/20170322085607.png
[2] https://dl.dropboxusercontent.com/u/168338/screenshots/20170322085852.png

Did this work before? N/A 

Chrome version: 57.0.2987.110  Channel: stable
OS Version: 
Flash Version:
 

Comment 1 by kochi@chromium.org, Mar 23 2017

Cc: kochi@chromium.org
Components: -Blink Blink>MemoryAllocator
Summary: A tab can consume up to 2GB of memory from JavaScript on 64bit Linux (was: Unpredictable out-of-memory issues due to Chrome shared process model)
jf.pamb...@ thanks for your report.  Indeed this is a multifaceted problem.

For 1), other than physical available memory, virtual memory system on
each platform, memory layout for each process, and per-process limits
vary wildly to guarantee that "at least this amount of memory is available
to your web app" in general.

I have no idea off-hand why 64bit linux process has 1.8GB limitation,
so one possible thing Chrome might fix could be to raise the memory limit
for each tab on Linux. We only distribute 64bit builds of Chrome, so each
process should have much wider address space than 4GB, although the usable
JS heap memory would be restricted by various factors.

(btw, I ran your jsfiddle[0] on my workstation which has plenty of RAM
and confirmed the tab crashes around 2GB)

For 2), I thought nowadays tabs opened from Gmail doesn't share the renderer
process with Gmail if the domain is different. So if you see tabs opened from
Gmail share the same process with Gmail, it smells a bug to me.
Could you file a separate bug for this?


If you would not mind, we'd like to reuse this issue to track the
memory limit of Linux tab exclusively, and for any other actionable
issues, could you file a separate bug?


FYI, Google has some advice on tackling on memory issues on Chrome:
https://developers.google.com/web/tools/chrome-devtools/memory-problems/?hl=en

Comment 2 by kochi@chromium.org, Mar 23 2017

Status: Untriaged (was: Unconfirmed)
Thanks for the quick and helpful response.

I will open a new ticket to track the site-instance process issue.

As for memory consumption tips, our product deals with visualization of large data sets that can exceed one gigabyte. Once settled, after loading the data, the overhead of the application is about 100 MB (i.e. total mem usage = the large set + ~100 MB). 

With devtools and the site-instance model, I see 'Aw snap' between 20 to 30 times a day.. so, if there is anything I can do to help..
I opened  Issue 704521 .

Also, I am attaching the screenshots from the original report for posterity.
20170322085607.png
29.0 KB View Download
20170322085852.png
11.1 KB View Download

Comment 5 by kochi@chromium.org, Mar 23 2017

I learned today that the memory allocator used in Blink (PartitionAlloc)
uses a dedicated memory pool (a partition) for array buffer, and each
partition cannot exceed 4GB as 32bit pointer is used internally.
(that contradicts that on Mac 8+GB? could be allocated, though.)

Still, the reason why the memory is limited to 2GB on Linux is unclear.


Comment 6 by kochi@chromium.org, Apr 5 2017

Someone responsible for MemoryAllocator, can you explain why the data usage
is capped around ~2GB where more plenty of memory is available?

For the original question, jf.pamb...@, how much memory do you need for
your application to run on Chrome Linux?
To allow much more memory (>4G), IIUC Blink needs considerable change
that cannot happen in short term and in that case general advice would be
give up using Chrome for your application, unfortunately.

Thanks for your response. To give some context, we are building a medical application geared toward radiologists. 

The memory requirements depend on the dataset that is being visualize. I don't see any current use cases that would require >4 GB per tab, but ~2 GB is possible. However, these datasets will become larger as technology continues to evolve and 4 GB might not be enough in just a few years. Furthermore, I see OOM issues with much smaller memory usage with the devtools opened.

In addition, opening multiple datasets in multiple tabs is a common workflow. The site-instance process sharing model (where all grouped tabs share the same 2 GB memory pool) exacerbate this issue considerably. We are going to try the workaround proposed in #704521, but I would be eternally grateful it the memory limit on Linux 64 bit could be raised to be on par with other platforms.

Comment 8 by kochi@chromium.org, Apr 7 2017

Thanks for the feedback!

Please note that with developer tools open, you may see more memory leaking,
as garbage collecter cannot collect unused memory because the tool is also
grabbing some reference. And also devtools uses memory on its own, of course.

Comment 9 by kochi@chromium.org, Apr 7 2017

Cc: hajimehoshi@chromium.org
Cc: keishi@chromium.org haraken@chromium.org
I have been using Electron lately, and to my surprise, it is not plagued with that issue.

On chrome 61 (which was released today), I still get Aw snaps extremely easily. I just need to open 2-3 medical exam tabs from our application and they will all crash simultaneously.

I can also get this issue consistently by opening gotomeeting links from Office 365 as they each take >700 MB of memory and are pooled together in the same process.

However, I notice that Electron does not have this issue. I can easily allocate 20+ GB of typed array (see attachement, I did 'electron http://example.com', open the console, spam new Uint8Array(1*1024*1024*1024)).

I can only allocate one such array before I get into issues (either Array buffer allocation failed, or uncatchable 'Aw snaps' in real applications) in Chrome 61.

How can this be? Obviously, it can be fixed..
20170906082137_3246bc7fd2fca520.png
282 KB View Download
Cc: -kochi@chromium.org
Looks like this is mostly fixed. I noticed on Version 62.0.3202.94.
I can now allocated up to 8 GB per process group. Once I reach the limit, I now get a catchable buffer allocation error instead of 'aw snaps'.

Keep up the good work!
Status: Available (was: Untriaged)
Is this still an issue?
The situation seems to have improved since I can now allocate up to 8gb in the console with new Uint8Array(1*1024*1024*1024) and I will get 'Array buffer allocation failed' instead of 'Aw snap' when that limit is reached.

However, our application can open multiple tabs that will end up in the same process due to Chrome's process sharing model. Each tab consume about 1 GB. Opening two concurrent tabs works flawlessly, but adding a third tab will result in 'Aw snaps' across all tabs with 100% reproducibility.

This was observed on Linux with Chromium Version 66.0.3359.139 (Official Build).

Sign in to add a comment