Headless Chrome crashes with "Abnormal renderer termination" when loading page with large image on Debian 7
Reported by
tollm...@gmail.com,
May 14 2018
|
||||
Issue descriptionChrome Version : 66.0.3359.170 URLs (if applicable) : https://tollmanz.github.io/lighthouse-bug-repro/page/index.html Other browsers tested: Add OK or FAIL, along with the version, after other browsers where you have tested this issue: Safari: N/A Firefox: N/A Edge: N/A What steps will reproduce the problem? When loading a page with a large image using Headless Chrome on CentOS 7.4.x, Chrome crashes with an "Abnormal renderer termination" error. Similarly, when running Headless Chrome on OS X, the same page crashes Chrome only when remote debugging is enabled. I've created a very basic page that reduces the bug: * Basic HTML * No script * No CSS * 1, 4.35 MB JPG ``` <!doctype html> <html lang=""> <head> <meta charset="utf-8"> <meta http-equiv="x-ua-compatible" content="ie=edge"> <title>Test Page</title> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"> </head> <body> <p>A Test Page</p> <img src="panda-bear-sleeping-on-the-ground.jpg" /> </body> </html> ``` I have setup a repo showing the bug, which includes a Docker container for replicating the environment: https://github.com/tollmanz/lighthouse-bug-repro/tree/headless-chrome (1) Clone repo and checkout branch `git clone https://github.com/tollmanz/lighthouse-bug-repro.git --branch headless-chrome` (2) Install Docker (3) Build image ``` cd lighthouse-bug-repro/env && docker build -t lighthouse-bug . ``` (4) Attach to image ``` docker run -it lighthouse-bug /bin/bash ``` (5) Load test page with big image using Headless Chrome ``` google-chrome-stable --headless --enable-logging --v=99 --no-sandbox --disable-gpu https://tollmanz.github.io/lighthouse-bug-repro/page/index.html ``` Sample output: ``` [root@00335566dbd5]# google-chrome-stable --headless --no-sandbox --disable-gpu https://tollmanz.github.io/lighthouse-bug-repro/page/index.html [0514/020857.234627:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context. Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf [0514/020857.693097:ERROR:crash_handler_host_linux.cc(432)] Failed to write crash dump for pid 1534 Cannot upload crash dump: failed to open [0514/020857.744852:ERROR:headless_shell.cc(348)] Abnormal renderer termination. --2018-05-14 02:08:57-- https://clients2.google.com/cr/report Resolving clients2.google.com (clients2.google.com)... 172.217.9.142, 2607:f8b0:4000:813::200e Connecting to clients2.google.com (clients2.google.com)|172.217.9.142|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2018-05-14 02:08:57 ERROR 403: Forbidden. Unexpected crash report id length Failed to get crash dump id. Report Id: ``` What is the expected result? The page should load and the application should exit without an error. What happens instead? An error is produced: "[0514/020857.744852:ERROR:headless_shell.cc(348)] Abnormal renderer termination." Please provide any additional information below. Attach a screenshot if possible. Originally, I thought this bug was a Lighthouse issue. I reported the bug in that repo (https://github.com/GoogleChrome/lighthouse/issues/5044#issuecomment-385451888) and was directed here as it seems that the issue is with Chrome.
,
May 14 2018
,
May 15 2018
I'm encountering a very similar error in our CI environment. -bash-4.2$ google-chrome-stable --headless --no-sandbox --disable-gpu https://google.com [0515/221910.462903:ERROR:gpu_process_transport_factory.cc(1007)] Lost UI shared context. Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf -bash-4.2$ rpm -qi google-chrome-stable.x86_64 Name : google-chrome-stable Version : 66.0.3359.117 Release : 1 Architecture: x86_64 Install Date: Tue 24 Apr 2018 11:02:11 PM UTC Group : Applications/Internet Size : 188335720 License : Multiple, see https://chrome.google.com/ Signature : DSA/SHA1, Tue 17 Apr 2018 04:16:00 AM UTC, Key ID a040830f7fac5991 Source RPM : google-chrome-stable-66.0.3359.117-1.src.rpm Build Date : Tue 17 Apr 2018 03:50:32 AM UTC Build Host : lin64-4-m0.official.chromium.org Relocations : /opt Packager : Chrome Linux Team <chromium-dev@chromium.org> Vendor : Google Inc. URL : https://chrome.google.com/ Summary : Google Chrome Description : The web browser from Google Google Chrome is a browser that combines a minimal design with sophisticated technology to make the web faster, safer, and easier.
,
May 15 2018
Environment: -bash-4.2$ uname -a Linux worker-10-132-1-166.ec2 3.10.0-514.10.2.el7.x86_64 #1 SMP Fri Mar 3 00:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
,
May 16 2018
I suspect the container is running out of memory and crashing because of that. As soon as https://chromium-review.googlesource.com/c/chromium/src/+/1059621 lands we should have working crash reports again, which should let us find pinpoint the crashing function here.
,
May 16 2018
> I suspect the container is running out of memory and crashing because of that. FWIW, the containers I'm using in production as well as my local test containers do not appear to be hitting OOM issues. This was my first guess as well, which caused me to increase memory to no avail. Additionally, all of the metrics I have available to me suggest that the container has ample memory and never reached a point where it was exceeded.
,
Aug 29
The CL in comment 5 landed in Chrome 68.0.3433.0. tollmanz/jhk if you want to retry with a newer chrome, you should try with Chrome, not Chromium. and then open chrome://crashes/ afterward to get us a crash ID. we can investigate with that.
,
Sep 5
Hi Paul, I'm having trouble getting you a crash report ID. I'm running this inside a docker image instance and cannot access the UI. Is there a method I can use to gather the crash ID (and perhaps turn on crash reporting) via CLI? I did try running this with Chrome 69.0.3497.81 and got more robust output. You can view it here: https://gist.github.com/tollmanz/ad8c377c3c98588a42359091ff52c40a
,
Sep 5
,
Nov 15
Any updates on this ?
,
Dec 17
We're running into the same problem. We do get a Crash dump id, but it looks uploading the crash dump failed with "Cannot upload crash dump: failed to open" /usr/bin/google-chrome --version Google Chrome 71.0.3578.80 --- Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank" [1217/104614.792391:ERROR:gpu_process_transport_factory.cc(967)] Lost UI shared context. [1217/104728.708922:ERROR:crash_handler_host_linux.cc(443)] Failed to write crash dump for pid 1526 Cannot upload crash dump: failed to open --2018-12-17 10:47:28-- https://clients2.google.com/cr/report Resolving clients2.google.com (clients2.google.com)... 64.233.167.102, 64.233.167.101, 64.233.167.100, ... Connecting to clients2.google.com (clients2.google.com)|64.233.167.102|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘/dev/fd/4’ 0K Crash dump id: 5437c746e2e5fff4 6.24M=0s 2018-12-17 10:47:29 (6.24 MB/s) - ‘/dev/fd/4’ saved [16] [1217/104730.490302:ERROR:headless_shell.cc(315)] Abnormal renderer termination. ---
,
Dec 17
Trying Chrome beta (Google Chrome 72.0.3626.17 beta) I get exactly the same. I'm happy to provide crash dumps on a different channel if uploading them indeed fails. Please let me know.
For completeness, using minidump_stackwalk on one of the crash dumps shows this, (doesn't look to helpful since I couldn't figure out how to symbolicate it):
---
Operating system: Linux
0.0.0 Linux 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64
CPU: amd64
family 6 model 158 stepping 9
3 CPUs
GPU: UNKNOWN
Crash reason: SIGBUS
Crash address: 0x7f9bcf1fa000
Process uptime: not available
Thread 2 (crashed)
0 libc-2.24.so + 0x128e2f
rax = 0x00007f9bcf1f95d1 rdx = 0x0000000000001f80
rcx = 0x0000000000001551 rbx = 0x0000000000001f80
rsi = 0x00002c539ff5da2f rdi = 0x00007f9bcf1fa000
rbp = 0x00007f9bdb5ad730 rsp = 0x00007f9bdb5ad6c8
r8 = 0x00002c539ff00000 r9 = 0x00002c539ff5ef80
r10 = 0x00246f72cc000000 r11 = 0x0000000000000246
r12 = 0x0000000000000000 r13 = 0x00002c539fd53060
r14 = 0x00007f9bcf1f95d1 r15 = 0x00002c539fa3e0e8
rip = 0x00007f9be4c5fe2f
Found by: given as instruction pointer in context
1 chrome + 0x3c5ce81
rbp = 0x00007f9bdb5ad760 rsp = 0x00007f9bdb5ad740
rip = 0x000055797164be81
Found by: previous frame's frame pointer
2 chrome + 0x3891a8d
rbp = 0x00007f9bdb5ad7f0 rsp = 0x00007f9bdb5ad770
rip = 0x0000557971280a8d
Found by: previous frame's frame pointer
3 chrome + 0x388fe4c
rbp = 0x00007f9bdb5ad8c0 rsp = 0x00007f9bdb5ad800
rip = 0x000055797127ee4c
Found by: previous frame's frame pointer
4 chrome + 0x3890893
rbp = 0x00007f9bdb5ad950 rsp = 0x00007f9bdb5ad8d0
rip = 0x000055797127f893
Found by: previous frame's frame pointer
5 chrome + 0x3890d1c
rbp = 0x00007f9bdb5ada50 rsp = 0x00007f9bdb5ad960
rip = 0x000055797127fd1c
Found by: previous frame's frame pointer
6 chrome + 0x391fdd9
rbp = 0x00007f9bdb5adab0 rsp = 0x00007f9bdb5ada60
rip = 0x000055797130edd9
Found by: previous frame's frame pointer
7 chrome + 0x38b20f4
rbp = 0x00007f9bdb5adad0 rsp = 0x00007f9bdb5adac0
rip = 0x00005579712a10f4
Found by: previous frame's frame pointer
8 chrome + 0x23344d4
rbp = 0x00007f9bdb5adaf0 rsp = 0x00007f9bdb5adae0
rip = 0x000055796fd234d4
Found by: previous frame's frame pointer
9 chrome + 0x38e54d0
rbp = 0x00007f9bdb5adb60 rsp = 0x00007f9bdb5adb00
rip = 0x00005579712d44d0
Found by: previous frame's frame pointer
10 chrome + 0x391c053
rbp = 0x00007f9bdb5adb90 rsp = 0x00007f9bdb5adb70
rip = 0x000055797130b053
Found by: previous frame's frame pointer
11 libpthread-2.24.so + 0x7494
rsp = 0x00007f9bdb5adba0 rip = 0x00007f9beaaab494
Found by: stack scanning
12 libc-2.24.so + 0xe8acf
rsp = 0x00007f9bdb5adc40 rip = 0x00007f9be4c1facf
Found by: stack scanning
---
,
Dec 17
We're able to work around the issue by allowing more shared memory. The issue reliably reproduced with 64MB shared memory: # df -h Filesystem Size Used Avail Use% Mounted on shm 64M 0 64M 0% /dev/shm Upping that to 1 GB makes the crash go way an the screenshot rendering works: # df -h | grep shm shm 1.0G 0 1.0G 0% /dev/shm |
||||
►
Sign in to add a comment |
||||
Comment 1 by skyos...@chromium.org
, May 14 2018