New issue
Advanced search Search tips

Issue 695212 link

Starred by 11 users

Issue metadata

Status: Unconfirmed
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment

Running headless chrome on CentOS 7 crashes with "Illegal instruction"

Reported by l...@getadmiral.com, Feb 22 2017

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36

Steps to reproduce the problem:
1. Install google-chrome-unstable on CentOS 7
2. run `google-chrome-unstable --headless`

What is the expected behavior?

What went wrong?
When I try to run in headless mode, it crashes.  If I strace it, I get this information:

--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x7f0f6845930b} ---
+++ killed by SIGILL +++
Illegal instruction

Kernel:
Linux luke.11-e.ninja 3.10.0-514.2.2.el7.x86_64

CPU info:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
stepping        : 4
microcode       : 0x1
cpu MHz         : 2399.998
cache size      : 15360 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm arat vnmi ept fsgsbase tsc_adjust smep erms xsaveopt
bogomips        : 4799.99
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Crashed report ID: 

How much crashed? Whole browser

Is it a problem with a plugin? No 

Did this work before? N/A 

Chrome version: 58.0.3018.3  Channel: dev
OS Version: CentOS 7
Flash Version:

 

Comment 1 by ajha@chromium.org, Feb 23 2017

Labels: Needs-Triage-M58 Proj-Headless
Labels: TE-NeedsTriageHelp
Cc: thomasanderson@chromium.org
are there any crash IDs in chrome:crashes?
Labels: Needs-Feedback
This is on physical hardware and not in a VM, right?

FWIW, 58.0.3018.3 runs fine here on a Core 2 Duo in normal mode. A newer Sandy Bridge Xeon shouldn't be missing any instructions.

Comment 5 by l...@getadmiral.com, Feb 23 2017

In dmesg I get this:
[74261.254048] traps: chrome[20497] trap invalid opcode ip:7f8675dcf30b sp:7ffe3115bba0 error:0 in chrome[7f86736a4000+6a16000]

Installing a fonts package such as gnu-free-sans-fonts seems to fix the illegal instruction. Chrome should handle this gracefully and print out some error. I just happen to get lucky and try installing a fonts package.

Comment 6 Deleted

Comment 7 by l...@getadmiral.com, Feb 23 2017

So I guess the updated steps to reproduce are:

1. Install google-chrome-unstable on CentOS 7
2. Have no fonts package installed
3. run `google-chrome-unstable --headless`

Comment 8 by l...@getadmiral.com, Feb 23 2017

I have verified that this happens on both VMs (Google Compute Engine and DigitalOcean) and hardware
I wonder if this is just a failing CHECK(). Could you try again with --enable-logging --v=2?
[root@sasha ~]# google-chrome-unstable --headless --enable-logging --v=2
[0227/174500.742581:VERBOSE1:zygote_main_linux.cc(626)] ZygoteMain: initializing 0 fork delegates
[0227/174500.743406:INFO:cpu_info.cc(50)] Available number of cores: 8
[0227/174500.746753:VERBOSE1:pulse_stubs.cc(683)] dlopen(libpulse.so.0) failed, dlerror() says:
libpulse.so.0: cannot open shared object file: No such file or directory
[0227/174500.746918:VERBOSE1:audio_manager_pulse.cc(245)] Failed on loading the Pulse library and symbols
[0227/174500.746943:WARNING:audio_manager.cc(322)] Multiple instances of AudioManager detected
[0227/174500.746956:WARNING:audio_manager.cc(281)] Multiple instances of AudioManager detected
[0227/174500.749014:VERBOSE1:webrtc_internals.cc(99)] Could not get the download directory.
[0227/174500.749413:VERBOSE1:proxy_service.cc(958)] PAC support disabled because there is no system implementation
[0227/174500.753706:VERBOSE1:sandbox_linux.cc(70)] Activated seccomp-bpf sandbox for process type: renderer.
Illegal instruction
[0100/000000.758489:ERROR:broker_posix.cc(41)] Invalid node channel message
[0227/174500.767850:ERROR:gl_implementation.cc(245)] Failed to load /opt/google/chrome-unstable/libosmesa.so: /opt/google/chrome-unstable/libosmesa.so: cannot open shared object file: No such file or directory
[0227/174500.767921:VERBOSE1:gpu_init.cc(186)] gl::init::InitializeGLOneOff failed
[0227/174500.768250:ERROR:broker_posix.cc(41)] Invalid node channel message
Have the same issue with Chrome 56 in CentOS7 VM.

Kernel:
3.10.0-327.10.1.el7.x86_64

CPU:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
stepping	: 4
microcode	: 0x428
cpu MHz		: 2793.370
cache size	: 25600 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips	: 5586.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
stepping	: 4
microcode	: 0x428
cpu MHz		: 2793.370
cache size	: 25600 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips	: 5586.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

Chrome Version:
Google Chrome 56.0.2924.87

I tried running Chrome 56 with Xvfb and saw the same "Illegal instruction" in log, where Chrome 52 works fine for me.
Project Member

Comment 12 by sheriffbot@chromium.org, Mar 1 2017

Cc: thestig@chromium.org
Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "thestig@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Trying 58.0.3029.14, I'm seeing it exit and complaining about /opt/google/chrome-unsatble/libosmesa.so being missing. However, the exit code is 0 and there's no crash.
Installing libOSMesa and symlinking /opt/google/chrome-unstable/libosmesa.so to pint to libOSMesa.so.8 gets rid of those messages, but there's still no crash here.
I also encountered this issue on a jenkins build slave running centos 7 based on a minimal install.  The workaround of installing a font package worked for me and adding the mesa package/symlink suppressed the missing mesa library error

yum install mesa-libOSMesa-devel gnu-free-sans-fonts
ln -s /usr/lib64/libOSMesa.so.8 /opt/google/chrome-unstable/libosmesa.so
I was facing this issue when trying to run Chrome headless in Jenkins Centos 7 and installing the fonts fixed it for me.

This was a step to enable Google/Lighthouse to run in jenkins, the only extra chrome param needed was --no-sandbox.

Out of interest Steve, how did you find the font fix?
Components: Internals>Headless
I faced the exact same issue with Google Chrome 59.0.3071.115 on CentOS 7 when I attempted to run the command "google-chrome --headless", and the fix suggested by Steve fixed it.

The error message I got was:

$ google-chrome --headless
[0717/061050.051485:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected
[0717/061050.051526:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected
Trace/breakpoint trap
[0100/000000.061179:FATAL:zygote_linux.cc(476)] Failed to synchronise with parent zygote process
[0717/061050.072793:ERROR:gl_implementation.cc(246)] Failed to load /opt/google/chrome/libosmesa.so: /opt/google/chrome/libosmesa.so: cannot open shared object file: No such file or directory

After installing mesa-libOSMesa-devel, I got this error:

$ google-chrome --headless
[0717/062429.641863:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected
[0717/062429.641900:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected
Trace/breakpoint trap

and after installing gnu-free-sans-font, google-chrome runs successfully.
Experienced the same problem on Jenkins under CentOS 7 and install gnu-free-sans-fonts is all it took to get things working. Thanks for the solution Steve!
libosmesa shouldn't be required anymore in the latest release now that we're using SwiftShader for rendering. Could you please re-test this on CentOS?
Labels: Needs-Feedback
Project Member

Comment 22 by sheriffbot@chromium.org, Oct 18

Status: Archived (was: Unconfirmed)
Issue has not been modified or commented on in the last 365 days, please re-open or file a new bug if this is still an issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
I've run into the same problem with a ubuntu 16.04-based docker on a centos 7 host. 

two identical docker images (though different data in them when running), both with 69.0.3497.100-1. one always fails, the other never, even though both dockers are running on the same VM (so no CPU difference). 

installing fonts-open-sans seems to fix it.
Status: Unconfirmed (was: Archived)
Reopening since this still seems to be happening.

Would someone be willing to temporarily grant access to an affected VM? I can try logging in via ssh and see what's going wrong.

Sign in to add a comment