New issue
Advanced search Search tips
Starred by 11 users

Issue metadata

Status: Unconfirmed
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug



Sign in to add a comment
link

Issue 695212: Running headless chrome on CentOS 7 crashes with "Illegal instruction"

Reported by l...@getadmiral.com, Feb 22 2017

Issue description

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36

Steps to reproduce the problem:
1. Install google-chrome-unstable on CentOS 7
2. run `google-chrome-unstable --headless`

What is the expected behavior?

What went wrong?
When I try to run in headless mode, it crashes.  If I strace it, I get this information:

--- SIGILL {si_signo=SIGILL, si_code=ILL_ILLOPN, si_addr=0x7f0f6845930b} ---
+++ killed by SIGILL +++
Illegal instruction

Kernel:
Linux luke.11-e.ninja 3.10.0-514.2.2.el7.x86_64

CPU info:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
stepping        : 4
microcode       : 0x1
cpu MHz         : 2399.998
cache size      : 15360 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm arat vnmi ept fsgsbase tsc_adjust smep erms xsaveopt
bogomips        : 4799.99
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Crashed report ID: 

How much crashed? Whole browser

Is it a problem with a plugin? No 

Did this work before? N/A 

Chrome version: 58.0.3018.3  Channel: dev
OS Version: CentOS 7
Flash Version:
 

Comment 1 by ajha@chromium.org, Feb 23 2017

Labels: Needs-Triage-M58 Proj-Headless

Comment 2 by msrchandra@chromium.org, Feb 23 2017

Labels: TE-NeedsTriageHelp

Comment 3 by thomasanderson@chromium.org, Feb 23 2017

Cc: thomasanderson@chromium.org
are there any crash IDs in chrome:crashes?

Comment 4 by thestig@chromium.org, Feb 23 2017

Labels: Needs-Feedback
This is on physical hardware and not in a VM, right?

FWIW, 58.0.3018.3 runs fine here on a Core 2 Duo in normal mode. A newer Sandy Bridge Xeon shouldn't be missing any instructions.

Comment 5 by l...@getadmiral.com, Feb 23 2017

In dmesg I get this:
[74261.254048] traps: chrome[20497] trap invalid opcode ip:7f8675dcf30b sp:7ffe3115bba0 error:0 in chrome[7f86736a4000+6a16000]

Installing a fonts package such as gnu-free-sans-fonts seems to fix the illegal instruction. Chrome should handle this gracefully and print out some error. I just happen to get lucky and try installing a fonts package.

Comment 6 Deleted

Comment 7 by l...@getadmiral.com, Feb 23 2017

So I guess the updated steps to reproduce are:

1. Install google-chrome-unstable on CentOS 7
2. Have no fonts package installed
3. run `google-chrome-unstable --headless`

Comment 8 by l...@getadmiral.com, Feb 23 2017

I have verified that this happens on both VMs (Google Compute Engine and DigitalOcean) and hardware

Comment 9 by skyos...@chromium.org, Feb 27 2017

I wonder if this is just a failing CHECK(). Could you try again with --enable-logging --v=2?

Comment 10 by l...@getadmiral.com, Feb 27 2017

[root@sasha ~]# google-chrome-unstable --headless --enable-logging --v=2
[0227/174500.742581:VERBOSE1:zygote_main_linux.cc(626)] ZygoteMain: initializing 0 fork delegates
[0227/174500.743406:INFO:cpu_info.cc(50)] Available number of cores: 8
[0227/174500.746753:VERBOSE1:pulse_stubs.cc(683)] dlopen(libpulse.so.0) failed, dlerror() says:
libpulse.so.0: cannot open shared object file: No such file or directory
[0227/174500.746918:VERBOSE1:audio_manager_pulse.cc(245)] Failed on loading the Pulse library and symbols
[0227/174500.746943:WARNING:audio_manager.cc(322)] Multiple instances of AudioManager detected
[0227/174500.746956:WARNING:audio_manager.cc(281)] Multiple instances of AudioManager detected
[0227/174500.749014:VERBOSE1:webrtc_internals.cc(99)] Could not get the download directory.
[0227/174500.749413:VERBOSE1:proxy_service.cc(958)] PAC support disabled because there is no system implementation
[0227/174500.753706:VERBOSE1:sandbox_linux.cc(70)] Activated seccomp-bpf sandbox for process type: renderer.
Illegal instruction
[0100/000000.758489:ERROR:broker_posix.cc(41)] Invalid node channel message
[0227/174500.767850:ERROR:gl_implementation.cc(245)] Failed to load /opt/google/chrome-unstable/libosmesa.so: /opt/google/chrome-unstable/libosmesa.so: cannot open shared object file: No such file or directory
[0227/174500.767921:VERBOSE1:gpu_init.cc(186)] gl::init::InitializeGLOneOff failed
[0227/174500.768250:ERROR:broker_posix.cc(41)] Invalid node channel message

Comment 11 by beeflam...@gmail.com, Mar 1 2017

Have the same issue with Chrome 56 in CentOS7 VM.

Kernel:
3.10.0-327.10.1.el7.x86_64

CPU:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
stepping	: 4
microcode	: 0x428
cpu MHz		: 2793.370
cache size	: 25600 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips	: 5586.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 62
model name	: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
stepping	: 4
microcode	: 0x428
cpu MHz		: 2793.370
cache size	: 25600 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 1
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips	: 5586.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

Chrome Version:
Google Chrome 56.0.2924.87

I tried running Chrome 56 with Xvfb and saw the same "Illegal instruction" in log, where Chrome 52 works fine for me.

Comment 12 by sheriffbot@chromium.org, Mar 1 2017

Project Member
Cc: thestig@chromium.org
Labels: -Needs-Feedback
Thank you for providing more feedback. Adding requester "thestig@chromium.org" to the cc list and removing "Needs-Feedback" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 13 by thestig@chromium.org, Mar 11 2017

Trying 58.0.3029.14, I'm seeing it exit and complaining about /opt/google/chrome-unsatble/libosmesa.so being missing. However, the exit code is 0 and there's no crash.

Comment 14 by thestig@chromium.org, Mar 11 2017

Installing libOSMesa and symlinking /opt/google/chrome-unstable/libosmesa.so to pint to libOSMesa.so.8 gets rid of those messages, but there's still no crash here.

Comment 15 by steve.tr...@renlearn.com, May 5 2017

I also encountered this issue on a jenkins build slave running centos 7 based on a minimal install.  The workaround of installing a font package worked for me and adding the mesa package/symlink suppressed the missing mesa library error

yum install mesa-libOSMesa-devel gnu-free-sans-fonts
ln -s /usr/lib64/libOSMesa.so.8 /opt/google/chrome-unstable/libosmesa.so

Comment 16 by jackpalm...@gmail.com, May 12 2017

I was facing this issue when trying to run Chrome headless in Jenkins Centos 7 and installing the fonts fixed it for me.

This was a step to enable Google/Lighthouse to run in jenkins, the only extra chrome param needed was --no-sandbox.

Out of interest Steve, how did you find the font fix?

Comment 17 by skyos...@chromium.org, May 15 2017

Components: Internals>Headless

Comment 18 by discrete...@gmail.com, Jul 17 2017

I faced the exact same issue with Google Chrome 59.0.3071.115 on CentOS 7 when I attempted to run the command "google-chrome --headless", and the fix suggested by Steve fixed it.

The error message I got was:

$ google-chrome --headless
[0717/061050.051485:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected
[0717/061050.051526:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected
Trace/breakpoint trap
[0100/000000.061179:FATAL:zygote_linux.cc(476)] Failed to synchronise with parent zygote process
[0717/061050.072793:ERROR:gl_implementation.cc(246)] Failed to load /opt/google/chrome/libosmesa.so: /opt/google/chrome/libosmesa.so: cannot open shared object file: No such file or directory

After installing mesa-libOSMesa-devel, I got this error:

$ google-chrome --headless
[0717/062429.641863:WARNING:audio_manager.cc(293)] Multiple instances of AudioManager detected
[0717/062429.641900:WARNING:audio_manager.cc(254)] Multiple instances of AudioManager detected
Trace/breakpoint trap

and after installing gnu-free-sans-font, google-chrome runs successfully.

Comment 19 by baron.ro...@gmail.com, Sep 12 2017

Experienced the same problem on Jenkins under CentOS 7 and install gnu-free-sans-fonts is all it took to get things working. Thanks for the solution Steve!

Comment 20 by skyos...@chromium.org, Oct 17 2017

libosmesa shouldn't be required anymore in the latest release now that we're using SwiftShader for rendering. Could you please re-test this on CentOS?

Comment 21 by thestig@chromium.org, Oct 17 2017

Labels: Needs-Feedback

Comment 22 by sheriffbot@chromium.org, Oct 18

Project Member
Status: Archived (was: Unconfirmed)
Issue has not been modified or commented on in the last 365 days, please re-open or file a new bug if this is still an issue.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 23 by arn...@webhare.nl, Oct 22

I've run into the same problem with a ubuntu 16.04-based docker on a centos 7 host. 

two identical docker images (though different data in them when running), both with 69.0.3497.100-1. one always fails, the other never, even though both dockers are running on the same VM (so no CPU difference). 

installing fonts-open-sans seems to fix it.

Comment 24 by thestig@chromium.org, Nov 30

Status: Unconfirmed (was: Archived)
Reopening since this still seems to be happening.

Would someone be willing to temporarily grant access to an affected VM? I can try logging in via ssh and see what's going wrong.

Sign in to add a comment