It looks like it's CHECK failing due at zygote_host_impl_linux.cc:197 due to

    CHECK_GT(real_pid, 1);

Do we have the output from that?  That's a pretty basic sanity check, so I'm really surprised that it could be failing.

CL 1976403002 didn't add that check, it just moved it around.  It used to be at content/browser/zygote_host/zygote_communication_linux.cc:352.  Can you check if there used to be crashes there too?

(Sorry, since I'm not on Chrome anymore, I no longer have crash.corp access myself.)

Assigning to one of the reviewers and tagging for a fix in M54.

That was written by kerrnel. I'm not a linux peep so I can do nothing here. Reassigning.

Under what criteria is this marked as a release block? Thank you.

I ran a dremel query to look for crashes at content/browser/zygote_host/zygote_communication_linux.cc:352 but found nothing.

Just to update the latest behavior of the bug, Crash is still seen on latest Beta and Dev channels with below instances. Last crash is observed on M54-54.0.2824.0 with 6 crash instances.

53.0.2785.70	7.81%	10 - Beta
54.0.2824.0	4.69%	6  - Dev

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:20

Though these crashes are very less in number and affecting few users but this is one of the top browser crash on Beta channel of Linux and is regressed in M-53 as per the Link attached in C#0.

kerrnel@: Could you please help in investigating this further. Please remove the blocker label if you feel so.

Appreciate your help! 

Investigating but I don't think this needs to be a blocker, as the actual numbers seem quite small.

It seems we don't get many crashes at all on Linux which is rather worrying, either that or we just don't have enough users.

Total browser crash reports >= M53 is 11711
Total caused by this signature is 135

so this is around 1.2% of all crashes. I don't think this is a RB

For all we know it's the same or a few broken users generating all the crashes - the client ID is server generated and not client generated in many of these. Taking crash/4cc2e26e00000000 as an example, reading the human readable bits of the .dmp file, it is some user running Ubuntu 16.04 on a laptop with what looks like a stock kernel.

We may want to ask ourselves if Chrome works on Ubuntu 16.04 out of the box for otherselves. If yes, then let's not worry about this. The user probably managed to shoot themselves in the foot somehow. Maybe they will eventually stop trying to start Chrome and fail every time, and file a bug. Then maybe we can work with the user to figure out how they managed to fail that CHECK().

Issue 645134 has been merged into this issue.

Looking at 53.0.2785.92 only. Total: 4769, this crash - 671 -> 14%. Based on the kernel versions and peeking at some dumps, I don't believe this is coming from a single user. Probably need to wait for one to complain and then figure out what's wrong.

Just to update the latest behavior of this crash, Issue is still seen on chrome latest stable and beta channels with below instances. Last crash is observed on 55.0.2868.3 with 15 instances and as of now no crashes observed on latest dev and canary channels.

53.0.2785.143	11.73%	300 - Stable	
54.0.2840.41	1.60%	41  - Beta

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:20

Just to update:

This crash is seen on latest builds as below

55.0.2883.21	0.28%	36	
55.0.2883.11	0.15%	20	
55.0.2868.3	0.21%	28	
54.0.2840.71	25.13%	3288	from 693 different client Ids
54.0.2840.59	48.78%	6381	
54.0.2840.50	0.26%	34	
54.0.2840.41	0.39%	51

Link to the builds:
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:20	

Currently this is the #2 browser crash in latest stable- 54.0.2840.71 and is gradually spiking for all stable release.

54.0.2840.71    4266 reports from 892 unique clients.
53.0.2785.143 	2455 reports from 607 unique clients.

Keeping in sheriffs queue for further debugging and monitoring.

I poked around a few of the crash reports and all of the affected clients I looked at are using kernel 4.4.0.  Could this be significant?

Nevermind, it's only 52% running 4.4.0

Just to update:

This is the top#2 browser crash on Linux in latest stable 54.0.2840.71

54.0.2840.71	39.56%	6878	from 1437 unique client Ids --- Stable
55.0.2883.28	0.14%	24	from 4 unique client ids  --- Beta
56.0.2902.0	0.01%	1                                 --- Dev

Currently this is top #1 browser crash on Linux having statistics as 31.64% and 274 crash instances from only 54 unique client Ids.

Link to list of builds where crashes are seen:
============================================
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

Added Releaseblock-Stable as its a top crasher now.Please modify if not applicable.

[Stability Sheriff]  This comment in unix_domain_socket.cc has a hint.  Is there an opportunity for the socket to be closed or a zero length message to be sent before this handshake completes?

  if (out_pid) {
    // |pid| will legitimately be -1 if we read EOF, so only DCHECK if we
    // actually received a message.  Unfortunately, Linux allows sending zero
    // length messages, which are indistinguishable from EOF, so this check
    // has false negatives.
    if (r > 0 || msg.msg_controllen > 0)
      DCHECK_GE(pid, 0);

    *out_pid = pid;
  }

I will look for any "user complaints."

mdempsky@, what exactly does false negative mean in this case? That it might think a message was not received when an EOF message was received?

I couldn't find anything specific enough filed through the wizard to associate with this.  There are a few users who filed general startup issues:   Bug 645398 ,  Bug 656347 .

Also, it's not clear to me how this overall handshake is supposed to work.  From some reading, the sending process has to send a message on the socket with a cmsg_type of SCM_CREDENTIALS populated with PID, UID, GID.   But I didn't find any code in the repo that does that.

Just to update its still the top # 1 browser crasher on Linux stable 54.0.2840.100 having 1621 crash instances from 372 unique client Ids.

https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20product.version%3D%2754.0.2840.100%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D

kerrnel@ Are you still working on this?  If not please unassign, otherwise no one will ever take a look at this issue

Thanks for the ping. Unfortunately, I made no progress investigating this crash so I will unassign it.

I only looked at the crash reports briefly and noticed that uptime <= 4.3 minutes and exploitability rating = INTERESTING for all crashes. Don't know if any of that is relevant :/

The crashes are not seen on M57 channel though, the crahses are huge on stable and reported from 1483 unique client Ids having 7023 crash instances.
56.0.2922.1	0.06%	24	--Dev
55.0.2883.59	0.01%	2	--Beta
54.0.2840.100	17.85%	7023    --Stable
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

Just to update its still the top # 1 browser crasher on Linux stable 54.0.2840.100 having 9809 crash instances from 1962 unique client Ids.

Just to update this is still the top # 1 browser crash on Linux stable having 13614 crash instances 2662 unique client Ids.
56.0.2924.3	0.01%	3	--Dev
56.0.2922.1	0.05%	26	--prev Dev
55.0.2883.59	0.10%	47	--Beta
54.0.2840.100	28.25%	13614	--Stable
54.0.2840.90	18.05%	8700	--prev stable
Added Releaseblock-Stable against M55 as M55 stable is nearing and would be good if it can be fixed before the stable push.

We are still seeing these crashes on Chrome_linux on all Chrome channels,  thomasanderson@ can we please find someone to take a look at this issue please given this been present in two Stable Mile stones i.e., M53 and M54 probably in M55(about to go stable. 


Note : I am removing the stable blocker given crashes are present in M53 and M54, but tagging it as M56 stable blocker. Please feel free to tag as M55 if you think that's appropriate.  

I think that in order for anything to be done about this, we need a repro, and AFAIK no one on Chrome team has hit this issue.  I also haven't seen any user complaints

Maybe we can unrestrict this and add something like "Chrome fails to launch" in the title in hopes of finding any affected users?

I found  bug 662407  reported by a user with this issue, however I don't want to merge since then users won't be able to find this bug

I found out this crash only occurs when running Chrome as root.  

AFAIK this configuration isn't supported, and Chrome used to refuse to run as root in earlier versions, exiting with a useful error message.  

However, now I'm able to run chrome as root without any error message.  So it seems this is just a matter of adding this check back.  I'll do a bisect and find out where this regression occurred.

 Issue 662407  has been merged into this issue.

I wanted to give a quick update as this is P1

So the check that I was thinking of is here:
https://cs.chromium.org/chromium/src/chrome/browser/ui/views/chrome_browser_main_extra_parts_views.cc?rcl=0&l=76

I guess we do support running as root, but only with a different user data dir?

The issue is caused by the user namespace sandbox.  For some reason, after setting up the sandbox as root, Chrome cannot execve() itself, because it can't stat the executable.  mdempsky@ or rickyz@ do you know why this could happen?

So I guess there are some options:
1.  Exit early when trying to run as root, printing an error message.  We won't be able to display a warning dialog like we do in CBMEPV
2.  Try to avoid execve() and instead call ZygoteMain directly
3.  Figure out why the namespace sandbox is forbidding stat'ing the chrome executable and fix it

Just to update:

This is the Top#1 browser crash on Linux dev version 56.0.2924.10 with 9 instances from 5 different client Ids

56.0.2924.10	0.02%	9	
56.0.2924.3	0.01%	7	
56.0.2922.1	0.05%	27	
56.0.2914.3	0.02%	11	
56.0.2906.0	0.04%	23	
56.0.2902.0	0.00%	2	
56.0.2897.0	0.01%	7	
55.0.2883.59	0.16%	86	
55.0.2883.52	0.04%	21	
55.0.2883.44	0.09%	51	
55.0.2883.35	0.12%	63	
55.0.2883.28	0.09%	51	
55.0.2883.21	0.07%	39	
55.0.2883.18	0.00%	2	
55.0.2883.11	0.04%	20	
55.0.2883.9	0.00%	1	
55.0.2882.0	0.01%	7	
55.0.2873.0	0.08%	44	
55.0.2868.3	0.05%	28	
55.0.2859.0	0.03%	16	
55.0.2853.0	0.01%	5	
54.0.2840.100	33.14%	17825	
54.0.2840.90	17.05%	9168	
54.0.2840.71	22.70%	12212	
54.0.2840.59	19.12%	10285	
54.0.2840.50	0.09%	46	
54.0.2840.41	0.10%	53	
54.0.2840.34	0.08%	42	
54.0.2840.27	0.05%	28	
54.0.2840.16	0.36%	192	
54.0.2840.14	0.02%	10	
54.0.2840.8	0.10%	55	
54.0.2840.6	0.00%	1	
54.0.2838.0	0.00%	1	
54.0.2837.0	0.02%	9	
54.0.2824.0	0.01%	7	
54.0.2816.0	0.00%	1	
54.0.2810.2	0.01%	6	

Observing a spike in stable channel.

Link to the builds:
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

thomasanderson: are you sure it's the stat of the executable that fails? If yes, did you try with Chrome installed as root (as a normal user would) or with a dev Chromium build (typically the executable is owned by your regular user id).

Which line of code do you specifically see failing on the Zygote side?

Yes I'm sure it's the stat that fails.  The executable is located in /usr/local/google/home/thomasanderson/dev/chromium_2/src/out/chrome.

stat "/" good
...
stat "/usr/local/google/home/thomasanderson/dev/" good
stat "/usr/local/google/home/thomasanderson/dev/chromium_2" bad
...
stat "/usr/local/google/home/thomasanderson/dev/chromium_2/src/out/chrome" bad

Since we're root, you'd think we could stat any file, right?  I'll try installing a debug build in a bit, but for now, when I run official Chrome as root, I also get a failure.

This is the line that fails:
https://cs.chromium.org/chromium/src/base/process/launch_posix.cc?rcl=0&l=509

I don't have the full stack trace, but I suspect this is being run after we drop capabilities, so no you can't stat any files as root anymore. You can only stat files that have uid 0 at this point.

But this shouldn't be hitting users, because users will have Chrome installed as root. However, what could be hitting users is trying to open files owned by the user (in their Chrome data directory) while running as root.

It still feels strange that so many users would "su" and run Chrome though. Did you try to login as root in a VM and run Chrome in there to see what happens? (we should still crash/abort with a verbose message in this case IIRC).

Julien

(err, I meant that have uid 0, or are world-readable, assuming you can traverse the path, etc. You understood me, I'm sure).

Just to update the crashes are seen on latest channels as below.

56.0.2924.18	0.00%	1	- Dev
55.0.2883.75	4.16%	2495	- Beta & Stable

Link to the list of builds:

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

Observing huge spike on Beta & Stable channels.

@thomasanderson : Could you please look into this issue.

Thank you!

Latest crash statistics on all latest channels are as below.

57.0.2944.0	0.00%	1	
56.0.2924.18	0.01%	7	
55.0.2883.75	6.58%	4159	

Link to the list of builds
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

@thomasanderson : Could you please look into this issue.

jln@ I tried an official build and the stat succeeds now with /usr/bin/google-chrome-stable.

Except now I get:
[1:1:1212/204159.465561:ERROR:launch_posix.cc(374)] fork: Operation not permitted
[1:1:1212/204159.468009:FATAL:sandbox_linux.cc(180)] Check failed: sandbox::Credentials::MoveToNewUserNS().

And then on the browser-side, the age-old CHECK:
[8687:8687:1212/204159.498081:FATAL:zygote_host_impl_linux.cc(196)] Check failed: ReceiveFixedMessage(fds[0], kZygoteHelloMessage, sizeof(kZygoteHelloMessage), &real_pid).

Also reducing to Pri2 since this is an unsupported configuration.

Your bug is labelled as Stable Release Block, please make sure to land the fix and get it merged into the release branch ASAP so we can take it for next week's Beta release for Desktop. Thank you!

jln: Tom points out that we have code to explicitly show a warning message when running Chrome as root (https://cs.chromium.org/chromium/src/chrome/browser/ui/views/chrome_browser_main_extra_parts_views.cc?rcl=0&l=76).

But if users proceed, they end up with sandboxed processes running as root. That seems really bad to me, but then we don't have non-root ids to privdrop back to anyway.

IMO, we should just refuse to run as root completely.

WDYT?

Just to update:

This crash is seen in latest builds as below

57.0.2950.4	0.02%	22	-Dev
56.0.2924.28	0.08%	72	-Beta
55.0.2883.87	16.14%	14665	-Stable

Link to the list of builds:
---------------------------
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productname,productversion:1000

@thomasanderson : Seeing huge spikes on stable.Could you please look on this issue.

Thank you!

Just to update:

Behavior of this crash seen on latest channels is as follows:

57.0.2950.4	0.03%	25	-Dev
56.0.2924.28	0.08%	74	-Beta
55.0.2883.87	18.84%	17891	-Stable

Link to the list of builds:
---------------------------
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productname,productversion:1000

thomasanderson@ : As this is a release block stable, could you please have a look on this issue.

Thanks...!!

Latest crash rates on all channels are as below
57.0.2970.0	0.00%	2	
56.0.2924.51	0.01%	13
55.0.2883.87	24.85%	26077		

Link to the list o fbuilds
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27%20AND%20product.name%3D%27Chrome_Linux%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productname,productversion:1000

thomasanderson@ Please look into this stable blocker issue.

Thanks,

Thanks for the investigation, can we have the latest updates on this issue?

FYI: This bug is tagged as M56 RB Stable, we are planning to ship stable soon. It would be great to have a fix asap.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/357c17552fb353ea9f3de6eca8a47b2d009067c8

commit 357c17552fb353ea9f3de6eca8a47b2d009067c8
Author: thomasanderson <thomasanderson@google.com>
Date: Wed Jan 11 23:55:46 2017

Namespace sandbox: add check for unprivileged use of CLONE_NEWUSER

Debian 8 restricts use of CLONE_NEWUSER to only processes with
CAP_SYS_ADMIN. (https://github.com/semplice/linux/blob/master/debian/patches/debian/add-sysctl-to-disallow-unprivileged-CLONE_NEWUSER-by-default.patch)
Chrome was previously checking if the kernel supported CLONE_NEWUSER
by running clone(CLONE_NEWUSER, ...) with the same capabilities chrome
was launched with.  This leads to 2 scenarios:

1. If Chrome was run as root:
   The check for CLONE_NEWUSER will succeed.  Chrome will then set up
   the namespace sandbox by clone()'ing and dropping CAP_SYS_ADMIN.
   Subsequent clone()'s with CLONE_NEWUSER will then fail.

2. If Chrome was run as a normal user:
   The check for CLONE_NEWUSER will fail.  Chrome will fallback to
   using the setuid sandbox.

The solution is to simply drop CAP_SYS_ADMIN before the check.

In addition, this CL disallows running Chromium as root unless launched
with --no-sandbox.

BUG= 638180 

Review-Url: https://codereview.chromium.org/2578483002
Cr-Commit-Position: refs/heads/master@{#443062}

[modify] https://crrev.com/357c17552fb353ea9f3de6eca8a47b2d009067c8/chrome/app/generated_resources.grd
[modify] https://crrev.com/357c17552fb353ea9f3de6eca8a47b2d009067c8/chrome/browser/ui/views/chrome_browser_main_extra_parts_views.cc
[modify] https://crrev.com/357c17552fb353ea9f3de6eca8a47b2d009067c8/sandbox/linux/services/credentials.cc
[modify] https://crrev.com/357c17552fb353ea9f3de6eca8a47b2d009067c8/sandbox/linux/suid/client/setuid_sandbox_client.cc

This bug requires manual review: There is .grd file changes and we are only 18 days from stable.
Please contact the milestone owner if you have questions.
Owners: amineer@(clank), cmasso@(bling), gkihumba@(cros), bustamante@(desktop)

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

LGTM for merging into M56 (2924)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8b6103c5b1029b52feb6f34e2594d7e125a28f3e

commit 8b6103c5b1029b52feb6f34e2594d7e125a28f3e
Author: thomasanderson <thomasanderson@google.com>
Date: Tue Jan 17 23:57:46 2017

Namespace sandbox: add check for unprivileged use of CLONE_NEWUSER

> Debian 8 restricts use of CLONE_NEWUSER to only processes with
> CAP_SYS_ADMIN. (https://github.com/semplice/linux/blob/master/debian/patches/debian/add-sysctl-to-disallow-unprivileged-CLONE_NEWUSER-by-default.patch)
> Chrome was previously checking if the kernel supported CLONE_NEWUSER
> by running clone(CLONE_NEWUSER, ...) with the same capabilities chrome
> was launched with.  This leads to 2 scenarios:
>
> 1. If Chrome was run as root:
>    The check for CLONE_NEWUSER will succeed.  Chrome will then set up
>    the namespace sandbox by clone()'ing and dropping CAP_SYS_ADMIN.
>    Subsequent clone()'s with CLONE_NEWUSER will then fail.
>
> 2. If Chrome was run as a normal user:
>    The check for CLONE_NEWUSER will fail.  Chrome will fallback to
>    using the setuid sandbox.
>
> The solution is to simply drop CAP_SYS_ADMIN before the check.
>
> In addition, this CL disallows running Chromium as root unless launched
> with --no-sandbox.
>
> BUG= 638180 
>
> Review-Url: https://codereview.chromium.org/2578483002
> Cr-Commit-Position: refs/heads/master@{#443062}

NOTRY=true
NOPRESUBMIT=true
BUG= 638180 
TBR=sky@chromium.org,mdempsky@chromium.org

Review-Url: https://codereview.chromium.org/2641513004
Cr-Commit-Position: refs/branch-heads/2924@{#785}
Cr-Branched-From: 3a87aecc31cd1ffe751dd72c04e5a96a1fc8108a-refs/heads/master@{#433059}

[modify] https://crrev.com/8b6103c5b1029b52feb6f34e2594d7e125a28f3e/chrome/app/generated_resources.grd
[modify] https://crrev.com/8b6103c5b1029b52feb6f34e2594d7e125a28f3e/chrome/browser/ui/views/chrome_browser_main_extra_parts_views.cc
[modify] https://crrev.com/8b6103c5b1029b52feb6f34e2594d7e125a28f3e/sandbox/linux/services/credentials.cc
[modify] https://crrev.com/8b6103c5b1029b52feb6f34e2594d7e125a28f3e/sandbox/linux/suid/client/setuid_sandbox_client.cc

Just to update the crashes are still seen on Linux stable # 57.0.2987.110 having 3084 crashes from 707 client Ids and ranked 3rd in Linux browser crashes.

Link to list of builds where crashes are seen:
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.channel%3D%27%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

59.0.3047.0	0.00%	2	--Canary
58.0.3029.19	0.02%	32	--Beta
57.0.2987.110	1.67%	3084	--Stable 
thomasanderson@: Could you please take a look into this.

Issue 705475 has been merged into this issue.

Reopening since the crash still exist in current Milestones M57, M58 and M59. 
Tom, please take a look, it would be great to have a fix before M58 hits stable.

Just to update the latest behaviour of crash,

This crash is seen in all channels as below

59.0.3053.3	0.02%	46        --- Latest Dev
58.0.3029.41	0.01%	25        --- Latest Beta
57.0.2987.133	1.13%	2100      --- Latest Stable	

Link to the builds:
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.channel%3D%27%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

thomasanderson@ - Could you please have a look into this issue.

Thanks...!!

A friendly reminder that M58 Stable is launch is coming soon (less than 2 weeks)! Your bug is labelled as Stable ReleaseBlock, pls make sure to land the fix and get it merged into the release branch ASAP so it gets enough baking time in Beta (before Stable promotion). Thank you!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7bc960609bc49b21be3240970b449cd1146af615

commit 7bc960609bc49b21be3240970b449cd1146af615
Author: cernekee <cernekee@chromium.org>
Date: Tue Apr 11 04:18:23 2017

Fix unprivileged user namespace regression

Prior to crrev.com/357c17552fb353ea9f3de6eca8a47b2d009067c8 ("Namespace
sandbox: add check for unprivileged use of CLONE_NEWUSER") it was
possible to run the entire browser inside an unprivileged user namespace.
This is useful because it lets normal desktop users create separate
network namespaces for different browser instances.  It is often used
to implement per-app VPN support on Linux[1], as an alternative to
having the VPN tunnel all traffic for the entire system.

Add a new check to differentiate the "bad" case of "Chrome running as
global UID 0" from the "good" case of "Chrome running as UID 0 inside
an unprivileged namespace."

Also, add missing includes for std::move and geteuid().

[1] https://github.com/cernekee/ocproxy#vpnns-experimental

BUG= 638180 
TEST=test by hand using vpnns

Review-Url: https://codereview.chromium.org/2707763002
Cr-Commit-Position: refs/heads/master@{#463532}

[modify] https://crrev.com/7bc960609bc49b21be3240970b449cd1146af615/chrome/browser/ui/views/chrome_browser_main_extra_parts_views.cc

Thanks for the fix, please verify in canary once it lands. If all goes well please request a merge to M58 ASAP.

FYI: M58 stable promotion is scheduled next week and RC cut on Monday 04/17.

c#64 The cl in c#65 was not meant to fix this crash.

Also, unassigning myself as I am thoroughly stumped by this.  I thought this issue occurred when running as root, but there are still a top crasher even after the CL in c#52.  There's also no startup bugs filed by users :S

Perhaps someone from the security team could pick this up?

I'm able to repro this issue under the following conditions:
1. Build chrome, but do not install it (leave it in the out directory)
2. Run chrome from the out directory as root

[56161:56161:0412/163245.290222:FATAL:zygote_host_impl_linux.cc(182)] Check failed: ReceiveFixedMessage(fds[0], kZygoteBootMessage, sizeof(kZygoteBootMessage), &boot_pid).

+cernekee@ You said on this CL https://codereview.chromium.org/2707763002 that changing the permissions of the out dir to 0755 fixes the issue, but I'm unable to get that working.

Also, a Google search for "Check failed: ReceiveFixedMessage" shows that users ARE hitting this issue (when running as root).

+cernekee@ for real

The key symptom for me was "failed to execvp": https://pastebin.com/sz7UhJkV

I don't see that on most of the top results for [Check failed: ReceiveFixedMessage] so those might indicate a different failure.

On my setup I verified the execve() error with strace, and then looked at each element of the path to see what might have been causing the denial.

Latest crash behaviour as below

59.0.3067.0	0.00%	8	
58.0.3029.68	0.01%	12	
58.0.3029.54	0.02%	30	
57.0.2987.133	7.08%	13284	

Link to the list of builds
https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.channel%3D%27%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,productversion:1000

Could any one please look into this stable blocker issue.

Thanks,

Just a heads up since this is marked as RB-Stable - we are aiming to launch M58 early stable this Wednesday, RC cut today @ 5:00 PM PT or latest by tomorrow noon if all goes well. Can you please take a look at this urgently and confirm if it's still a blocker for M58?

Removing ReleaseBlock-Stable.  This crash only occurs when running chrome as root, which is an unsupported configuration.

Just to update, latest crash rates on all channels are as below:
This crash not seen on latest Canary & Dev.

59.0.3071.29	0.01%	26	-Beta
58.0.3029.81	7.98%	17969	-Stable

Link to the list of builds:
---------------------------
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Observing huge spikes in latest stable.Could some one from "CC" please look in to this issue.

Thank You!

Just to update the latest behaviour of crash,

This crash is seen only in stable channel as below:

58.0.3029.110	1.22%	2956    --- Latest Stable
Note: Crash is not seen in latest dev, beta and canary channels.	

Link to the builds:
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Thanks...!!

Just to update, latest crash rates on all channels are as below:
This crash not seen on latest Canary.

60.0.3100.0	0.00%	3	-Dev
59.0.3071.61	0.01%	32	-Beta
58.0.3029.110	8.43%	20873	-Stable

Link to the list of builds:
---------------------------
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Could some one from CC please look in to this issue.

Thank You!

Just to update the latest behaviour of crash,

This crash is seen in all channels as below

59.0.3071.71	0.02%	54        --- Latest Beta
58.0.3029.110	14.36%	37569     --- Latest Stable	

Link to the builds:
https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Could anyone from Content>Core team please have a look into this issue.

Thanks...!!

User experienced this crash while running command

`sudo google-chrome-beta --remote-debugging-port=9222 --enable-benchmarking --enable-net-benchmarking`

The command quits after this output:

--2017-06-05 20:42:25--  https://clients2.google.com/cr/report
Resolving clients2.google.com (clients2.google.com)... 216.58.197.78, 216.58.197.78, 216.58.197.78, ...
Connecting to clients2.google.com (clients2.google.com)|216.58.197.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/dev/fd/4’

     0K
 Crash dump id:  20cb2e2e40000000 
                                                    --2017-06-05 20:42:26--  https://clients2.google.com/cr/report
Resolving clients2.google.com (clients2.google.com)... 216.58.197.78, 2404:6800:4007:800::200e
Connecting to clients2.google.com (clients2.google.com)|216.58.197.78|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘/dev/fd/4’


     0KCrash dump id:   d6ce91bef0000000 
                                                      806K=0s
Forum : https://productforums.google.com/forum/#!topic/chrome/F6dF_CFC4X4

Just to update the latest behavior of the crash.

This crash is still observed on latest beta and stable. Below are the instances.

60.0.3112.32	0.01%	24 .      -  Beta
59.0.3071.104	1.05%	3073	  -  Stable	

Link to the list of the builds
==============================

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Thanks!!

[desktop stability sheriff] This signature now accounts for nearly 50% of browser crashes on Linux Stable.[1] It has a strong effect on our crash per page load health metrics. See issue 736476.

Reading this bug, it seems like Chrome takes pains to allow running as root (c#56), yet we also say it's not a supported configuration (c#71). Naive question: if it's not supported, could we just print an error message and quit instead of crashing? I could easily create a CHECK failure using the repro steps in c#66.

[1] https://crash.corp.google.com/browse?q=product.name%3D%27Chrome_Linux%27%20AND%20custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20product.Version%3D%2759.0.3071.109%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D

As mentioned above, this accounts for more than half the browser crashes on Linux stable now. Is there anything that could explain the increased rate of crash?

There's a numerology to the crash reports where roughly (for small counts, exactly) half these crashes are SIGTRAP and half are SIGILL. I guess that's the forked process and the parent process dying together?

Just to update the latest behavior of the crash.

This crash is still observed on latest dev, beta and stable. Below are the instances.

61.0.3141.7	0.01%	18       - Dev
60.0.3112.50	0.01%	38       -  Beta
59.0.3071.115	6.48%	21221	 -  Stable	

Link to the list of the builds
==============================

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Thanks!!

(stability sheriff hat)

Given this is top browser crash on Linux Stable (50% of browser crashes), could you help us find an owner for this and/or have a decision made on how to proceed here? (re: comment 79)

jschuh@, Any update on finding an appropriate owner for this issue?

Reassigning to the person who erroneously assigned this to me.

jschuh: I assigned to you because (my understanding is) that you manage Chrome Linux development and thus would be the right person to find an owner for this. Is that not correct?

Note: I was stability sheriff when I assigned earlier. I am not anymore so please coordinate with the stability sheriff currently on rotation.

[Stability sheriff]

jschuh@ told me offline that they're not the best person to own this
jln@ hasn't visited the bug tracker in over 30 days so probably also not the right owner

Assigning to jorgelo@ who's listed as the owner for sandbox_ipc_linux

jorgelo@ can you take a quick look and decide what the next steps here are? This is a top crasher on linux stable (see issue 736476).

We should just not support running as root. What I don't understand is:

CL from c#52 should have fixed things, but Thomas can still repro the bug. So the CL from c#52 is incomplete?

Can't we just error out when we run as global UID 0? Is that not what we're doing?

c#89.  We do have a check that forbids root, but it happens later in initialization because it pops up an Aura dialog giving the error message.  This crash happens in early sandbox initialization.

The CL in c#52 fixed one instance of the issue, but apparently there are more, as per c#66.

So I think to really fix this we need to either:
1. Add the check that forbids root in the sandbox and sacrifice having a pretty dialog.  The main downside to doing this would be confusing users who launch chrome from a gui instead of the command line (aka most people)
2. Fix the case in c#66 and possibly others

At this point, I'm fine with 1.

If we're drowning out other crashes, we should make this error out without crashing, yeah. At least for the time being.

Just to update the latest behavior of the crash.

This crash is still observed on latest dev, beta and stable. Below are the instances.

61.0.3153.4	0.01%	24	-   Dev
60.0.3112.66	0.01%	20	-   Beta
59.0.3071.115	11.18%	40042	-   Stable

Link to the list of the builds
==============================

https://crash.corp.google.com/browse?q=custom_data.ChromeCrashProto.ptype%3D%27browser%27%20AND%20custom_data.ChromeCrashProto.magic_signature_1.name%3D%27content%3A%3AZygoteHostImpl%3A%3ALaunchZygote%27&ignore_case=false&enable_rewrite=true&omit_field_name=&omit_field_value=&omit_field_opt=%3D#samplereports:5,+productversion:1000

Thanks!!

I'll take a shot at writing a small CL to exit instead of crashing.

https://chromium-review.googlesource.com/c/585603/ draft CL up, need to figure out the best way to structure some helper functions.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa99e0990143359527b93a52c69dc2ec1a939fb3

commit aa99e0990143359527b93a52c69dc2ec1a939fb3
Author: Jorge Lucangeli Obes <jorgelo@google.com>
Date: Wed Jul 26 18:22:24 2017

Exit immediately if running as root without --no-sandbox.

BUG= 638180 
TEST=Before, sudo out/chrome crashes.
TEST=After, sudo out/chrome prints error message.

Change-Id: I4d2b03088b2d70b8431f41306e0edf4330ea61fe
Reviewed-on: https://chromium-review.googlesource.com/585603
Reviewed-by: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Julien Tinnes <jln@chromium.org>
Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org>
Cr-Commit-Position: refs/heads/master@{#489701}
[modify] https://crrev.com/aa99e0990143359527b93a52c69dc2ec1a939fb3/content/browser/zygote_host/zygote_host_impl_linux.cc
[modify] https://crrev.com/aa99e0990143359527b93a52c69dc2ec1a939fb3/sandbox/linux/services/credentials.cc
[modify] https://crrev.com/aa99e0990143359527b93a52c69dc2ec1a939fb3/sandbox/linux/services/credentials.h

Let me know if the crashing numbers get better, and feel free to file another bug.

for watching the stability, the CL in #96 landed in 62.0.3168.0. not yet in dev.

Looks like r489701 did help (if not entirely eliminating these crash reports).  Here's the drop in crashes when 62.0.3168.0 hit dev channel:

62.0.3180.0	0.00%	2	
62.0.3178.0	0.00%	4	
61.0.3163.39	0.01%	55	
61.0.3163.31	0.00%	21