New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 660868 link

Starred by 4 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug

Blocked on:
issue 669633



Sign in to add a comment

Enterprise-originated crash reports should be tagged

Project Member Reported by siggi@chromium.org, Oct 31 2016

Issue description

Enterprise users are severely under-represented in UMA and crash. This leads to Stable regressions for Enterprise environments, as even when we have representative crash samples on the lower channels, they're not often not common enough to get attention.
To allow up-weighing these issues, it's necessary to tag enterprise-originated crashes.

Looks like we have two main signals: 
1. The GGRV brand code = MSI.
2. The computer is domain-joined.
 

Comment 1 by siggi@chromium.org, Oct 31 2016

Components: Enterprise
Cc: mmand...@chromium.org
A couple of other signals:
- Has at least one policy applied
- Is running a Pro version of Windows (we haven't tested this one yet to see if it's good or not, but worth mentioning).

Comment 4 by siggi@chromium.org, Nov 1 2016

@georgesak: as you're trying to select for a small sub-population, you have to be EXCEEDINGLY wary of false positives. Imagine for argument's sake the enterprise population is 1% of Canary. If you introduce a signal with 100% true positive rate, but 2% false positive rate, then that signal will apply to the 1%, but also to 2% of the remaining 99%. As result, your selection is diluted 1/3rd.

@georgesak, @blumberg: are you going to take this bug on to acquire the base signal?
Cc: -georgesak@chromium.org
Owner: georgesak@chromium.org
Status: Assigned (was: Untriaged)
I'll tackle it, assigning it to myself.
As an enterprise user with 1000+ installs who does NOT turn off telemetry, I appreciate you're keeping us in mind!
Cc: ligim...@chromium.org
Project Member

Comment 9 by bugdroid1@chromium.org, Nov 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a185fbad8292f03144c35895c04db3fbe63c43a9

commit a185fbad8292f03144c35895c04db3fbe63c43a9
Author: iclelland <iclelland@chromium.org>
Date: Tue Nov 29 20:26:42 2016

Revert of Add a crash key to record whether a windows machine is domain joined. (patchset #3 id:60001 of https://codereview.chromium.org/2514483002/ )

Reason for revert:
Reverting this; it appears to be causing 140+ failures in telmetry tests on Win7 (See the failures in https://uberchromegw.corp.google.com/i/chromium.win/builders/Win7%20Tests%20%28dbg%29%281%29/builds/55150/steps/telemetry_unittests%20on%20Windows-7-SP1/logs/stdio )

As far as I can tell (the log output is being suppressed,) it looks it's hitting the DCHECK in SetCrashKeyValue that would indicate that the key isn't registered as expected.

Also filed https://bugs.chromium.org/p/chromium/issues/detail?id=669633

Original issue's description:
> Add a crash key to record whether a windows machine is domain joined.
>
> BUG=660868
>
> Committed: https://crrev.com/bf33256aee8db58c30b9d36e991055f2a4ed62a4
> Cr-Commit-Position: refs/heads/master@{#435018}

TBR=siggi@chromium.org,sky@chromium.org,georgesak@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=660868

Review-Url: https://codereview.chromium.org/2536293002
Cr-Commit-Position: refs/heads/master@{#435069}

[modify] https://crrev.com/a185fbad8292f03144c35895c04db3fbe63c43a9/chrome/browser/chrome_browser_main.cc
[modify] https://crrev.com/a185fbad8292f03144c35895c04db3fbe63c43a9/chrome/common/crash_keys.cc
[modify] https://crrev.com/a185fbad8292f03144c35895c04db3fbe63c43a9/chrome/common/crash_keys.h

Comment 10 by kbr@chromium.org, Nov 29 2016

Blockedon: 669633
Thanks!
I filed bugs on Crash server side to surface this bit and to affect signatures prioritization.

Qq, are enterprise clients all on stable channel? So we shouldn't expect this bit set until M57 is on stable, right?
This is an attempt to flag the few enterprise clients that run Canary/Dev/Beta with UMA&crash reporting turned on. Hopefully we'll see a few of these on the lower channels.
We consider users being enterprise if either they are domain joined (any channel) or their brand code is GGRV, which is the brand code for the MSI installer (Stable only).

Instead of surfacing this bit, would it make more sense to have an "enterprise" label and label users that match either criterion?
Ohk, sorry for the confusion. 
So the submitted CL marks reports from users that are domain joined.
Are the GGRV brand-coded not marked yet?
We get the brand code in the system profile proto that P-A added.

Comment 17 Deleted

George, Siggi, thanks for clarifying. I updated the crash bugs accordingly.
Project Member

Comment 19 by bugdroid1@chromium.org, Jan 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/80353b52b8d21e3481f6e1b69eec86f3399bc5cf

commit 80353b52b8d21e3481f6e1b69eec86f3399bc5cf
Author: georgesak <georgesak@chromium.org>
Date: Tue Jan 10 21:18:51 2017

Add enrolled to domain bit to crash keys in renderer process.

This is a follow up to 2514483002 which adds this flag to the browser process.

BUG=660868

Review-Url: https://codereview.chromium.org/2618623003
Cr-Commit-Position: refs/heads/master@{#442690}

[modify] https://crrev.com/80353b52b8d21e3481f6e1b69eec86f3399bc5cf/content/renderer/render_process_impl.cc

Comment 20 by term...@gmail.com, Mar 9 2017

I recently upgraded from Chrome 56.0.2924.87 to Chrome 57.0.2987.98 on Windows 7 x64 Enterprise and now Chrome will not run. On these machines there is a proprietary tool that locks down the workstation and limits certain types of access to some devices such as \Device\NamedPipe\wkssvc. Because of this the new Chrome will not run.

I bisected Chromium:
You are probably looking for a change made after 442679 (known good), but no later than 442694 (first known bad).
CHANGELOG URL:
  https://chromium.googlesource.com/chromium/src/+log/42968739363fbecc9515b3631d2e05ac7046fb02..63f9112d35ca0514c9eab4b8bc538f277a6343b0

On review of the changes made by these commits this is the only one I can see likely to have caused the issue. While I'm aware this is atypical I would ask that you consider offering a way to disable whatever this is doing, or make it continue on error, should it turn out to be the problem. If you have a trybuild (firefox dialect) or some such I can try it for you to help narrow the problem further and confirm this is the offending commit. Also if you'd prefer I open a new issue that is fine as well.

Thanks
@20

Please do file a new bug and assign it to me, I'll look into it. Thank you!

Comment 22 by term...@gmail.com, Mar 11 2017

George I filed at https://bugs.chromium.org/p/chromium/issues/detail?id=700615

I don't see a way to CC anyone. The only way to create the bug was through the wizard. I may not have the required privileges.
Project Member

Comment 23 by bugdroid1@chromium.org, Mar 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/bb0b93c87ade82c71918630893164754ea4a1f1e

commit bb0b93c87ade82c71918630893164754ea4a1f1e
Author: georgesak <georgesak@chromium.org>
Date: Mon Mar 13 20:38:47 2017

Reverting 2618623003.

Call to IsOS will fail if \Device\NamedPipe\wkssvc is blocked (ie. Chrome is running in a sandbox).

BUG= 700615 ,660868
TBR=nasko

Review-Url: https://codereview.chromium.org/2747873002
Cr-Commit-Position: refs/heads/master@{#456469}

[modify] https://crrev.com/bb0b93c87ade82c71918630893164754ea4a1f1e/content/renderer/render_process_impl.cc

Project Member

Comment 24 by bugdroid1@chromium.org, Mar 14 2017

Labels: merge-merged-2987
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/7e40940cf24e1f459433a1f7207217ee6b37958a

commit 7e40940cf24e1f459433a1f7207217ee6b37958a
Author: georgesak <georgesak@chromium.org>
Date: Tue Mar 14 20:36:13 2017

Reverting 2618623003.

Call to IsOS will fail if \Device\NamedPipe\wkssvc is blocked (ie. Chrome is running in a sandbox).

BUG= 700615 ,660868
TBR=nasko
NOTRY=true
NOPRESUBMIT=true

Review-Url: https://codereview.chromium.org/2747873002
Cr-Commit-Position: refs/heads/master@{#456469}
(cherry picked from commit bb0b93c87ade82c71918630893164754ea4a1f1e)

Review-Url: https://codereview.chromium.org/2749863003
Cr-Commit-Position: refs/branch-heads/2987@{#823}
Cr-Branched-From: ad51088c0e8776e8dcd963dbe752c4035ba6dab6-refs/heads/master@{#444943}

[modify] https://crrev.com/7e40940cf24e1f459433a1f7207217ee6b37958a/content/renderer/render_process_impl.cc

Project Member

Comment 25 by bugdroid1@chromium.org, Mar 14 2017

Labels: merge-merged-3029
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/81d88b89a7d02871ec89836c626c7de314498b81

commit 81d88b89a7d02871ec89836c626c7de314498b81
Author: georgesak <georgesak@chromium.org>
Date: Tue Mar 14 20:50:41 2017

Reverting 2618623003.

Call to IsOS will fail if \Device\NamedPipe\wkssvc is blocked (ie. Chrome is running in a sandbox).

BUG= 700615 ,660868
TBR=nasko
NOTRY=true
NOPRESUBMIT=true

Review-Url: https://codereview.chromium.org/2747873002
Cr-Commit-Position: refs/heads/master@{#456469}
(cherry picked from commit bb0b93c87ade82c71918630893164754ea4a1f1e)

Review-Url: https://codereview.chromium.org/2747083004
Cr-Commit-Position: refs/branch-heads/3029@{#196}
Cr-Branched-From: 939b32ee5ba05c396eef3fd992822fcca9a2e262-refs/heads/master@{#454471}

[modify] https://crrev.com/81d88b89a7d02871ec89836c626c7de314498b81/content/renderer/render_process_impl.cc

Comment 26 by siggi@chromium.org, Mar 15 2017

Cc: mark@chromium.org
It looks like this would be better done in the Crashpad handler, post-crash. This would avoid the sandboxing issues, plus would avoid doing redundant work for non-crashing processes.
Can we get at the brand code and the other bits of information you need outside the browser process?

Comment 27 by siggi@chromium.org, Mar 15 2017

Cc: siggi@chromium.org

Comment 28 by mark@chromium.org, Mar 15 2017

The brand code is known by install_static, right? I think we have that somewhat handy when starting crashpad_handler. We could set the brand code as a process annotation when we start the handler. This is the way that we deal with the product version and channel, and the brand code is similar to those.

Sign in to add a comment