New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 412384 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug

Blocked on:
issue 464733



Sign in to add a comment

Watch and log browser exit status

Project Member Reported by siggi@chromium.org, Sep 9 2014

Issue description

There is approximately one crash dump uploaded per 20 unclean shutdowns. As far as I can tell, this ratio has never been better than 1/10 or so, and it's not understood why this is so. If this is due to some kind of a random sampling effect, then there's no issue here, but otherwise we may be blind to the cause for ~95% of crashes in the field.
I'm going to add code to watch for the browser's exit code, to try and get metrics for the main causes behind these mystery unclean shutdowns.
 

Comment 1 by siggi@chromium.org, Sep 9 2014

Cc: -a deleted user erikwright@chromium.org
Project Member

Comment 3 by bugdroid1@chromium.org, Nov 17 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29

commit c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29
Author: siggi <siggi@chromium.org>
Date: Mon Nov 17 23:29:19 2014

Browser watcher first installment.

This CL carves out the component directory and the process watcher itself.
See https://codereview.chromium.org/717223002/ for the full change.

BUG= 412384 

Review URL: https://codereview.chromium.org/729363004

Cr-Commit-Position: refs/heads/master@{#304516}

[add] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/browser_watcher.gypi
[add] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/browser_watcher/OWNERS
[add] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/browser_watcher/watcher_win.cc
[add] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/browser_watcher/watcher_win.h
[add] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/browser_watcher/watcher_win_unittest.cc
[modify] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/components.gyp
[modify] https://chromium.googlesource.com/chromium/src.git/+/c0d0a0e65cad9079d4c0878f1f3766aa59a9cf29/components/components_tests.gyp

Project Member

Comment 4 by bugdroid1@chromium.org, Nov 18 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87

commit 420541c797295004d91fb44099a36f4d66548e87
Author: siggi <siggi@chromium.org>
Date: Tue Nov 18 23:16:32 2014

Browser watcher, part deux.

Implementation of the watcher client class and its tests.
See https://codereview.chromium.org/717223002/ for full change.

BUG= 412384 

Review URL: https://codereview.chromium.org/743463002

Cr-Commit-Position: refs/heads/master@{#304693}

[modify] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher.gypi
[rename] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/exit_code_watcher_win.cc
[rename] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/exit_code_watcher_win.h
[rename] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/exit_code_watcher_win_unittest.cc
[add] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/watcher_client_win.cc
[add] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/watcher_client_win.h
[add] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/browser_watcher/watcher_client_win_unittest.cc
[modify] https://chromium.googlesource.com/chromium/src.git/+/420541c797295004d91fb44099a36f4d66548e87/components/components_tests.gyp

Project Member

Comment 5 by bugdroid1@chromium.org, Nov 19 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da

commit ce04138298b8564e8027c98aee0b28a766ec76da
Author: siggi <siggi@chromium.org>
Date: Wed Nov 19 16:04:53 2014

Browser watcher part three.

Implementation of the metrics provider and its tests.
The metrics reporter reports the data captured by the watcher,
communicating through through registry.
See https://codereview.chromium.org/717223002/ for full change.

BUG= 412384 

Review URL: https://codereview.chromium.org/742523002

Cr-Commit-Position: refs/heads/master@{#304819}

[modify] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/browser_watcher.gypi
[add] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/browser_watcher/DEPS
[add] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/browser_watcher/watcher_metrics_provider_win.cc
[add] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/browser_watcher/watcher_metrics_provider_win.h
[add] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/browser_watcher/watcher_metrics_provider_win_unittest.cc
[modify] https://chromium.googlesource.com/chromium/src.git/+/ce04138298b8564e8027c98aee0b28a766ec76da/components/components_tests.gyp

Project Member

Comment 6 by bugdroid1@chromium.org, Nov 21 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7

commit 5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7
Author: siggi <siggi@chromium.org>
Date: Fri Nov 21 21:55:45 2014

Build the chrome_watcher.dll binary, spawn the watcher process and glue in the metrics provider.
This will add a new process to each chrome process tree, the --type="watcher". This process type will load the chrome_watcher.dll, whose sole purpose in life (for now) is to wait for the browser to exit, and to record its exit code in registry.

Ultimately the OOP crash handler will move out of Omaha, and end up co-habitating this process and the chrome_watcher.dll.

BUG= 412384 

Review URL: https://codereview.chromium.org/741743002

Cr-Commit-Position: refs/heads/master@{#305293}

[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/app/DEPS
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/app/client_util.cc
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/app/client_util.h
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/browser/DEPS
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/browser/metrics/chrome_metrics_service_client.cc
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/browser/metrics/chrome_metrics_service_client.h
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome.gyp
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_browser.gypi
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_exe.gypi
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/DEPS
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/OWNERS
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/chrome_watcher.def
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/chrome_watcher.gypi
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/chrome_watcher.ver
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/chrome_watcher/chrome_watcher_main.cc
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/common/chrome_constants.cc
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/common/chrome_constants.h
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/installer/mini_installer/chrome.release
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/chrome/tools/build/win/FILES.cfg
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/components/browser_watcher.gypi
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/components/browser_watcher/exit_code_watcher_win_unittest.cc
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/components/browser_watcher/watcher_main_api_win.cc
[add] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/components/browser_watcher/watcher_main_api_win.h
[modify] http://crrev.com/5e62f7e775c0a4cd5e5686e51c29e42e748cbeb7/components/components_tests.gyp

Project Member

Comment 7 by bugdroid1@chromium.org, Nov 22 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/646bd2dfd0ba8a8048aa730f29602ae1c1228fb6

commit 646bd2dfd0ba8a8048aa730f29602ae1c1228fb6
Author: siggi <siggi@chromium.org>
Date: Sat Nov 22 00:19:35 2014

Add the Stability.BrowserExitCodes metric (added in https://chromium.googlesource.com/chromium/src/+/ce04138298b8564e8027c98aee0b28a766ec76da) and a couple of relevant exit codes.

BUG= 412384 

Review URL: https://codereview.chromium.org/748193002

Cr-Commit-Position: refs/heads/master@{#305327}

[modify] http://crrev.com/646bd2dfd0ba8a8048aa730f29602ae1c1228fb6/tools/metrics/histograms/histograms.xml

Project Member

Comment 8 by bugdroid1@chromium.org, Nov 26 2014

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f3feaa9c705b4a844ec37d52cb91c12e8c9e8dd9

commit f3feaa9c705b4a844ec37d52cb91c12e8c9e8dd9
Author: siggi <siggi@chromium.org>
Date: Wed Nov 26 20:31:20 2014

Add defined content and chrome exit codes, as well as commonly seen Windows errors to the WindowsExitCode enum

BUG= 412384 

Review URL: https://codereview.chromium.org/759313002

Cr-Commit-Position: refs/heads/master@{#305870}

[modify] http://crrev.com/f3feaa9c705b4a844ec37d52cb91c12e8c9e8dd9/tools/metrics/histograms/histograms.xml

Comment 9 by siggi@chromium.org, Dec 5 2014

Status: Started
This new instrumentation shows that on Canary, roughly 18% of clients experience an exit code of 0x40010004 on a given day. This exit code appears to occur when hung UI processes are terminated by the window manager, as it's trying to shake off it's clients. I don't see Windows Error Reporting capturing crashes nor logging anything when this happens, but WinSvc does create an entry in the application log.

Additionally, roughly 3.5% of clients experience exit code 20, which occurs when the process singleton (and the profile) can't be acquired on rendezvous.

I suspect these are due to deadlocks in processing WM_ENDSESSION and on regular quit. Will add more temporary instrumentation to narrow this down.
 Issue 421594  has been merged into this issue.
Project Member

Comment 18 by bugdroid1@chromium.org, Jan 14 2015

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e4462c0e439c489ad057060ba014d8899398c117

commit e4462c0e439c489ad057060ba014d8899398c117
Author: siggi <siggi@chromium.org>
Date: Wed Jan 14 01:49:20 2015

Use a window to catch WM_ENDSESSION.

Rejig the watcher to run a message loop on the main thread, and to
do the blocking wait on a dedicated background thread.

BUG= 412384 

Review URL: https://codereview.chromium.org/848033005

Cr-Commit-Position: refs/heads/master@{#311390}

[modify] http://crrev.com/e4462c0e439c489ad057060ba014d8899398c117/chrome/chrome_watcher/chrome_watcher_main.cc

Project Member

Comment 19 by bugdroid1@chromium.org, Jan 14 2015

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1ab69a5ac8b732e6f5a335e8c09e726ef32d80ca

commit 1ab69a5ac8b732e6f5a335e8c09e726ef32d80ca
Author: siggi <siggi@chromium.org>
Date: Wed Jan 14 20:43:38 2015

Record mode of endsession in watcher.

R=erikwright@chromium.org
BUG= 412384 

Review URL: https://codereview.chromium.org/846383003

Cr-Commit-Position: refs/heads/master@{#311539}

[modify] http://crrev.com/1ab69a5ac8b732e6f5a335e8c09e726ef32d80ca/chrome/chrome_watcher/chrome_watcher_main.cc

Project Member

Comment 21 by bugdroid1@chromium.org, Feb 11 2015

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c855c8079492d78efdabf1e93586acf4dda8726f

commit c855c8079492d78efdabf1e93586acf4dda8726f
Author: siggi <siggi@chromium.org>
Date: Wed Feb 11 21:02:40 2015

Stretch watcher's lifetime if it sees WM_ENDSESSION before process exit.

R=erikwright@chromium.org
BUG= 412384 

Review URL: https://codereview.chromium.org/915123003

Cr-Commit-Position: refs/heads/master@{#315833}

[modify] http://crrev.com/c855c8079492d78efdabf1e93586acf4dda8726f/chrome/chrome_watcher/chrome_watcher_main.cc

Project Member

Comment 22 by bugdroid1@chromium.org, Feb 24 2015

Blockedon: chromium:464733
Labels: Hotlist-Recharge
This issue likely requires triage.  The current issue owner may be inactive (i.e. hasn't fixed an issue in the last 30 days or commented in this particular issue in the last 90 days).  Thanks for helping out!

-Anthony
Project Member

Comment 25 by sheriffbot@chromium.org, Jul 10 2016

Labels: Hotlist-OpenBugWithCL
A change has landed for this issue, but it's been open for over 6 months. Please review and close it if applicable. If this issue should remain open, remove the "Hotlist-OpenBugWithCL" label. If no action is taken, it will be archived in 30 days.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Hotlist-OpenBugWithCL
Removing Hotlist-OpenBugWithCL label as this bug has blocked on issue.

Comment 27 by siggi@chromium.org, Jan 30 2017

Status: Fixed (was: Started)

Sign in to add a comment