Per UMA metrics, it looks like the Crashpad handler is crashing on about 25% of attempted crash reports. Mostly this looks like AV and C++ exceptions that could reasonably be OOM.
See also https://bugs.chromium.org/p/crashpad/issues/detail?id=143 .
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/99cf69e880acece59c03028c86eba754fba1f82f commit 99cf69e880acece59c03028c86eba754fba1f82f Author: siggi <siggi@chromium.org> Date: Tue Jan 10 18:53:59 2017 Revert of A simple, practically zero cost fallback crash handler for Crashpad handler process. (patchset #13 id:240001 of https://codereview.chromium.org/2596463002/ ) Reason for revert: Failing on Win10 Tests x64 bot on main waterfall. Original issue's description: > A simple, practically zero cost fallback crash handler for Crashpad handler process. > > This implements a fallback crash handler launcher. The launcher will be instantiated in the Crashpad handler process, where it pre-computes a command line to be launched on crash. This will be triggered off the UEF, under some form of single-instance locking. > > BUG= 678959 > > Review-Url: https://codereview.chromium.org/2596463002 > Cr-Commit-Position: refs/heads/master@{#442596} > Committed: https://chromium.googlesource.com/chromium/src/+/d4e43bb5146c7a59123ff5f3b6e853d8e8c29130 TBR=scottmg@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 678959 Review-Url: https://codereview.chromium.org/2621113002 Cr-Commit-Position: refs/heads/master@{#442641} [modify] https://crrev.com/99cf69e880acece59c03028c86eba754fba1f82f/components/crash/content/app/BUILD.gn [delete] https://crrev.com/30fb31340be3ffd85db42b176f997527fa92538f/components/crash/content/app/fallback_crash_handler_launcher_win.cc [delete] https://crrev.com/30fb31340be3ffd85db42b176f997527fa92538f/components/crash/content/app/fallback_crash_handler_launcher_win.h [delete] https://crrev.com/30fb31340be3ffd85db42b176f997527fa92538f/components/crash/content/app/fallback_crash_handler_launcher_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f87fdaa12b09ff286d873e7a3ea459507d7adb83 commit f87fdaa12b09ff286d873e7a3ea459507d7adb83 Author: siggi <siggi@chromium.org> Date: Tue Jan 10 21:41:51 2017 A simple, practically zero cost fallback crash handler for Crashpad handler process. This implements a fallback crash handler launcher. The launcher will be instantiated in the Crashpad handler process, where it pre-computes a command line to be launched on crash. This will be triggered off the UEF, under some form of single-instance locking. BUG= 678959 Review-Url: https://codereview.chromium.org/2596463002 Cr-Original-Commit-Position: refs/heads/master@{#442596} Committed: https://chromium.googlesource.com/chromium/src/+/d4e43bb5146c7a59123ff5f3b6e853d8e8c29130 Review-Url: https://codereview.chromium.org/2596463002 Cr-Commit-Position: refs/heads/master@{#442698} [modify] https://crrev.com/f87fdaa12b09ff286d873e7a3ea459507d7adb83/components/crash/content/app/BUILD.gn [add] https://crrev.com/f87fdaa12b09ff286d873e7a3ea459507d7adb83/components/crash/content/app/fallback_crash_handler_launcher_win.cc [add] https://crrev.com/f87fdaa12b09ff286d873e7a3ea459507d7adb83/components/crash/content/app/fallback_crash_handler_launcher_win.h [add] https://crrev.com/f87fdaa12b09ff286d873e7a3ea459507d7adb83/components/crash/content/app/fallback_crash_handler_launcher_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a37e7afabb6be7d7a12667fb29d9f74556589abf commit a37e7afabb6be7d7a12667fb29d9f74556589abf Author: dschuyler <dschuyler@chromium.org> Date: Wed Jan 11 00:29:12 2017 Revert of A simple, practically zero cost fallback crash handler for Crashpad handler process. (patchset #14 id:260001 of https://codereview.chromium.org/2596463002/ ) Reason for revert: Sorry, it looks like the new patch also has a build failure. See crbug 679920. (I'm not 100% sure, this is my best guess, apologies if it turns out incorrect). Original issue's description: > A simple, practically zero cost fallback crash handler for Crashpad handler process. > > This implements a fallback crash handler launcher. The launcher will be instantiated in the Crashpad handler process, where it pre-computes a command line to be launched on crash. This will be triggered off the UEF, under some form of single-instance locking. > > BUG= 678959 > > Review-Url: https://codereview.chromium.org/2596463002 > Cr-Original-Commit-Position: refs/heads/master@{#442596} > Committed: https://chromium.googlesource.com/chromium/src/+/d4e43bb5146c7a59123ff5f3b6e853d8e8c29130 > Review-Url: https://codereview.chromium.org/2596463002 > Cr-Commit-Position: refs/heads/master@{#442698} > Committed: https://chromium.googlesource.com/chromium/src/+/f87fdaa12b09ff286d873e7a3ea459507d7adb83 TBR=scottmg@chromium.org,siggi@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 678959 Review-Url: https://codereview.chromium.org/2620213002 Cr-Commit-Position: refs/heads/master@{#442742} [modify] https://crrev.com/a37e7afabb6be7d7a12667fb29d9f74556589abf/components/crash/content/app/BUILD.gn [delete] https://crrev.com/0aafa24f1ad6b4331bdb65db12ffc1842ba69b1d/components/crash/content/app/fallback_crash_handler_launcher_win.cc [delete] https://crrev.com/0aafa24f1ad6b4331bdb65db12ffc1842ba69b1d/components/crash/content/app/fallback_crash_handler_launcher_win.h [delete] https://crrev.com/0aafa24f1ad6b4331bdb65db12ffc1842ba69b1d/components/crash/content/app/fallback_crash_handler_launcher_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/21b194219385cd209294216842e192978cc2b424 commit 21b194219385cd209294216842e192978cc2b424 Author: siggi <siggi@chromium.org> Date: Wed Jan 11 19:15:00 2017 A simple, practically zero cost fallback crash handler for Crashpad handler process. This implements a fallback crash handler launcher. The launcher will be instantiated in the Crashpad handler process, where it pre-computes a command line to be launched on crash. This will be triggered off the UEF, under some form of single-instance locking. BUG= 678959 Review-Url: https://codereview.chromium.org/2596463002 Cr-Original-Original-Commit-Position: refs/heads/master@{#442596} Committed: https://chromium.googlesource.com/chromium/src/+/d4e43bb5146c7a59123ff5f3b6e853d8e8c29130 Review-Url: https://codereview.chromium.org/2596463002 Cr-Original-Commit-Position: refs/heads/master@{#442698} Committed: https://chromium.googlesource.com/chromium/src/+/f87fdaa12b09ff286d873e7a3ea459507d7adb83 Review-Url: https://codereview.chromium.org/2596463002 Cr-Commit-Position: refs/heads/master@{#442974} [modify] https://crrev.com/21b194219385cd209294216842e192978cc2b424/components/crash/content/app/BUILD.gn [add] https://crrev.com/21b194219385cd209294216842e192978cc2b424/components/crash/content/app/fallback_crash_handler_launcher_win.cc [add] https://crrev.com/21b194219385cd209294216842e192978cc2b424/components/crash/content/app/fallback_crash_handler_launcher_win.h [add] https://crrev.com/21b194219385cd209294216842e192978cc2b424/components/crash/content/app/fallback_crash_handler_launcher_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/92651104e939dbe94378a607e7acd381ca394fd3 commit 92651104e939dbe94378a607e7acd381ca394fd3 Author: siggi <siggi@chromium.org> Date: Thu Jan 12 16:12:12 2017 Part two of fallback crash handler for Crashpad handler process. This implements the crash handler companion to the launcher in https://codereview.chromium.org/2596463002/. The handler parses the command line as created by the launcher. After this, it can write a minidump with MinidumpWriteDump, then augment it with Crashpad metadata and massage such that Crashpad can parse it. After this, the dump is dropped into the Crashpad database for eventual upload. BUG= 678959 Review-Url: https://codereview.chromium.org/2611393002 Cr-Commit-Position: refs/heads/master@{#443249} [modify] https://crrev.com/92651104e939dbe94378a607e7acd381ca394fd3/components/crash/content/app/BUILD.gn [add] https://crrev.com/92651104e939dbe94378a607e7acd381ca394fd3/components/crash/content/app/fallback_crash_handler_win.cc [add] https://crrev.com/92651104e939dbe94378a607e7acd381ca394fd3/components/crash/content/app/fallback_crash_handler_win.h [add] https://crrev.com/92651104e939dbe94378a607e7acd381ca394fd3/components/crash/content/app/fallback_crash_handler_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/65d726a3d10b3a55ea563fb7089a34f129cd95f2 commit 65d726a3d10b3a55ea563fb7089a34f129cd95f2 Author: siggi <siggi@chromium.org> Date: Thu Jan 12 18:12:18 2017 Revert of Part two of fallback crash handler for Crashpad handler process. (patchset #11 id:220001 of https://codereview.chromium.org/2611393002/ ) Reason for revert: Breaks components_unittests on Win10. Original issue's description: > Part two of fallback crash handler for Crashpad handler process. > > This implements the crash handler companion to the launcher in https://codereview.chromium.org/2596463002/. The handler parses the command line as created by the launcher. After this, it can write a minidump with MinidumpWriteDump, then augment it with Crashpad metadata and massage such that Crashpad can parse it. > After this, the dump is dropped into the Crashpad database for eventual upload. > > BUG= 678959 > > Review-Url: https://codereview.chromium.org/2611393002 > Cr-Commit-Position: refs/heads/master@{#443249} > Committed: https://chromium.googlesource.com/chromium/src/+/92651104e939dbe94378a607e7acd381ca394fd3 TBR=scottmg@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= 678959 Review-Url: https://codereview.chromium.org/2628863005 Cr-Commit-Position: refs/heads/master@{#443290} [modify] https://crrev.com/65d726a3d10b3a55ea563fb7089a34f129cd95f2/components/crash/content/app/BUILD.gn [delete] https://crrev.com/7417bea53a78a97ae815479e5d73cf0840d90c88/components/crash/content/app/fallback_crash_handler_win.cc [delete] https://crrev.com/7417bea53a78a97ae815479e5d73cf0840d90c88/components/crash/content/app/fallback_crash_handler_win.h [delete] https://crrev.com/7417bea53a78a97ae815479e5d73cf0840d90c88/components/crash/content/app/fallback_crash_handler_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb commit b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb Author: siggi <siggi@chromium.org> Date: Fri Jan 13 00:19:36 2017 Part two of fallback crash handler for Crashpad handler process. This implements the crash handler companion to the launcher in https://codereview.chromium.org/2596463002/. The handler parses the command line as created by the launcher. After this, it can write a minidump with MinidumpWriteDump, then augment it with Crashpad metadata and massage such that Crashpad can parse it. After this, the dump is dropped into the Crashpad database for eventual upload. BUG= 678959 Review-Url: https://codereview.chromium.org/2611393002 Cr-Original-Commit-Position: refs/heads/master@{#443249} Committed: https://chromium.googlesource.com/chromium/src/+/92651104e939dbe94378a607e7acd381ca394fd3 Review-Url: https://codereview.chromium.org/2611393002 Cr-Commit-Position: refs/heads/master@{#443406} [modify] https://crrev.com/b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb/components/crash/content/app/BUILD.gn [add] https://crrev.com/b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb/components/crash/content/app/fallback_crash_handler_win.cc [add] https://crrev.com/b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb/components/crash/content/app/fallback_crash_handler_win.h [add] https://crrev.com/b0cb5ce73c6171b11cee16e6f512a4e0ca6a07eb/components/crash/content/app/fallback_crash_handler_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/7f06b265d5c842b32d069e09329bc40fc0b1d1c7 commit 7f06b265d5c842b32d069e09329bc40fc0b1d1c7 Author: siggi <siggi@chromium.org> Date: Fri Jan 13 20:41:40 2017 Change the GenerateCrashDump test to use the launcher for end-to-end testing. BUG= 678959 Review-Url: https://codereview.chromium.org/2637433002 Cr-Commit-Position: refs/heads/master@{#443654} [modify] https://crrev.com/7f06b265d5c842b32d069e09329bc40fc0b1d1c7/components/crash/content/app/fallback_crash_handler_win.cc [modify] https://crrev.com/7f06b265d5c842b32d069e09329bc40fc0b1d1c7/components/crash/content/app/fallback_crash_handler_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226 commit 62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226 Author: siggi <siggi@chromium.org> Date: Tue Jan 17 00:34:22 2017 Integrate fallback crash handling in chrome TBR=grt BUG= 678959 Review-Url: https://codereview.chromium.org/2638433002 Cr-Commit-Position: refs/heads/master@{#443963} [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/chrome/app/chrome_crash_reporter_client_win.cc [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/chrome/app/chrome_exe_main_win.cc [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/chrome/install_static/install_util.cc [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/chrome/install_static/install_util.h [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/BUILD.gn [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handler_launcher_win.cc [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handler_win.cc [modify] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handler_win.h [add] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handling_win.cc [add] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handling_win.h [add] https://crrev.com/62fd0c1ec30ee121aa0720d8d7d390cc7e9ba226/components/crash/content/app/fallback_crash_handling_win_unittest.cc
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/24b021071bd3cec503565d1df1094e4d49129265 commit 24b021071bd3cec503565d1df1094e4d49129265 Author: siggi <siggi@chromium.org> Date: Tue Jan 17 18:37:00 2017 Grab more fallback crash detail on canary/dev. BUG= 678959 Review-Url: https://codereview.chromium.org/2632123007 Cr-Commit-Position: refs/heads/master@{#444097} [modify] https://crrev.com/24b021071bd3cec503565d1df1094e4d49129265/components/crash/content/app/fallback_crash_handler_win.cc
Calling this fixed, as the reports have started trickling in.
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/07f414e052e398fc109f8792e3347feeba533033 commit 07f414e052e398fc109f8792e3347feeba533033 Author: Sigurdur Asgeirsson <siggi@chromium.org> Date: Fri Aug 25 20:15:23 2017 Eliminate static initializer in fallback crash handler. Bug: 678959 Change-Id: I900d3b5551e4d6be4c560ef9df876186d4698950 Reviewed-on: https://chromium-review.googlesource.com/636295 Reviewed-by: Scott Graham <scottmg@chromium.org> Commit-Queue: Sigurður Ásgeirsson <siggi@chromium.org> Cr-Commit-Position: refs/heads/master@{#497514} [modify] https://crrev.com/07f414e052e398fc109f8792e3347feeba533033/components/crash/content/app/fallback_crash_handling_win.cc
Comment 1 by scottmg@chromium.org
, Jan 6 2017