@brucedawson / @pennymac - any idea who the right owners for this are?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b1c36b320a7338815c737f1fb2769b25ec573061

commit b1c36b320a7338815c737f1fb2769b25ec573061
Author: Dirk Pranke <dpranke@chromium.org>
Date: Wed Jul 11 02:42:00 2018

Mark content_shell_crash_test experimental on Win10 Tests x64 (dbg).

It's quite flaky.

TBR=martiniss@chromium.org
BUG=861730

Change-Id: Id1ec406b75c54451244976108fd44fc1937a1419
Reviewed-on: https://chromium-review.googlesource.com/1132600
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Cr-Commit-Position: refs/heads/master@{#574045}
[modify] https://crrev.com/b1c36b320a7338815c737f1fb2769b25ec573061/testing/buildbot/chromium.win.json
[modify] https://crrev.com/b1c36b320a7338815c737f1fb2769b25ec573061/testing/buildbot/test_suite_exceptions.pyl

mark@ - are there any known race conditions that could cause crashpad to miss a crash?

Local repro steps:

cd out\Default
ninja content_shell copy_cdb_to_output
python ..\..\testing\scripts\content_shell_crash_test.py --isolated-script-test-output=C:\src\temp\temp1\output.json --isolated-script-test-perf-output=C:\src\temp\temp1\perftest-output.json

(adjust temp directory to taste)


Looking at the output from successful and unsuccessful runs I see this for the successful runs (environment details elided):

#READY

DevTools listening on ws://127.0.0.1:49751/devtools/browser/a07f30e4-e9c8-4529-a5ad-aa8cd01fd011
[2000:1988:0710/151828.063:ERROR:render_frame_impl.cc(1009)] Intentionally crashing (with null pointer dereference) because user navigated to chrome://crash/
#CRASHED - renderer
[952:1856:0710/151828.535:WARNING:discardable_shared_memory_manager.cc(431)] Some MojoDiscardableSharedMemoryManagerImpls are still alive. They will be leaked.
#CRASHED - renderer
#READY

DevTools listening on ws://127.0.0.1:49759/devtools/browser/2e60a308-5c67-4e75-927a-befd7a90a7c2
[4004:7256:0710/151830.043:ERROR:render_frame_impl.cc(1009)] Intentionally crashing (with null pointer dereference) because user navigated to chrome://crash/
#CRASHED - renderer
[4464:6084:0710/151830.449:WARNING:discardable_shared_memory_manager.cc(431)] Some MojoDiscardableSharedMemoryManagerImpls are still alive. They will be leaked.
#CRASHED - renderer
# Running test without trap handler.
# Run content_shell and make it crash.
.\content_shell.exe --run-web-tests chrome://crash --enable-crash-reporter --crash-dumps-dir=c:\b\s\w\itltqj8z\tmpxa2wnx
# Retrieve crash dump.
# Symbolize crash dump.
.\cdb\cdb.exe -y . -c .lines;.excr;k30;q -z c:\b\s\w\itltqj8z\tmpxa2wnx\reports\cae143c2-862f-46bd-8a24-ed918d8727bd.dmp
# Running test with trap handler.
# Run content_shell and make it crash.
.\content_shell.exe --run-web-tests chrome://crash --enable-crash-reporter --crash-dumps-dir=c:\b\s\w\itltqj8z\tmpxa2wnx --enable-features=WebAssemblyTrapHandler
# Retrieve crash dump.
# Symbolize crash dump.
.\cdb\cdb.exe -y . -c .lines;.excr;k30;q -z c:\b\s\w\itltqj8z\tmpxa2wnx\reports\b1bb9cff-a368-4770-88b6-c1074171ee7b.dmp
PASS: Breakpad integration test ran successfully.



And this for the unsuccessful runs:

#READY

DevTools listening on ws://127.0.0.1:63375/devtools/browser/f1805a5a-c346-4911-acaf-0bd1a67296a0
#CRASHED - renderer
[7448:6560:0710/163136.505:WARNING:discardable_shared_memory_manager.cc(431)] Some MojoDiscardableSharedMemoryManagerImpls are still alive. They will be leaked.
#CRASHED - renderer
# Running test without trap handler.
# Run content_shell and make it crash.
.\content_shell.exe --run-web-tests chrome://crash --enable-crash-reporter --crash-dumps-dir=c:\b\s\w\itpvqlrc\tmpumsl4l
# Retrieve crash dump.
FAIL: Expected 1 crash dump, found 0.


In particular, this line is missing from the failed runs:
[4004:7256:0710/151830.043:ERROR:render_frame_impl.cc(1009)] Intentionally crashing (with null pointer dereference) because user navigated to chrome://crash/
However the "#CRASHED - renderer" text still appears which should only be printed when the process exits abnormally. Given the apparent out-of-order output it's hard to know whether this really means anything or not, especially given the "#CRASHED - renderer" text which suggests that the renderer genuinely did crash, and given that the "Intentionally crashing" line doesn't show up on my local tests.

It feels like there might be a genuine race condition in crashpad - maybe it can miss some percentage of crashes? Adding Mark for comment.

I created crrev.com/c/1135786 to print out the exit status code in case that gives any clues.