New issue
Advanced search Search tips

Issue 715223 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 715228
Owner: ----
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Chrome crashes should generated smaller core dumps?

Project Member Reported by nxia@chromium.org, Apr 25 2017

Issue description

We had a CQ outage because offloading lots of big Chrome crash dump files overloaded the lab network. Is it possible to make the chrome crash core dumps smaller?

see more context about the outage at  crbug.com/713856 
 

Comment 1 by nxia@chromium.org, Apr 25 2017

Cc: steve...@chromium.org
Steven, can you assign this to the right people to comment and work?
Cc: abodenha@chromium.org ihf@chromium.org
Components: Internals>CrashReporting
Labels: Stability-Crash Hotlist-CrOS-Gardener OS-Chrome
I filed issue 714718 so that we can better track crashes. It is possible that a solution for that might allow us to reduce the size of the crash dumps we use (i.e. if we can rely on minidumps instead).

I don't really know much about how we handle chrome crashes in general. cc'ing some folks and adding some labels to help with triage.

Comment 3 by vapier@chromium.org, Apr 25 2017

Chrome should already be producing minidumps exclusively.  it creates one by hand from its crashing child and passes that to the system crash reporter.  there is no coredump in play anywhere here that i know of.

where are you seeing large coredumps from Chrome ?

Comment 4 by nxia@chromium.org, Apr 25 2017

You can follow https://bugs.chromium.org/p/chromium/issues/detail?id=713856#c1. there're 3 core dumps, so 3 * ~300 MB gives ~ 1G test logs.
Cc: hashimoto@chromium.org vapier@chromium.org hidehiko@chromium.org
Re #3, hashimoto@ gave me this link [1] that says:

> Chrome handles its own crashes by linking in Breakpad, which does not generate core files.  In order to have crash_reporter not ignore Chrome crashes, though, you can touch the /mnt/stateful_partition/etc/collect_chrome_crashes file. This file is normally used by the autotests in order for Chrome crashes to still be handled.

per hidehiko@, this file is indeed created in the auto tests [2].

That said, I wonder if core files are needed. As mentioned by vapier@ at #3, Chrome by default creates minidump files, that are much smaller, and will be uploaded to the crash server at some point [3]. Ideas:

1) Instead of core files, include minidump files in the test results
2) Run crash_sender to upload minidump files to the crash server, then include only the crash IDs in the test results

I'm guessing 2) would be nicer if it works. That way, the complicated symbolization business is handled by the crash server.

[1] http://www.chromium.org/chromium-os/packages/crash-reporting/faq#TOC-Why-would-Chrome-crashes-not-generate-a-core-file-on-dev-builds-
[2] https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/client/common_lib/cros/chrome.py?type=cs&q=CHROME_CORE_MAGIC_FILE+package:%5Echromeos_public$&l=153
[3] https://cs.corp.google.com/chromeos_public/src/platform2/crash-reporter/init/crash-sender.conf?q=crash_sender+package:%5Echromeos_public$&l=11&dr=C
There has been discussion of (2) on chrome-os-gardeners / chromeos-infra-discuss:

https://groups.google.com/a/google.com/forum/?utm_medium=email&utm_source=footer#!msg/chrome-os-gardeners/y1JV9fnJSSI/ju0UFAF1AwAJ

There is some concern about the additional load on the crash server, but also some openness to slowly, carefully, trying it.

I am a huge proponent of this approach because it solves multiple problems:
a) Using mini dumps will help load issues across the board.
b) Crashes will be detectable even when tests succeed.
c) We get all of the benefits of the crash reporter tools for lab crashes. 

> That said, I wonder if core files are needed. [ ... ]
[ ... ]
> 1) Instead of core files, include minidump files in the test results

We're making code changes that will cause us not to gather the
core files, but still gather the minidumps, see  bug 715228  and
the associated fix in http://crosreview.com/486968.

> 2) Run crash_sender to upload minidump files to the crash
>    server, then include only the crash IDs in the test results

I've just filed  bug 716202  relating to this idea.

Status: WontFix (was: Untriaged)
Closing this in favor of bugs  715228  and  716202 .

Mergedinto: 715228
Status: Duplicate (was: WontFix)
Duplicate instead of WontFix to CC relevant folks.

Sign in to add a comment