New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 736432 link

Starred by 3 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug


Show other hotlists

Hotlists containing this issue:
chrome-client-infra-backlog


Sign in to add a comment

Symbolize crash stacks on the bots for sanitizer builds

Project Member Reported by dcheng@chromium.org, Jun 23 2017

Issue description

https://codereview.chromium.org/2857643002/ disabled symbolization on sanitizer builds. However, that means stacks on the sanitizer bots aren't symbolized now.  Issue 736193  is one instance where it could have been nice to have symbols. I think we could get this by just piping the output through asan_symbolize.py on the bots?
 

Comment 1 by dcheng@chromium.org, Jun 23 2017

Cc: thestig@chromium.org

Comment 2 by thakis@chromium.org, Jun 23 2017

Are you sure that analysis is correct? https://codereview.chromium.org/2857643002/ disabled in-process symbolication for asan builds because in that setup we run an external symbolizer that's better at symbolizing. The bots should already pipe through asan_symbolize. It's possible that that part broke, but that'd be independent of https://codereview.chromium.org/2857643002/

Comment 3 by nasko@chromium.org, Jun 23 2017

I was also recently surprised by the removal of the symbolization, as I run all of my development with ASan builds. This means I need to now have an explicit step of copy/run symbolizer/paste to decode or pipe through the symbolizer. However, I already pipe output through some other tools that help out with output, which makes the command line even longer. I might be one of the rare cases, but wanted to voice that this change made my dev workflow much harder :(.

Comment 4 by thakis@chromium.org, Jun 23 2017

Cc: thakis@chromium.org
Owner: ----
Status: Untriaged (was: Assigned)
nasko: That sounds unrelated to what this bug is about. Feel free to say that on ttps://codereview.chromium.org/2857643002/ . The motivation behind that change was to a) _improve_ local dev by getting better stacks b) let devs do the same thing that bots already do (...which used to be "pipe stuff through asan_symbolize", which is why I'm fairly certain that the lost symbols on the asan bots aren't due to my change)

Comment 5 by dcheng@chromium.org, Jun 23 2017

thakis@: I see, I didn't realize the bots should already do this. Hopefully infra team can triage this into the right component and we can figure out what's going on here.

(I've noticed this issue for crashes in layout test runs as well... if I see an instance, I'll post it on here)
Components: -Infra>CQ Infra>Client>Chrome
on behalf of infra, we have no component for this bug, other than general Infra>Client>Chrome, for which we have no owners.

Somebody who understands what's this about has to either
 * run recipe locally and try to repro, then fix problem
 * ssh to bot, and try to repro it there.

Comment 8 by dcheng@chromium.org, Jun 26 2017

Where are the docs for running the recipe locally?

Comment 9 by thakis@chromium.org, Jun 27 2017

You don't need to run a recipe locally. Just do

out/gn/base_unittests | tools/valgrind/asan/asan_symbolize.py
(You might also need to export ASAN_SYMBOLIZER_PATH=/path/to/chromium/src/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer)
Fwiw here's a symbolized stack on the same bot: https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.linux%2Flinux_chromium_asan_rel_ng%2F402239%2F%2B%2Frecipes%2Fsteps%2Fmedia_unittests__with_patch_%2F0%2Flogs%2FCdmAdapter__x2f_AesDecryptorTest.LatestKeyUsed__x2f_0%2F0

This shows that symbolizing on the bots basically works, and there's just some config issue somewhere independent of my cl.
https://codereview.chromium.org/2961603003 looks pretty related. Maybe we need that for more binaries.
Cc: jbudorick@chromium.org
Status: Available (was: Untriaged)
I'm not sure if this issue belongs to I>C>C, but it sounds important enough that someone should at least keep track of it. I'm adding it to chrome-client-infra-backlog hotlist for tracking and possibly later re-triage. +jbudorick@ FYI.

Project Member

Comment 15 by sheriffbot@chromium.org, Jan 10

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Cc: hzl@chromium.org
 Issue 781100  has been merged into this issue.
Status: Available (was: Untriaged)

Sign in to add a comment