New issue
Advanced search Search tips

Issue 640967 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 3
Type: Bug



Sign in to add a comment

msan instructions on testing/libfuzzer/reproducing.md are wrong

Project Member Reported by thakis@chromium.org, Aug 25 2016

Issue description

I got assigned a libfuzzer + msan build. I tried to repro it by running

 gn gen out/libfuzzermsan '--args=use_libfuzzer=true is_msan=true msan_track_origins=2 use_prebuilt_instrumented_libraries=true enable_nacl=false proprietary_codecs=true'


as https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reproducing.md instructs me to. That resulted in:

ERROR at //build/config/sanitizers/sanitizers.gni:175:1: Assertion failed.
assert(!is_debug || !(is_msan || is_ubsan || is_ubsan_null || is_ubsan_vptr),
^-----
Sanitizers should generally be used in release (set is_debug=false).


I tried again with is_debug=false:

thakis@thakis:~/src/chrome/src$ gn gen out/libfuzzermsan '--args=use_libfuzzer=true is_msan=true msan_track_origins=2 use_prebuilt_instrumented_libraries=true enable_nacl=false proprietary_codecs=true is_debug=false'
ERROR at //media/BUILD.gn:27:3: Assertion failed.
  assert(
  ^-----
proprietary codecs and ffmpeg_branding set to Chromium are incompatible
See //media/BUILD.gn:28:7: 
      ffmpeg_branding != "Chromium",
      ^----------------------------
This is where it was set.
See //BUILD.gn:279:7: which caused the file to be included.
      "//media:media_unittests",
      ^------------------------


It'd be nice if that page contained instructions that actually work :-)
 

Comment 1 by thakis@chromium.org, Aug 25 2016

(I removed proprietary_codecs=true which made things go, but not sure if that's what you want to recommend)

Comment 2 by aarya@google.com, Aug 25 2016

Cc: dalecur...@chromium.org
Owner: mmoroz@chromium.org
This is broken after https://chromium.googlesource.com/chromium/src/+/ded04b8261c21b7a0a968c5f3cdd404c27e4131e. Dale, should we recommend using the two flags in docs, or is there a plan for declaring chrome_with_codecs flag ?
You need to use both flags. chrome_with_codecs is just a recipe mixin.

Comment 4 by mmoroz@chromium.org, Aug 26 2016

Status: Assigned (was: Untriaged)
I'll update the documentation, thanks for catching this!

thanks@, did you ask about this issue on #chromium IRC channel or about another one?

16:44 <+thakis> does anyone here have experience with debugging fuzzer stuff? is there a way to run the fuzzer in 
                single process mode so that gdb works?
16:44 <+thakis> mmoroz, ^

Comment 5 by thakis@chromium.org, Aug 26 2016

That irc ping was about something else. I don't know where to post usability issues with libfuzzer, or really with msan/asan, so let me post it right here :-) It took me some time to start working on bugs libfuzzer found for me 'cause:

1. ASan / MSAn don't print symbolized stacks by default, I had to `export MSAN_SYMBOLIZER_PATH=$PWD/third_party/llvm-build/Release+Asserts/bin/llvm-symbolizer` first. The libfuzzer docs don't mention this.

2. gdb wouldn't break on asan/msan reports, I had to run `export MSAN_OPTIONS=abort_on_error=1` first.

It'd be nice if the "how to debug a libfuzzer bug" page mentioned these, or at least linked to pages that mention it.

Comment 6 by aarya@google.com, Aug 26 2016

Cc: kcc@chromium.org och...@chromium.org
Weird, i thought symbolize=1 was the default https://cs.chromium.org/chromium/src/build/sanitizers/sanitizer_options.cc?rcl=0&l=68. Max, can you check why it is not working for libFuzzer.

And yes, gdb point 2), we should add to docs.

Comment 7 by kcc@chromium.org, Aug 26 2016

for 1) yes, you need llvm-symbolizer in the PATH (or use MSAN_SYMBOLIZER_PATH)
This belongs to the general *san documentation, not to libFuzzer docs.
https://github.com/google/sanitizers/wiki/AddressSanitizerCallStack

Of course, ideally we would embed llvm-symbolizer into the binary, 
as we do internally, but that's quite some extra work (we still want it). 

for 2) there is https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger


>> I don't know where to post usability issues with libfuzzer
chrome-related (intenal): chrome-fuzzing@
general (public): libfuzzer@googlegroups.com

Comment 8 by thakis@chromium.org, Aug 26 2016

For 1, it'd be cool if *San could look for llvm-symbolizer next to the executable in addition to PATH, then it'd Just Work for chrome devs.

https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger doesn't work for MSan, right?

Comment 9 by kcc@chromium.org, Aug 26 2016

>>  llvm-symbolizer next to the executable
Maybe... This will also be an extra security risk -- *san is already being criticized for doing lots of things under the hood. http://seclists.org/oss-sec/2016/q1/363

For msan the best thing is probably MSAN_OPTIONS=abort_on_error=1.
(I think you are the first person to ask though; I don't think we ever used gdb with msan)
> Maybe

It could be behind an off-by-default toggle that we turn on in Chromium. This would save many hours of dev time.

How do others debug msan reports? The diagnostic in my case was something along the lines of "uninitialized memory allocated by 'tmp.i' here" where tmp.i looked a lot like LLVM IR and not like my C++ code, and I had no idea what was going on without being able to inspect locals and whatnot.

Comment 11 by kcc@chromium.org, Aug 26 2016

Anything off-by-default and special for chrome will just add confusion. 
Besides, copying a binary to another directory will break this hack.
We should instead build llvm-symbolizer into the binary (as I've said in #7).
This is on our radar (with no ETA), but feel free to file a separate crbug. 

I usually debug an msan report (with msan_track_origins=2) using printfs :)
Filed a separate bug 641694 for embedding of llvm-symbolizer.

Regarding symbolization while reproducing a crash, I'll add a link to the general *san documentation into the "libFuzzer in Chrome" doc. Thanks!




Project Member

Comment 13 by bugdroid1@chromium.org, Aug 28 2016

Status: Started (was: Assigned)
Regarding c#6, I've checked: symbolize=1 is the default. The only issue is a requirement to have llvm-symbolizer in PATH or in a separate env var.

CL for adding symbolization and debugging instructions to the documentation: https://codereview.chromium.org/2288753002/
Project Member

Comment 15 by bugdroid1@chromium.org, Aug 29 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/03e2e556db0bd18809f75059d0fc1804324d0e82

commit 03e2e556db0bd18809f75059d0fc1804324d0e82
Author: mmoroz <mmoroz@chromium.org>
Date: Mon Aug 29 15:49:03 2016

[libfuzzer] Add "Symbolization" and "Debugging" sections to reproducing.md page.

R=aizatsky@chromium.org, inferno@chromium.org, ochang@chromium.org, thakis@chromium.org
BUG= 640967 

Review-Url: https://codereview.chromium.org/2288753002
Cr-Commit-Position: refs/heads/master@{#415014}

[modify] https://crrev.com/03e2e556db0bd18809f75059d0fc1804324d0e82/testing/libfuzzer/reproducing.md

Status: Fixed (was: Started)

Sign in to add a comment