New issue
Advanced search Search tips

Issue 831877 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug
Build-Toolchain



Sign in to add a comment

ChromiumOS fuzzers should be able to use a dictionary, seed corpus and/or options file

Project Member Reported by metzman@chromium.org, Apr 11 2018

Issue description

One of the easiest ways for developers to write effective fuzzers is by providing a seed corpus and/or dictionary for the fuzzer.

A seed corpus is just sample inputs (eg: png files if fuzzing a png parser). ClusterFuzz accepts these in zip format.
A dictionary is a file that contains tokens that are important for the target (eg: https://cs.chromium.org/chromium/src/testing/libfuzzer/fuzzers/dicts/png.dict).

Could the ChromeOS build system support something like we have in Chromium where the user provides a directory containing the seed corpus and/or a file containing the dictionary (see https://cs.chromium.org/chromium/src/net/BUILD.gn?l=6058) then the build system puts fuzzer_name_seed_corpus.zip and fuzzer_name.dict in /usr/libexec/fuzzers/ for ClusterFuzz to use?



 
Labels: OS-Chrome
Owner: manojgupta@chromium.org
Adding support for this should be easy. So lets say, user puts everything together in a single file e.g. my_fuzzer_corpus.zip that contains dict files (1 file or more?) and a corpus directory.
Fuzzer builder will upload this file in same /usr/libexec/fuzzer/ directory and CF can simply unzip this file.

Does this look feasible?
I think I was unclear. This is different than what Chromium does and it would be easier for us if ChromeOS did the same thing.

In chromium, dictionaries and seed corpus are completely separate.

A dev can define a dictionary file for a particular fuzzer (my_fuzzer). This should be copied to /usr/libexec/fuzzers/my_fuzzer.dict.

Similarly, a dev can define a corpus directory for a particular fuzzer (my_fuzzer) which points to a directory containing corpus files. This should be zipped and copied to /usr/libexec/fuzzers/my_fuzzer_seed_corpus.zip

Do they need to be copied as one file for you to support this?


Ok, I was thinking of something else. Adding the ebuild helper functions to install dict/corpus files should be trivial. 
Project Member

Comment 4 by bugdroid1@chromium.org, Apr 24 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/1a7f4663d683545f9f98189d7bcc514a62fdf229

commit 1a7f4663d683545f9f98189d7bcc514a62fdf229
Author: Manoj Gupta <manojgupta@google.com>
Date: Tue Apr 24 01:09:13 2018

platform: Improve platform_fuzzer_function to install extra files.

Change current platform_fuzzer_install function to install extra
files if specified.
Previous implementation supported installing multiple fuzzers
with a single owners files but currently no ebuilds use that feature.

If an ebuild wants to install multiple fuzzers, it can
instead call platform_fuzzer_install multiple times.

BUG= chromium:831877 
TEST=puffin/midis_seq_handler fuzzers still work.

Change-Id: I42650f26a1ef6842a731ef1b433b1b57bf83a556
Reviewed-on: https://chromium-review.googlesource.com/1024875
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>

[modify] https://crrev.com/1a7f4663d683545f9f98189d7bcc514a62fdf229/eclass/platform.eclass

Comment 5 Deleted

Sorry for not mentioning this earlier, another thing they should be able to support is an options file. This file is optional and specifies commandline options that will be passed to the libFuzzer binary.

Here is documentation on what these files should look like:

https://github.com/google/oss-fuzz/blob/b5833a7826b6b389993ff28c5bd0dc9be08c5ea4/docs/new_project_guide.md#custom-libfuzzer-options-for-clusterfuzz

A simple implementation of this will install a user defined options file to the right path (ie: /usr/libexec/fuzzers/my_fuzzer.options) as we do in OSS-Fuzz.

A more complicated implementation is what we do in chromium (see https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md#fuzzer-runtime-options) where the user defines an array in a build file, which causes the options file to be built whenever the fuzzer is built.

From what I know about the ChromeOS build system, the simpler implementation seems like the better option.
Summary: ChromiumOS fuzzers should be able to use a dictionary, seed corpus and/or options file (was: ChromiumOS fuzzers should be able to use a dictionary, seed corpus and/or options file)
I'll add documentation on how to use dicts, seed corpora and options files once their implementation is more complete.
Project Member

Comment 8 by bugdroid1@chromium.org, Apr 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/ae1252d54d2773fa5de08dafd78c269e8dfaa539

commit ae1252d54d2773fa5de08dafd78c269e8dfaa539
Author: Manoj Gupta <manojgupta@google.com>
Date: Mon Apr 30 20:55:42 2018

platform: Update platform_fuzzer_install function.

Made changes as per comments in CL:1024875.

BUG= chromium:831877 
TEST=puffin/midis_seq_handler fuzzers still work.

Change-Id: Ib19c10045e91bf887c034c1ab5de1eef9af41b40
Reviewed-on: https://chromium-review.googlesource.com/1025969
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Trybot-Ready: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>

[modify] https://crrev.com/ae1252d54d2773fa5de08dafd78c269e8dfaa539/eclass/platform.eclass

Project Member

Comment 9 by bugdroid1@chromium.org, May 1 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/5603f33087107386bdb80d1cc04662a5eb823c2b

commit 5603f33087107386bdb80d1cc04662a5eb823c2b
Author: Manoj Gupta <manojgupta@google.com>
Date: Tue May 01 05:54:22 2018

authpolicy: Update fuzzer files installation.

Switch to the platform_fuzzer_install function for installing
seed corpus and dict files.

CQ-DEPEND=CL:1025969
BUG= chromium:832044 
BUG= chromium:831877 
TEST=No change in installed files with USE="asan fuzzer".

Change-Id: I3bed69c4562aad434d0c267b85369430442803f1
Reviewed-on: https://chromium-review.googlesource.com/1027085
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Lutz Justen <ljusten@chromium.org>
Reviewed-by: A Olsen <olsen@chromium.org>

[modify] https://crrev.com/5603f33087107386bdb80d1cc04662a5eb823c2b/chromeos-base/authpolicy/authpolicy-9999.ebuild

Status: Verified (was: Untriaged)
Marking as done.

Sign in to add a comment