New issue
Advanced search Search tips

Issue 776875 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Nov 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Feature

Blocking:
issue 648042
issue 762342



Sign in to add a comment

Upload unit test binaries as part of build artifacts

Project Member Reported by jorgelo@chromium.org, Oct 20 2017

Issue description

The Chrome Security Bugs-- team is interested in starting fuzzing efforts for Chrome OS code. One easy way to start this is by fuzzing unit test binaries.

Currently, unit test binaries are generated on the bots but not uploaded as artifacts (since they're not currently used beyond the bot). Can we start uploading them as part of the build artifacts?

Assigning to Aviv for triaging/sanity checking.
 
Cc: vapier@chromium.org
Sounds feasible. Some questions:
- Which build configs? Just -release builders, or -chromium-pfq and -paladin as well?
- Which unit test binaries? A whitelist of certain ebuilds, or everything?

One approach might be to add a "archive unittests" eclass that implements this behavior, and have certain ebuilds inherit it. +vapier ?

A simpler approach would be to just write a script that hunts through the buildroot and vacuums up the binaries of interest.

Comment 2 by vapier@chromium.org, Oct 20 2017

we're already generating the binpkgs on every build that has a UnitTest phase.  from the logs:
11:58:31: INFO: RunCommand: sudo 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmptZdxL4' 'FEATURES=test' 'PKGDIR=/build/daisy/test-packages' -- /mnt/host/source/chromite/bin/parallel_emerge '--sysroot=/build/daisy' '--jobs=10' ...all the packages...

so we can update one of the Archive steps to upload /build/$BOARD/test-packages/.

Comment 3 by mmoroz@chromium.org, Oct 20 2017

Cc: cmt...@chromium.org metzman@chromium.org mbarbe...@chromium.org
Adding more bugs-- folks + Caroline.

Do you build the tests with ASan?
It didn't sound like a problem when we last chatted, but just want to emphasize that we intend to use running ASanified unit tests as a stepping stone on our way to running actual fuzzers. If this ends up creating a lot of extra work for you by being very specific to unit tests, we could consider skipping right to fuzzers.

If the process makes sense to you there's no need to change anything, just wanted to bring this up now since I'm not familiar with how much work this would be on the infra side. Hopefully once we have something to upload the unit tests in place and libfuzzer working properly, a similar implementation for actual fuzz targets shouldn't be too hard to develop.

Comment 5 by vapier@chromium.org, Oct 20 2017

only ASAN bots build packages with ASAN enabled.  uploading of test-packages would be the same regardless of the config.

Comment 6 Deleted

Comment 7 Deleted

Cc: akes...@chromium.org
Owner: jorgelo@chromium.org
Returning to jorgelo@. This all sounds feasible. #2 has spelled out how we can proceed.
Status: Assigned (was: Untriaged)
Who owns the artifact upload code? I.e. who should either change the Archive step or would be a good person to review such a CL?
vapier or somebody on cbuildbot side of infra (nxia, dgarrett, myself) can review.
Blocking: 648042
Cc: jorgelo@chromium.org
Owner: manojgupta@chromium.org
I can look into uploading the UnitTest packages in buildbot scripts. 
Thanks Manoj!
Project Member

Comment 15 by bugdroid1@chromium.org, Nov 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/097cb280beefc3c3255fdc97046abe19a3c634a3

commit 097cb280beefc3c3255fdc97046abe19a3c634a3
Author: Manoj Gupta <manojgupta@google.com>
Date: Tue Nov 07 01:26:20 2017

Package UnitTest Binaries in a tarball.

To enable fuzzing, package the unit tests artifacts in a tar ball.

BUG= chromium:776875 
TEST=buildbot + pre-cq testing -  Artifacts are uploaded. Chromite Unittests pass.

Change-Id: Ic135feed7a0d7ab20956c8bdca7a0cabf7c26e33
Reviewed-on: https://chromium-review.googlesource.com/740871
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/097cb280beefc3c3255fdc97046abe19a3c634a3/cbuildbot/stages/test_stages_unittest.py
[modify] https://crrev.com/097cb280beefc3c3255fdc97046abe19a3c634a3/lib/constants.py
[modify] https://crrev.com/097cb280beefc3c3255fdc97046abe19a3c634a3/lib/chroot_util.py
[modify] https://crrev.com/097cb280beefc3c3255fdc97046abe19a3c634a3/cbuildbot/commands.py
[modify] https://crrev.com/097cb280beefc3c3255fdc97046abe19a3c634a3/cbuildbot/stages/test_stages.py

UnitTest binaries are now uploaded.
https://pantheon.corp.google.com/storage/browser/chromeos-image-archive/amd64-generic-asan/R64-10105.0.0-b21855 -> unit_tests.tar

$ gsutil ls -l  gs://chromeos-image-archive/amd64-generic-asan/R64-10105.0.0-b21855/unit_tests.tar
6991759360  2017-11-07T03:25:59Z  gs://chromeos-image-archive/amd64-generic-asan/R64-10105.0.0-b21855/unit_tests.tar


Status: Verified (was: Assigned)
Blocking: 762342
Cool! I've download unit_tests.tar, and it seems to have a bunch of archive in bunch of folders. What should I do to run any tests?
This is what I am thinking:

1. Create a working sysroot for the binaries to put in.

amd64-generic-full builder ( https://uberchromegw.corp.google.com/i/chromiumos/builders/amd64-generic-full/ ) uploads a sysroot_chromeos-base_chromeos-chrome.tar.xz for chrome builds. Maybe that will work?

$ gsutil ls gs://chromeos-image-archive/amd64-generic-full/R64-10107.0.0-b20197|grep -e sysroot
gs://chromeos-image-archive/amd64-generic-full/R64-10107.0.0-b20197/sysroot_chromeos-base_chromeos-chrome.tar.xz

$ gsutil cp gs://chromeos-image-archive/amd64-generic-full/R64-10107.0.0-b20197/sysroot_chromeos-base_chromeos-chrome.tar.xz .
$ mkdir -p sysroot_dir
$ tar -Jxf sysroot_chromeos-base_chromeos-chrome.tar.xz -C sysroot_dir

2. Extract the unit test packages to the sysroot directory (since some packages may depend on others):

find . -iname "*.tbz2" -exec tar -C sysroot_dir -jxvf {} \;

3. sudo chroot sysroot_dir /path/to/uni_test ? (Maybe setting LD_LIBRARY_PATH is enough instead of running through chroot?)
Thanks for the instructions. I filed another bug for the pending actions items:  issue 785518 .

Sign in to add a comment