New issue
Advanced search Search tips

Issue 876830 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Aug 24
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug

Blocking:
issue 875971



Sign in to add a comment

ubsan builder can't write logs

Project Member Reported by manojgupta@chromium.org, Aug 22

Issue description

https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Prod/b8937488448520390240

From Unit Test logs:
libbrillo-0.0.1-r1356: ERROR: Can't open file: /var/log/asan/asan.17Error: /var/cache/portage/chromeos-base/libbrillo/out/Default/libbrillo-395517_unittests: failed with exit code 1
libbrillo-0.0.1-r1356:  * ERROR: chromeos-base/libbrillo-0.0.1-r1356::chromiumos failed (test phase):
libbrillo-0.0.1-r1356:  *   (no error message)
libbrillo-0.0.1-r1356:  * 
libbrillo-0.0.1-r1356:  * Call stack:
libbrillo-0.0.1-r1356:  *     ebuild.sh, line  133:  Called src_test
libbrillo-0.0.1-r1356:  *   environment, line 3816:  Called platform_src_test
libbrillo-0.0.1-r1356:  *   environment, line 3386:  Called platform_pkg_test
libbrillo-0.0.1-r1356:  *   environment, line 3365:  Called platform_test 'run' '/build/amd64-generic/var/cache/portage/chromeos-base/libbrillo/out/Default/libbrillo-395517_unittests'
libbrillo-0.0.1-r1356:  *   environment, line 3419:  Called die
libbrillo-0.0.1-r1356:  * The specific snippet of code:
libbrillo-0.0.1-r1356:  *       "${cmd[@]}" || die


 
Cc: vapier@chromium.org
1: Added an env dump in platform_src_test ebuild function
@@ -182,6 +182,7 @@ platform_src_test() {
        # We pass SRC along so unittests can access data files in their checkout.
        # It's also the name used by the common.mk framework.
        export SRC="${S}"
+       env |& grep SAN

I see in logs that UBSAN_OPTIONS is set
UBSAN_OPTIONS= log_path=/tmp/portage/chromeos-base/libbrillo-9999/temp/asan_logs/asan


2: I added an env print in platform2_test.py.

+    for key in os.environ.keys():
+      print( "%30s %s \n" % (key,os.environ[key]))

After this, I see in logs that ASAN_OPTIONS and LSAN_OPTIONS are passed but there is no UBSAN_OPTIONS here.

Mike,  Any idea what is going on? Something is filtering the env variables being passed and letting in ASAN_OPTIONS but not UBSAN_OPTIONS but I do not know where.

I see some filtering for ASAN_OPTIONS in authpolicy code (https://cs.corp.google.com/chromeos_public/src/platform2/authpolicy/process_executor.cc?l=39) but changing that to include UBSAN_OPTIONS does not make any difference.
I built both authpolicy and libbrillo as cros_workon packages to be sure.
look at ENV_PASSTHRU_REGEX_LIST in platform2_test.py
Project Member

Comment 3 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/98eb44e578e3a267aa6fdc5bb006459d7869eb2a

commit 98eb44e578e3a267aa6fdc5bb006459d7869eb2a
Author: Manoj Gupta <manojgupta@google.com>
Date: Thu Aug 23 19:09:10 2018

common-mk: Whitelist more sanitizer env variables.

Whitelist *SAN_OPTIONS env variables used by msan/tsan/ubsan
in platform2_test.

BUG= chromium:876830 
TEST=ubsan logs are printed as expected in logs.

Change-Id: I92c0dad925cd410b2b08443b833bbe03e02398e8
Reviewed-on: https://chromium-review.googlesource.com/1185865
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>

[modify] https://crrev.com/98eb44e578e3a267aa6fdc5bb006459d7869eb2a/common-mk/platform2_test.py

Project Member

Comment 4 by bugdroid1@chromium.org, Aug 23

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/63b7926d108afe0639fa202bfbe33dd2c3a94ecb

commit 63b7926d108afe0639fa202bfbe33dd2c3a94ecb
Author: Manoj Gupta <manojgupta@google.com>
Date: Thu Aug 23 20:21:09 2018

profile.bashrc: Symbolize ubsan crashes.

ubsan does not symbolize the crashes by default.
So set symbolize=1:print_stacktrace=1 for ubsan.

BUG= chromium:876830 
TEST=ubsan logs are symbolized with stack traces.

Change-Id: Ibd8c28707c9e1d5ede968dd83b83390e3e9ada3f
Reviewed-on: https://chromium-review.googlesource.com/1186037
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/63b7926d108afe0639fa202bfbe33dd2c3a94ecb/profiles/base/profile.bashrc

Status: Verified (was: Untriaged)
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 25

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/d93e771c96a8499aff8d891a4fbcc99d7bde6a42

commit d93e771c96a8499aff8d891a4fbcc99d7bde6a42
Author: Manoj Gupta <manojgupta@google.com>
Date: Sat Aug 25 14:52:06 2018

authpolicy: Filter more sanitizer env variables.

Filter {M,T,UB}SAN_OPTIONS in addition to the current
set of sanitizer env variables.

BUG= chromium:876830 
TEST=pre-cq passes.

Change-Id: I3f665a84911484dbb64f072f0d7e35578c736f52
Reviewed-on: https://chromium-review.googlesource.com/1188624
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Lutz Justen <ljusten@chromium.org>

[modify] https://crrev.com/d93e771c96a8499aff8d891a4fbcc99d7bde6a42/authpolicy/process_executor.cc

Sign in to add a comment