New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 848066 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner: ----
Closed: Jun 2018
Cc:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug



Sign in to add a comment

chrome_elf_unittests ThirdPartyTest tests fail in a few cases with Windows ASan

Project Member Reported by r...@chromium.org, May 30 2018

Issue description

First failing build:
https://ci.chromium.org/buildbot/chromium.clang/CrWinAsan/665

Probable culprit CL:
https://chromium-review.googlesource.com/c/chromium/src/+/1077347

The logs say:
[ RUN      ] ThirdPartyTest.WideCharEncoding
../../chrome_elf/third_party_dlls/main_unittest.cc(303): error: Value of: module_data.image_name.empty()
  Actual: false
Expected: true
Stack trace:
Backtrace:
	StackTraceGetter::CurrentStackTrace [0x00B8D8A6+330] (C:\b\c\b\CrWinAsan\src\third_party\googletest\custom\gtest\internal\custom\stack_trace_getter.cc:22)
	testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop [0x00BB4B9D+201] (C:\b\c\b\CrWinAsan\src\third_party\googletest\src\googletest\src\gtest.cc:810)
	testing::internal::AssertHelper::operator= [0x00BB3ABB+201] (C:\b\c\b\CrWinAsan\src\third_party\googletest\src\googletest\src\gtest.cc:386)
	third_party_dlls::`anonymous namespace'::ThirdPartyTest_WideCharEncoding_Test::TestBody [0x00B73A70+2618] (C:\b\c\b\CrWinAsan\src\chrome_elf\third_party_dlls\main_unittest.cc:303)
[  FAILED  ] ThirdPartyTest.WideCharEncoding (47 ms)
 
Hi,

If all the other bots are passing, how do I replicate this locally to debug?  Can I get binaries off this bot?  Or GN args?

Is this bot using a different toolchain right now?

Thanks!

Comment 2 by thakis@chromium.org, May 30 2018

Same toolchain, just different gn args. From https://ci.chromium.org/buildbot/chromium.clang/CrWinAsan/665 -> generate_build_files -> stdout:

clang_use_chrome_plugins = false
enable_ipc_fuzzer = true
is_asan = true
is_clang = true
is_component_build = false
is_debug = false
llvm_force_head_revision = true
symbol_level = 2
target_cpu = "x86"
v8_enable_verify_heap = true


is_asan = true is probably the important bit, target_cpu and is_debug look important too.

Comment 3 by r...@chromium.org, May 30 2018

target_cpu = "x86" is also possibly important, it is currently 32-bit.

Sorry I didn't fill in more detail, I'm just triaging red bots.
Status: Started (was: Untriaged)
Thanks - I've managed to debug locally with an asan build.

So, it looks like asan is adding an export to any PE files it builds.  This specific test expects/depends on a test DLL having NO export directory in its headers.  So it asserts when there IS unexpectedly an export section there.

dumpbin /EXPORTS output:

Dump of file E:\src\work4\src\out\hooktests\main_unittest_dll_1.dll

File Type: DLL

  Section contains the following exports for main_unittest_dll_1.dll

    00000000 characteristics
           0 time date stamp
        0.00 version
           0 ordinal base
           2 number of functions
           1 number of names

    ordinal hint RVA      name

          1    0 00004CE0 __asan_wrap__except_handler4 = ___asan_wrap__except_handler4

  Summary

        1000 .00cfg
        1000 .DLLTH
        5000 .data
        1000 .gfids
        9000 .rdata
        2000 .reloc
       15000 .text


*The exception handler export being added is the problem.

Reid or Nico, do you have any ideas for how to maybe disable this test only on this ASan bot... or to prevent the asan tooling from being added to this specific test dll (a GN project setting or something)?  The test DLL needs to have no exports.


Comment 5 by r...@chromium.org, May 31 2018

I think disabling ASan instrumentation for this DLL is probably the best option. I think you just have to remove the gn configs from the target, like this:
 configs -= [ "//build/config/sanitizers:default_sanitizer_flags" ]

I can see examples of that throughout chrome.

Thanks for investigating this!
Unfortunately removing default_sanitizer_flags isn't going to cut it!  The binary still has the exception handler export.

I've spent a lot of time trying to figure out any other configs I can remove from this project, but I think it's hidden under default_shared_library_configs.

Even after removing default_sanitizer_flags, if I "gn desc out\blah TargetName --tree", I still have the following in there:


---------------------------------------------
configs tree (in order applying)
  //build/config:feature_flags
  //build/config/compiler:afdo
  //build/config/compiler:afdo_optimize_size
  //build/config/compiler:compiler
    //build/config/win:compiler
    //build/config/compiler:clang_revision
    //build/config/compiler:compiler_cpu_abi
    //build/config/compiler:compiler_codegen
  //build/config/compiler:clang_stackrealign
  //build/config/compiler:compiler_arm_fpu
  //build/config/compiler:compiler_arm_thumb
  //build/config/compiler:chromium_code
    //build/config/compiler:default_warnings
  //build/config/compiler:default_include_dirs
  //build/config/compiler:default_optimization
    //build/config/compiler:optimize
  //build/config/compiler:default_stack_frames
  //build/config/compiler:default_symbols
    //build/config/compiler:symbols
  //build/config/compiler:no_exceptions
  //build/config/compiler:no_rtti
  //build/config/compiler:runtime_library
    //build/config/win:runtime_library
  //build/config/compiler:thin_archive
  //build/config/coverage:default_coverage
  //build/config/win:default_crt
    //build/config/win:static_crt
  //build/config/win:lean_and_mean
  //build/config/win:nominmax
  //build/config/win:unicode
  //build/config/win:winver
  //build/config/win:vs_code_analysis
  //build/config/win:default_cygprofile_instrumentation
  //build/config/clang:find_bad_constructs
  //build/config/clang:extra_warnings
  //build/config:release
  //build/config:default_libs
  //build/config:shared_library_config
    //build/config/win:sdk_link
    //build/config/win:common_linker_setup
    //build/config/sanitizers:link_shared_library
  //build/config/win:default_incremental_linking
  //build/config/win:console
  //build/config/sanitizers:sanitizer_options_link_helper
  //build/config/sanitizers:default_sanitizer_ldflags

and

Dependency tree
  //build/config:exe_and_shlib_deps
    //build/config/sanitizers:deps
      //build/config/sanitizers:copy_asan_runtime
      //build/config/sanitizers:options_sources
--------------------------------------------------

If I try to remove any of the other "*/sanitizers:*" configs, it fails to be able to do so from inside this tiny project:
https://cs.chromium.org/chromium/src/chrome_elf/BUILD.gn?type=cs&q=main_unittest+file:.gn&sq=package:chromium&g=0&l=368

"You were trying to remove X from the list but it wasn't there."  I guess I can't disable all sanitization from inside this project.


Thoughts or other solutions welcome!
Confirmed:

If I locally remove //build/config/sanitizers:link_shared_library from the main shared_library_config
(https://cs.chromium.org/chromium/src/build/config/BUILD.gn?type=cs&g=0&l=367)

There is no Export added to my test DLL.  But there's no way exposed right now to remove that from a "leaf" build target.  Trying to add 
"configs -= [ "//build/config/sanitizers:link_shared_library" ]" to my test project causes a GN failure.
"You were trying to remove "//build/config/sanitizers:link_shared_library"
from the list but it wasn't there."  ...something else caused the file to be included.
Yeah, it's pulled in via shared_library_config: https://cs.chromium.org/chromium/src/build/config/BUILD.gn?type=cs&q=link_shared_library+file:%5C.gn&sq=package:chromium&g=0&l=346

And you can't remove sub-configs.

You should be able to remove //build/config:shared_library_config -- on Windows, it currently adds nothing but the sanitizer config (but if someone adds things to it on Windows later it's going to be confusing)
On Win, shared_library_config adds linker stuff:
https://cs.chromium.org/chromium/src/build/config/BUILD.gn?type=cs&g=0&l=354

Which currently includes
//build/config/win:sdk_link
//build/config/win:common_linker_setup


I tried anyways, removing shared_library_config (and then again adding in the two linker configs above), but no dice.  ASan linker is expecting things:

e:\src\work4\src>ninja -C out\hooktests main_unittest_dll_1
ninja: Entering directory `out\hooktests'
[1/1] Regenerating ninja files
[1/1] LINK(DLL) main_unittest_dll_1.dll main_unittest_dll_1.dll.lib main_unittest_dll_1.dll.pdb
FAILED: main_unittest_dll_1.dll main_unittest_dll_1.dll.lib main_unittest_dll_1.dll.pdb
C:/work/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x86 False ../../third_party/llvm-build/Release+Asserts/bin/lld-link.exe /nologo /IMPLIB:./main_unittest_dll_1.dll.lib /DLL /OUT:./main_unittest_dll_1.dll /PDB:./main_unittest_dll_1.dll.pdb @./main_unittest_dll_1.dll.rsp
e:\src\work4\src\third_party\llvm-build\Release+Asserts\bin\lld-link.exe: error: undefined symbol: ___asan_init
>>> referenced by obj/chrome_elf/main_unittest_dll_1/main_unittest_dll_1.obj:(_asan.module_ctor)


e:\src\work4\src\third_party\llvm-build\Release+Asserts\bin\lld-link.exe: error: undefined symbol: ___asan_version_mismatch_check_v8
>>> referenced by obj/chrome_elf/main_unittest_dll_1/main_unittest_dll_1.obj:(_asan.module_ctor)

ninja: build stopped: subcommand failed.


Comment 10 by r...@chromium.org, Jun 11 2018

I think we just need to remove both configs to disable the instrumentation, and then to disable the extra runtime library stuff. I came up with this:
https://chromium-review.googlesource.com/#/c/chromium/src/+/1096371

Seems to fix things. I checked x64, and this isn't a problem there for some reason? I don't get it.
Project Member

Comment 11 by bugdroid1@chromium.org, Jun 12 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/67a570db7c196dd128d3a1b023c654f37358c3a2

commit 67a570db7c196dd128d3a1b023c654f37358c3a2
Author: Reid Kleckner <rnk@google.com>
Date: Tue Jun 12 20:19:18 2018

Disable sanitizer instrumentation for chrome_elf test DLLs

Otherwise, they will have exports from bits of the ASan runtime library
that are linked statically into each ASan-instrumented DLL.

R=pennymac@chromium.org,thakis@chromium.org
BUG= 848066 

Change-Id: I93ebfe118423aa5c429465a901e709c31ccd80e4
Reviewed-on: https://chromium-review.googlesource.com/1096371
Commit-Queue: Penny MacNeil <pennymac@chromium.org>
Reviewed-by: Penny MacNeil <pennymac@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#566548}
[modify] https://crrev.com/67a570db7c196dd128d3a1b023c654f37358c3a2/chrome_elf/BUILD.gn

Comment 12 by r...@chromium.org, Jun 12 2018

Status: Fixed (was: Started)

Sign in to add a comment