unit_tests doesn't compile on Mac with enable_clang_coverage=true |
||||
Issue description
unit_tests doesn't compile on Mac with enable_clang_coverage=true, here is the error log:
ninja: Entering directory `out/coverage/'
[32191/33994] LINK ./v8_context_snapshot_generator
FAILED: v8_context_snapshot_generator
export DEVELOPER_DIR=/Users/liaoyuke/chromium/src/build/mac_files/Xcode.app; TOOL_VERSION=1505258040 ../../build/toolchain/mac/linker_driver.py ../../third_party/llvm-build/Release+Asserts/bin/clang++ -stdlib=libc++ -arch x86_64 -segprot PROTECTED_MEMORY rw r -Werror -isysroot ../../build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -mmacosx-version-min=10.9.0 -fprofile-instr-generate -Wl,-ObjC -L../../build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/lib -o "./v8_context_snapshot_generator" -Wl,-filelist,"./v8_context_snapshot_generator.rsp" -framework Cocoa -framework Foundation -framework IOKit -framework Security -framework SystemConfiguration -framework CoreFoundation -framework ApplicationServices -framework AppKit -lbsm -framework CFNetwork -framework CoreServices -lresolv -framework CoreGraphics -framework CoreText -framework Accelerate -framework AudioUnit -framework Carbon -framework CoreVideo -framework IOSurface -framework AudioToolbox -framework CoreAudio -framework OpenGL -framework Quartz -framework AVFoundation -framework CoreMedia -framework VideoToolbox -framework QuartzCore -framework DiskArbitration -lsandbox -framework IOBluetooth
final section layout:
__TEXT/__text addr=0x100002900, size=0x1025492D, fileOffset=0x00002900, type=1
__TEXT/__stubs addr=0x11025722E, size=0x00001926, fileOffset=0x1025722E, type=28
__TEXT/__stub_helper addr=0x110258B54, size=0x000029BE, fileOffset=0x10258B54, type=32
__TEXT/__cstring addr=0x11025B520, size=0x003FFBCC, fileOffset=0x1025B520, type=13
__TEXT/__const addr=0x11065B100, size=0x0024D228, fileOffset=0x1065B100, type=0
__TEXT/__ustring addr=0x1108A8328, size=0x00000D78, fileOffset=0x108A8328, type=16
__TEXT/__gcc_except_tab addr=0x1108A90A0, size=0x0001168C, fileOffset=0x108A90A0, type=0
__TEXT/__objc_methname addr=0x1108BA72C, size=0x00005A97, fileOffset=0x108BA72C, type=14
__TEXT/__objc_classname addr=0x1108C01C3, size=0x000006F5, fileOffset=0x108C01C3, type=14
__TEXT/__objc_methtype addr=0x1108C08B8, size=0x0000F655, fileOffset=0x108C08B8, type=14
__TEXT/__unwind_info addr=0x1108CFF10, size=0x000463B0, fileOffset=0x108CFF10, type=22
__TEXT/__eh_frame addr=0x1109162C0, size=0x00045CF0, fileOffset=0x109162C0, type=19
__DATA/__nl_symbol_ptr addr=0x11095C000, size=0x00000010, fileOffset=0x1095C000, type=29
__DATA/__got addr=0x11095C010, size=0x00000CF0, fileOffset=0x1095C010, type=29
__DATA/__la_symbol_ptr addr=0x11095CD00, size=0x00002188, fileOffset=0x1095CD00, type=27
__DATA/__mod_init_func addr=0x11095EE88, size=0x00000088, fileOffset=0x1095EE88, type=33
__DATA/__const addr=0x11095EF20, size=0x00264230, fileOffset=0x1095EF20, type=0
__DATA/__cfstring addr=0x110BC3150, size=0x00000F60, fileOffset=0x10BC3150, type=17
__DATA/__objc_classlist addr=0x110BC40B0, size=0x00000190, fileOffset=0x10BC40B0, type=0
__DATA/__objc_catlist addr=0x110BC4240, size=0x00000078, fileOffset=0x10BC4240, type=24
__DATA/__objc_protolist addr=0x110BC42B8, size=0x00000068, fileOffset=0x10BC42B8, type=0
__DATA/__objc_imageinfo addr=0x110BC4320, size=0x00000008, fileOffset=0x10BC4320, type=0
__DATA/__objc_const addr=0x110BC4328, size=0x00007368, fileOffset=0x10BC4328, type=0
__DATA/__objc_selrefs addr=0x110BCB690, size=0x00001980, fileOffset=0x10BCB690, type=15
__DATA/__objc_protorefs addr=0x110BCD010, size=0x00000020, fileOffset=0x10BCD010, type=0
__DATA/__objc_classrefs addr=0x110BCD030, size=0x00000348, fileOffset=0x10BCD030, type=23
__DATA/__objc_superrefs addr=0x110BCD378, size=0x00000130, fileOffset=0x10BCD378, type=0
__DATA/__objc_ivar addr=0x110BCD4A8, size=0x00000310, fileOffset=0x10BCD4A8, type=0
__DATA/__objc_data addr=0x110BCD7B8, size=0x00000FA0, fileOffset=0x10BCD7B8, type=0
__DATA/__data addr=0x110BCE760, size=0x000FBDD0, fileOffset=0x10BCE760, type=0
__DATA/__llvm_prf_cnts addr=0x110CCA530, size=0x00F7C958, fileOffset=0x10CCA530, type=0
__DATA/__llvm_prf_data addr=0x111C46E88, size=0x029A2EE0, fileOffset=0x11C46E88, type=0
__DATA/__llvm_prf_names addr=0x1145E9D70, size=0x280817B6, fileOffset=0x145E9D70, type=0
__DATA/__thread_vars addr=0x13C66B528, size=0x00000030, fileOffset=0x3C66B528, type=38
__DATA/crashpad_info addr=0x13C66B558, size=0x00000038, fileOffset=0x3C66B558, type=0
__DATA/__llvm_prf_vnds addr=0x13C66B590, size=0x00006000, fileOffset=0x3C66B590, type=0
__DATA/__thread_data addr=0x13C671590, size=0x00000010, fileOffset=0x3C671590, type=40
__DATA/__thread_bss addr=0x13C6715A0, size=0x00000004, fileOffset=0x00000000, type=39
__DATA/__bss addr=0x13C6715B0, size=0x000AF07C, fileOffset=0x00000000, type=25
__DATA/__common addr=0x13C720640, size=0x00087F3C, fileOffset=0x00000000, type=25
__DATA/__huge addr=0x13C7A857C, size=0x00000000, fileOffset=0x00000000, type=25
__LLVM_COV/__llvm_covmap addr=0x13C7A9000, size=0x76810924, fileOffset=0x3C672000, type=0
PROTECTED_MEMORY/protected_memory addr=0x1B2FBA000, size=0x00000010, fileOffset=0xB2E83000, type=0
ld: 32-bit RIP relative reference out of range (2845947809 max is +/-4GB): from __ZN2gl23SetGLGetProcAddressProcEPFPFvvEPKcE (0x10959E780) to __ZN2gl12_GLOBAL__N_118g_get_proc_addressE (0x1B2FBA008) in '__ZN2gl23SetGLGetProcAddressProcEPFPFvvEPKcE' from obj/ui/gl/libgl_wrapper.a(gl_implementation.o) for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Traceback (most recent call last):
File "../../build/toolchain/mac/linker_driver.py", line 229, in <module>
Main(sys.argv)
File "../../build/toolchain/mac/linker_driver.py", line 79, in Main
subprocess.check_call(compiler_driver_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['../../third_party/llvm-build/Release+Asserts/bin/clang++', '-stdlib=libc++', '-arch', 'x86_64', '-segprot', 'PROTECTED_MEMORY', 'rw', 'r', '-Werror', '-isysroot', '../../build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk', '-mmacosx-version-min=10.9.0', '-fprofile-instr-generate', '-Wl,-ObjC', '-L../../build/mac_files/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/lib', '-o', './v8_context_snapshot_generator', '-Wl,-filelist,./v8_context_snapshot_generator.rsp', '-framework', 'Cocoa', '-framework', 'Foundation', '-framework', 'IOKit', '-framework', 'Security', '-framework', 'SystemConfiguration', '-framework', 'CoreFoundation', '-framework', 'ApplicationServices', '-framework', 'AppKit', '-lbsm', '-framework', 'CFNetwork', '-framework', 'CoreServices', '-lresolv', '-framework', 'CoreGraphics', '-framework', 'CoreText', '-framework', 'Accelerate', '-framework', 'AudioUnit', '-framework', 'Carbon', '-framework', 'CoreVideo', '-framework', 'IOSurface', '-framework', 'AudioToolbox', '-framework', 'CoreAudio', '-framework', 'OpenGL', '-framework', 'Quartz', '-framework', 'AVFoundation', '-framework', 'CoreMedia', '-framework', 'VideoToolbox', '-framework', 'QuartzCore', '-framework', 'DiskArbitration', '-lsandbox', '-framework', 'IOBluetooth']' returned non-zero exit status 1
ninja: build stopped: subcommand failed.
,
Dec 19 2017
Interesting! I've seen some issues with fuzz targets ( issue 790747 ), but this one is different.
,
Dec 19 2017
>> ld: 32-bit RIP relative reference out of range (2845947809 max is +/-4GB): from __ZN2gl23SetGLGetProcAddressProcEPFPFvvEPKcE (0x10959E780) to __ZN2gl12_GLOBAL__N_118g_get_proc_addressE (0x1B2FBA008) in '__ZN2gl23SetGLGetProcAddressProcEPFPFvvEPKcE' from obj/ui/gl/libgl_wrapper.a(gl_implementation.o) for architecture x86_64 That's weird. First of all, 2,845,947,809 is less than 4GB, error message probably meant +/- 2GB, not 4 GB. Other than that, I wonder whether on Linux we have the same limit or not.
,
Dec 19 2017
>> __LLVM_COV/__llvm_covmap addr=0x13C7A9000, size=0x76810924, fileOffset=0x3C672000, type=0
checked section sizes on Linux:
[38] __llvm_covmap PROGBITS 0000000000000000 65a28978
00000000b8a1c28c 0000000000000000 0 0 8
0xb8a1c28c is certainly greater than 0x76810924, so there should be a higher limit on Linux :/
,
Dec 19 2017
Ha! That makes sense! Max, can you explain what does the "section" mean? Is it the amount of logical memory allocated for the final binary? And I wonder if there is a way to increase the limit.
,
Dec 19 2017
That means how large is coverage mapping data embedded into the binary. A regular binary without code coverage instrumentation doesn't have that section. An instrumented binary does have, and the larger binary is, the larger that section is. That section is the biggest part of the final binary. I guess there are some reasons for that limit, so most likely it's not easy to change. Filed a bug upstream to get some insight on why it's happening: https://bugs.llvm.org/show_bug.cgi?id=35701
,
Dec 19 2017
I guess this is a separate question, but I'm compiling on a 64-bit machine, why is a 32-bit RIP used instead of 64 bits? Max, do you have any idea?
,
Dec 19 2017
I think that's a limitation of RIP-relative addressing mode of MOV instruction in assembly. Not sure why 32 bits in particular, but that is. Another addressing mode called "large code model", it uses absolute 64-bit addresses, but Darwin doesn't support it as per comment from LLVM bug tracker: https://bugs.llvm.org/show_bug.cgi?id=35701 CC'ing Peter, when you get a couple minutes, could you please take a look at the LLVM issue linked above and give us any pointers regarding "linker magic" we might use? :)
,
Dec 19 2017
Looks like this is related to the PROTECTED_MEMORY segment that vtsyrklevich added. Here is the logic in ld64 that determines the segment order: https://github.com/michaelweiser/ld64/blob/master/src/ld/ld.cpp#L281 It appears to be based on order in which segments are seen. So perhaps one thing that you could do is to create a dummy object file that is linked first that contains a section in the PROTECTED_MEMORY segment. That will cause the PROTECTED_MEMORY segment to be seen first and therefore sorted first.
,
Dec 20 2017
Thanks Peter, that makes sense! Yuke, do you want to give it a shot? Otherwise, feel free to assign this to me.
,
Dec 21 2017
I experimented a few things, but didn't get it to work, I don't have a very good understanding on how linker works, Max, do you have any suggestions on what I should do?
,
Dec 21 2017
I'm able to build v8_context_snapshot_generator by declaring a global PROTECTED_MEMORY at the top of the source file, so the build made it through v8_context_snapshot_generator, but when building unit_tests, it got stuck on some other issues, I'll try to work them around.
,
Jan 30 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e9592c502eefc7101342bf97f0606f94e1ae612e commit e9592c502eefc7101342bf97f0606f94e1ae612e Author: Yuke Liao <liaoyuke@chromium.org> Date: Tue Jan 30 00:47:42 2018 [Coverage] Enable experimental limited code coverage This CL sets -limited-coverage-experimental=true to allow building large unit test target on Mac. Bug: 796290 Change-Id: Ief6277ae2ea3d4e1372c4fd796e97c2a1ae0bfba Reviewed-on: https://chromium-review.googlesource.com/891799 Commit-Queue: Yuke Liao <liaoyuke@chromium.org> Reviewed-by: Abhishek Arya <inferno@chromium.org> Cr-Commit-Position: refs/heads/master@{#532700} [modify] https://crrev.com/e9592c502eefc7101342bf97f0606f94e1ae612e/build/config/coverage/BUILD.gn
,
Jan 30 2018
This issue has been worked around by setting -limited-coverage-experimental=true. |
||||
►
Sign in to add a comment |
||||
Comment 1 by liaoyuke@chromium.org
, Dec 19 2017