Issue metadata
Sign in to add a comment
|
SmallPie: Ensure binaries using new glibc with smallpie changes are compatible with glibc without changes |
||||||||||||||||||||||
Issue descriptionhttps://bugs.chromium.org/p/chromium/issues/detail?id=823609 has a problem where the binary using new glibc is not compatible with the other glibc (loader issues). This happened when the smallpie is not even enabled in linker. Since most distributed binaries (not just Chrome OS) rely on loader being compatible, please verify that smallpie changes are not causing any incompatibility.
,
Mar 20 2018
,
Mar 20 2018
I'm not sure what the incompatibility is. The only thing the glibc patch did was add support for processing SHT_RELR sections. There are no binaries being generated with SHT_RELR sections right now, so the code in the patch is not even active. The patch did define new section type and new dynamic tags to add support for SHT_RELR sections. Is it possible that fuzzer or other tools are already using these section types or dynamic tag values without documenting it?
,
Mar 20 2018
Can you download this file and test as following: https://storage.googleapis.com/chromeos-fuzzing-artifacts/libfuzzer-asan/R67-10504.0.0-b2397990/sysroot_virtual_target-os.tar.xz LD_LIBRARY_PATH=extract_path/lib64:extract_path/usr/lib64 extract_path/usr/libexec/fuzzers/puffin_fuzzer
,
Mar 20 2018
Extracted it in /tmp in my chroot and ran the command. It seems to be running fine: $ LD_LIBRARY_PATH=lib64:usr/lib64 usr/libexec/fuzzers/puffin_fuzzer INFO: Seed: 2187967260 INFO: Loaded 1 modules (127 inline 8-bit counters): 127 [0x55f4050770a8, 0x55f405077127), INFO: Loaded 1 PC tables (127 PCs): 127 [0x55f405077128,0x55f405077918), INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: A corpus is not provided, starting from an empty corpus #2 INITED cov: 54 ft: 55 corp: 1/1b exec/s: 0 rss: 54Mb #7 NEW cov: 62 ft: 67 corp: 2/11b exec/s: 0 rss: 54Mb L: 10/10 MS: 5 ShuffleBytes-CopyPart-EraseBytes-InsertByte-CMP- DE: "\xff\xff\xff\xff\xff\xff\xff\x16"- #8 NEW cov: 62 ft: 72 corp: 3/21b exec/s: 0 rss: 54Mb L: 10/10 MS: 1 ChangeByte- ...
,
Mar 20 2018
Yes, it runs in chroot but not outside the chroot. It did used to run before the glibc changes so we need to figure out what is causing the problem.
,
Mar 20 2018
Seems to be some symbol lookup issue. With the dynamic loader outside the chroot, I get:
$ LD_DEBUG=reloc LD_LIBRARY_PATH=lib64:usr/lib64 usr/libexec/fuzzers/puffin_fuzzer
32583:
32583: relocation processing: lib64/libc.so.6
32583:
32583: relocation processing: lib64/libpthread.so.0
32583:
32583: relocation processing: lib64/libpcre.so.1
32583:
32583: relocation processing: usr/lib64/libbrotlicommon.so.1.0.1
32583:
32583: relocation processing: lib64/libz.so.1
32583:
32583: relocation processing: usr/lib64/libevent_core-2.1.so.6
32583:
32583: relocation processing: usr/lib64/libglib-2.0.so.0
32583:
32583: relocation processing: usr/lib64/libbrotlidec.so.1.0.1
32583:
32583: relocation processing: lib64/libbz2.so.1
32583:
32583: relocation processing: lib64/libdl.so.2
32583:
32583: relocation processing: lib64/librt.so.1
32583:
32583: relocation processing: lib64/libm.so.6
32583:
32583: relocation processing: usr/lib64/libc++abi.so.1
32583:
32583: relocation processing: usr/lib64/libc++.so.1
32583:
32583: relocation processing: usr/lib64/libprotobuf-lite.so.13
32583:
32583: relocation processing: usr/lib64/libbase-core-395517.so
32583:
32583: relocation processing: usr/lib64/libbspatch.so
32583:
32583: relocation processing: usr/libexec/fuzzers/puffin_fuzzer
32583:
32583: relocation processing: /lib64/ld-linux-x86-64.so.2
32583:
32583: calling init: lib64/libpthread.so.0
32583:
32583:
32583: calling preinit: usr/libexec/fuzzers/puffin_fuzzer
32583:
Segmentation fault
Using the dynamic loader from the sysroot, I get:
$ LD_DEBUG=reloc LD_LIBRARY_PATH=lib64:usr/lib64 lib64/ld-2.23.so usr/libexec/fuzzers/puffin_fuzzer
33210:
33210: relocation processing: lib64/libc.so.6
33210:
33210: relocation processing: lib64/libpthread.so.0
33210:
33210: relocation processing: lib64/libpcre.so.1
33210:
33210: relocation processing: usr/lib64/libbrotlicommon.so.1.0.1
33210:
33210: relocation processing: lib64/libz.so.1
33210:
33210: relocation processing: usr/lib64/libevent_core-2.1.so.6
33210:
33210: relocation processing: usr/lib64/libglib-2.0.so.0
33210:
33210: relocation processing: usr/lib64/libbrotlidec.so.1.0.1
33210:
33210: relocation processing: lib64/libbz2.so.1
33210:
33210: relocation processing: lib64/libdl.so.2
33210:
33210: relocation processing: lib64/librt.so.1
33210:
33210: relocation processing: lib64/libm.so.6
33210:
33210: relocation processing: usr/lib64/libc++abi.so.1
33210:
33210: relocation processing: usr/lib64/libc++.so.1
33210:
33210: relocation processing: usr/lib64/libprotobuf-lite.so.13
33210:
33210: relocation processing: usr/lib64/libbase-core-395517.so
33210:
33210: relocation processing: usr/lib64/libbspatch.so
33210:
33210: relocation processing: usr/libexec/fuzzers/puffin_fuzzer
33210:
33210: relocation processing: lib64/ld-2.23.so
33210:
33210: calling init: lib64/libpthread.so.0
33210:
33210:
33210: calling preinit: usr/libexec/fuzzers/puffin_fuzzer
33210:
33210: usr/libexec/fuzzers/puffin_fuzzer: error: symbol lookup error: undefined symbol: swift_demangle (fatal)
33210:
33210: calling init: lib64/libc.so.6
33210:
33210:
33210: calling init: lib64/libpcre.so.1
33210:
...
33210: calling init: usr/lib64/libbspatch.so
33210:
33210:
33210: initialize program: usr/libexec/fuzzers/puffin_fuzzer
33210:
33210:
33210: transferring control: usr/libexec/fuzzers/puffin_fuzzer
33210:
INFO: Seed: 3693881681
INFO: Loaded 1 modules (127 inline 8-bit counters): 127 [0x7f1545ef10a8, 0x7f1545ef1127),
INFO: Loaded 1 PC tables (127 PCs): 127 [0x7f1545ef1128,0x7f1545ef1918),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2 INITED cov: 54 ft: 55 corp: 1/1b exec/s: 0 rss: 53Mb
#3 NEW cov: 54 ft: 60 corp: 2/6b exec/s: 0 rss: 54Mb L: 5/5 MS: 1 CMP- DE: "\x00\x00\x00\x00"-
#14 REDUCE cov: 54 ft: 60 corp: 2/5b exec/s: 0 rss: 54Mb L: 4/4 MS: 1 EraseBytes-
Note the symbol lookup error for 'swift_demangle'. The system loader seems to abort at the error (but it didn't print an error message, so maybe it aborted before reaching here). The sysroot loader keeps going and starts the program.
Has the system loader changed recently? Is the reference to 'swift_demangle' new?
,
Mar 20 2018
Note: if you use an older glibc files in Chrome OS and copy over to lib64, the fuzzer binary works. The only change here is Chrome OS glibc files. e.g. Extract the file below and copy the files from lib64 to the non-working one. https://storage.googleapis.com/chromeos-image-archive/trybot-amd64-generic-fuzzer/R66-10450.0.0-b30/sysroot_virtual_target-os.tar.xz
,
Mar 21 2018
Dug a little deeper. Simply copying one file (lib64/libc.so.6) from the old sysroot to the new one makes the new one work. The two libc.so files are *almost* identical, but not quite. Doing a diff on their disassembly: $ objdump -d -M intel -Rr old-libc.so.6 > old.s $ objdump -d -M intel -Rr new-libc.so.6 > new.s $ wc -l old.s new.s 327697 old.s 327697 new.s $ diff old.s new.s | wc -l 184 Some sample diffs: 1202c1202 < 207df: 49 81 c4 70 04 00 00 add r12,0x470 --- > 207df: 49 81 c4 90 04 00 00 add r12,0x490 26330c26330 < 3821c: 41 ff 94 24 08 0f 00 call QWORD PTR [r12+0xf08] --- > 3821c: 41 ff 94 24 28 0f 00 call QWORD PTR [r12+0xf28] 26337c26337 < 3823d: f0 48 83 80 50 04 00 lock add QWORD PTR [rax+0x450],0x1 --- > 3823d: f0 48 83 80 70 04 00 lock add QWORD PTR [rax+0x470],0x1 26341c26341 < 3824e: 41 ff 94 24 10 0f 00 call QWORD PTR [r12+0xf10] --- > 3824e: 41 ff 94 24 30 0f 00 call QWORD PTR [r12+0xf30] 26386c26386 < 382e3: f0 48 83 a8 50 04 00 lock sub QWORD PTR [rax+0x450],0x1 --- > 382e3: f0 48 83 a8 70 04 00 lock sub QWORD PTR [rax+0x470],0x1 281049c281049 < 12bac9: 41 ff 95 08 0f 00 00 call QWORD PTR [r13+0xf08] --- > 12bac9: 41 ff 95 28 0f 00 00 call QWORD PTR [r13+0xf28] 281071c281071 < 12bb27: 48 3b 93 40 03 00 00 cmp rdx,QWORD PTR [rbx+0x340] --- > 12bb27: 48 3b 93 60 03 00 00 cmp rdx,QWORD PTR [rbx+0x360] 281073c281073 < 12bb30: 48 3b 93 48 03 00 00 cmp rdx,QWORD PTR [rbx+0x348] --- > 12bb30: 48 3b 93 68 03 00 00 cmp rdx,QWORD PTR [rbx+0x368] and so on.. all these diffs have a common pattern, a 0x20 difference in a constant argument to an instruction.
,
Mar 21 2018
I don't think we need to investigate this further. According to https://bugs.chromium.org/p/chromium/issues/detail?id=823609#c10 using LD_LIBRARY_PATH with a loader from outside the sysroot is unsupported. Using the ld.so from the sysroot works fine.
,
Mar 21 2018
Luis was concerned that why glibc changes needed for smallpie are causing this incompatibility. We need to figure out the root cause before the glibc changes can be sent upstream. This is also a concern for standalone toolchain users that will build against the bundled glibc in toolchain but may not be shipping it.
,
Mar 21 2018
to be clear, the failure in question occurred when mixing glibc versions ... namely, using the ldso from one version of glibc (the host) and LD_LIBRARY_PATH libs from a completely different version of glibc (the CrOS sysroot). this is 100% unsupported, not only by us, but upstream glibc as well. the glibc ldso & libs are built with complete assumption that they will only ever be used together as a single unit. i might find it concerning if the report is that you built using one toolchain (our CrOS one), and then executing that program on a system with equiv/newer version of glibc is failing (using only the other version of glibc). glibc is supposed to be backwards compatible, and usually when introducing newer ELF features, we try to have it either fail quickly & clearly (e.g. with an error like XXX reloc/dynamic section unsupported), or silently/automatically falling back and continuing to work.
,
Mar 21 2018
Thanks Mike for the explanation, I think its fine to close this bug unless Luis has some concerns.
,
Mar 22 2018
It seems that you folks fully understand this issue. So, I don't need to fully understand it myself.
,
Mar 24 2018
|
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by manojgupta@chromium.org
, Mar 20 2018