New issue
Advanced search Search tips

Issue 887272 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Oct 11
Cc:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug-Regression

Blocked on:
issue 887429
issue 880827



Sign in to add a comment

ThinLTO crashes for Android builds

Project Member Reported by acindhe@chromium.org, Sep 20

Issue description

SHERIFFS: Use go/clank-official-builds to debug!

Version: 71.0.3557.0

Last successful build: 71.0.3556.0

Regression range: 
https://chromium.googlesource.com/chromium/src/+log/71.0.3556.0..71.0.3557.0?pretty=fuller&n=10000

Affected builders: arm, arm-next & arm_63 builds failed @ Non-Channel Targets 

Builder link: 
https://uberchromegw.corp.google.com/i/official.android/builders/official-arm/builds/3670

Failing step: 
https://uberchromegw.corp.google.com/i/official.android/builders/official-arm/builds/3670/steps/Build%20Non-Channel%20Targets/logs/stdio

Error message: 
FAILED: lib_net_unittests__library.so lib_net_unittests__library.so.TOC lib.unstripped/lib_net_unittests__library.so lib.unstripped/lib_net_unittests__library.so.map.gz 
python "../../build/toolchain/gcc_solink_wrapper.py" --readelf="../../third_party/android_ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-readelf" --nm="../../third_party/android_ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-nm" --strip=../../third_party/eu-strip/bin/eu-strip --sofile="./lib.unstripped/lib_net_unittests__library.so" --tocfile="./lib_net_unittests__library.so.TOC" --map-file "./lib.unstripped/lib_net_unittests__library.so.map.gz" --output="./lib_net_unittests__library.so" -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed --gcc-toolchain=../../third_party/android_ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -fuse-ld=lld -Wl,--icf=all -Wl,--color-diagnostics -flto=thin -Wl,--thinlto-jobs=8 -Wl,--thinlto-cache-dir=thinlto-cache -Wl,--thinlto-cache-policy,cache_size=10\%:cache_size_bytes=10g:cache_size_files=100000 -Wl,--lto-O0 -march=armv7-a -Wl,--no-undefined -Wl,--exclude-libs=libgcc.a -Wl,--exclude-libs=libc++_static.a -Wl,--exclude-libs=libvpx_assembly_arm.a --target=arm-linux-androideabi -Werror -Wl,--warn-shared-textrel -Wl,-O2 -Wl,--gc-sections --sysroot=../../third_party/android_ndk/platforms/android-16/arch-arm -nostdlib -Wl,--warn-shared-textrel -Wl,--version-script=../../build/android/android_only_jni_exports.lst -Wl,-wrap,calloc -Wl,-wrap,free -Wl,-wrap,malloc -Wl,-wrap,memalign -Wl,-wrap,posix_memalign -Wl,-wrap,pvalloc -Wl,-wrap,realloc -Wl,-wrap,valloc -L../../third_party/android_ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a -o "./lib.unstripped/lib_net_unittests__library.so" -Wl,-soname="lib_net_unittests__library.so" @"./lib_net_unittests__library.so.rsp"
ld.lld: /b/swarming/w/ir/kitchen-workdir/src/third_party/llvm/lib/Analysis/ValueTracking.cpp:1583: void computeKnownBits(const llvm::Value *, llvm::KnownBits &, unsigned int, const (anonymous namespace)::Query &): Assertion `(V->getType()->isIntOrIntVectorTy(BitWidth) || V->getType()->isPtrOrPtrVectorTy()) && "Not integer or pointer type!"' failed.
Stack dump:
0.	Running pass 'Function Pass Manager' on module 'obj/v8/v8_base/parser.o'.
1.	Running pass 'CodeGen Prepare' on function '@_ZN2v88internal10ParserBaseINS0_6ParserEE20ParseUnaryExpressionEPb'
#0 0x0000000001ce7b1d memcmp (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x1ce7b1d)
#1 0x0000000001ce7d8c memcmp (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x1ce7d8c)
#2 0x00007f38b7f0c330 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10330)
#3 0x00007f38b69c1c37 gsignal /build/eglibc-oGUzwX/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0
#4 0x00007f38b69c5028 abort /build/eglibc-oGUzwX/eglibc-2.19/stdlib/abort.c:91:0
#5 0x00007f38b69babf6 __assert_fail_base /build/eglibc-oGUzwX/eglibc-2.19/assert/assert.c:92:0
#6 0x00007f38b69baca2 (/lib/x86_64-linux-gnu/libc.so.6+0x2fca2)
#7 0x0000000003eb4b4d (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3eb4b4d)
#8 0x0000000003ec7bec (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3ec7bec)
#9 0x0000000003eb4823 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3eb4823)
#10 0x0000000003ecb896 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3ecb896)
#11 0x0000000003eb4823 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3eb4823)
#12 0x0000000003eb4d31 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3eb4d31)
#13 0x0000000003d6090e (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x3d6090e)
#14 0x00000000034a5307 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x34a5307)
#15 0x000000000349c505 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x349c505)
#16 0x0000000004153491 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x4153491)
#17 0x00000000041538b8 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x41538b8)
#18 0x0000000004153f5a (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x4153f5a)
#19 0x00000000031da79e (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31da79e)
#20 0x00000000031db338 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31db338)
#21 0x00000000031d430a (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31d430a)
#22 0x00000000031d3db7 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31d3db7)
#23 0x00000000031d75c8 (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31d75c8)
#24 0x0000000001cfd44e memcmp (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x1cfd44e)
#25 0x00007f38b7f09a80 __GI___pthread_once /build/eglibc-oGUzwX/eglibc-2.19/nptl/../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:107:0
#26 0x00000000031d745d (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31d745d)
#27 0x00000000031d6fdf (/b/build/slave/official-arm/build/src/out/Release/../../third_party/llvm-build/Release+Asserts/bin/ld.lld+0x31d6fdf)
#28 0x00007f38b7017a60 ~__shared_count /build/gcc-4.8-mW1ufQ/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:545:0
#29 0x00007f38b7017a60 ~__shared_ptr /build/gcc-4.8-mW1ufQ/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:781:0
#30 0x00007f38b7017a60 ~shared_ptr /build/gcc-4.8-mW1ufQ/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/include/bits/shared_ptr.h:93:0
#31 0x00007f38b7017a60 execute_native_thread_routine /build/gcc-4.8-mW1ufQ/gcc-4.8-4.8.4/build/x86_64-linux-gnu/libstdc++-v3/src/c++11/../../../../../src/libstdc++-v3/src/c++11/thread.cc:95:0
#32 0x00007f38b7f04184 start_thread /build/eglibc-oGUzwX/eglibc-2.19/nptl/pthread_create.c:312:0
#33 0x00007f38b6a8537d clone /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113:0
clang: error: unable to execute command: Aborted
clang: error: linker command failed due to signal (use -v to see invocation)
 
Description: Show this description
Its Blocking canary, Can you PTAL Thanks!
Status: Started (was: Assigned)
also happening on trunk, for example: https://ci.chromium.org/buildbot/internal.client.clank/arm-builder-rel/20832
For the latter, the range has 71 commit, and I would blame the Clang roll there:

commit 34893e97f49e129ceb545f4edb3758e4b1bee1e5
Author:     Nico Weber <thakis@chromium.org>
AuthorDate: Wed Sep 19 20:04:32 2018 +0000
Commit:     Commit Bot <commit-bot@chromium.org>
CommitDate: Wed Sep 19 20:04:32 2018 +0000

    Roll clang 340925:342523.
    
    Ran `tools/clang/scripts/upload_revision.py 342523`.
    
    Bug:  880827 
    Change-Id: I8f747ab4a2de74a0f0b4b5a201d37e111f323077
    Reviewed-on: https://chromium-review.googlesource.com/1232590
    Reviewed-by: Reid Kleckner <rnk@chromium.org>
    Commit-Queue: Nico Weber <thakis@chromium.org>
    Cr-Commit-Position: refs/heads/master@{#592531}
Labels: sheriff-android
Cc: benmason@chromium.org
pasko@ Thanks for the revert, can you please merge this to 3557 branch.
so we can trigger new canary build.
Status: Fixed (was: Started)
Hopefully it should do it. Maring as Fixed, feel free to reopen.
I have triggered a new build.
Cc: p...@chromium.org
Status: Available (was: Fixed)
We need some bug for actually fixing and tracking the problem, so reopening this for that.

I'll reland the roll with lto on android disabled for now, given that we're far away from branch and that that config has no clang tot waterfall coverage.
Blockedon: 887429
For the record, this clang update made one tiny WebRTC unittest fail:
https://webrtc-review.googlesource.com/101104

Blockedon: 880827
yvesg: This bug here is about the clang roll breaking the clank build. For other, unrelated issues with the roll, please file a detailed bug and mention it on  issue 880827  (the bug tracking the roll).
Cc: agrieve@chromium.org pasko@chromium.org g...@chromium.org
Labels: -Restrict-View-Google -ReleaseBlock-Dev
Owner: p...@chromium.org
Status: Assigned (was: Available)
Summary: ThinLTO crashes for Android builds (was: Clank Official Build Failure - 71.0.3557.0)
thakis: yeah disabling LTO is better at this point than blocking the roll. Sorry about that - did not think about this option.

pcc: can you please investigate what's going on with LTO?

agrieve: this will probably impact the binary size today

Removing releaseblock, restrictview and updating the title, as this bug got repurposed.
Labels: -sheriff-android
Project Member

Comment 17 by bugdroid1@chromium.org, Oct 2

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/67fc4269d8974b6d8d2facd203502497ec641c66

commit 67fc4269d8974b6d8d2facd203502497ec641c66
Author: Nico Weber <thakis@chromium.org>
Date: Tue Oct 02 19:14:09 2018

Reenable thinlto on android tot bot, to see if it works again.

Bug:  887272 
Change-Id: Ia9e6ddf68f47f0dceb8616582969199a90d52912
Reviewed-on: https://chromium-review.googlesource.com/1257202
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#595940}
[modify] https://crrev.com/67fc4269d8974b6d8d2facd203502497ec641c66/build/config/compiler/compiler.gni

Project Member

Comment 18 by bugdroid1@chromium.org, Oct 3

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/92d129d85141f78e7100f160ece42e627d17c3e3

commit 92d129d85141f78e7100f160ece42e627d17c3e3
Author: Nico Weber <thakis@chromium.org>
Date: Wed Oct 03 09:25:26 2018

android: Re-enable thinlto in official builds.

ToTAndroidOfficial is green after https://chromium-review.googlesource.com/1257202
so chances are this is fixed in the recent clang roll.

Bug:  887272 ,889062
Change-Id: Iea7a64507cea2cc0410f851af68a5f9fc008b5ae
Reviewed-on: https://chromium-review.googlesource.com/c/1258287
Commit-Queue: Egor Pasko <pasko@chromium.org>
Reviewed-by: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/master@{#596159}
[modify] https://crrev.com/92d129d85141f78e7100f160ece42e627d17c3e3/build/config/compiler/compiler.gni

Status: Fixed (was: Assigned)
Looks like this is working again.

Sign in to add a comment