+Hans in case there are work hours left on the west coast. Else I'll look tomorrow.

Does this happen without ffmpeg_branding / proprietary_codecs?

My flags:

target_cpu="arm" run_findbugs=true target_os="android" use_goma=true symbol_level=1 is_debug=true is_clang=true

I can repro with dalecurtis's flags at Chromium #419896:

$ gn gen out/Release --args='is_component_build=true ffmpeg_branding="Chrome" proprietary_codecs=true enable_nacl=false dcheck_always_on=true disable_incremental_isolated_processes=true is_debug=false target_os="android" is_clang=true'
$ ninja -C out/Release libboringssl.cr.so
ninja: Entering directory `out/Release'
[319/319] SOLINK ./libboringssl.cr.so
FAILED: libboringssl.cr.so libboringssl.cr.so.TOC lib.unstripped/libboringssl.cr.so 
python "/usr/local/google/work/chromium/src/build/toolchain/gcc_solink_wrapper.py" --readelf="../../third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-readelf" --nm="../../third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-nm" --strip=../../third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-strip --sofile="./lib.unstripped/libboringssl.cr.so" --tocfile="./libboringssl.cr.so.TOC" --output="./libboringssl.cr.so"  -- ../../third_party/llvm-build/Release+Asserts/bin/clang++ -shared -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed -fuse-ld=gold --gcc-toolchain=../../third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -Wl,--icf=all -Wl,--build-id=sha1 -Wl,--no-undefined -Wl,--exclude-libs=libgcc.a -Wl,--exclude-libs=libc++_static.a -Wl,--exclude-libs=libvpx_assembly_arm.a --target=arm-linux-androideabi -Wl,--warn-shared-textrel -Wl,-O1 -Wl,--gc-sections -nostdlib -Wl,--warn-shared-textrel --sysroot=../../third_party/android_tools/ndk/platforms/android-16/arch-arm  -L../../third_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a -o "./lib.unstripped/libboringssl.cr.so" -Wl,-soname="libboringssl.cr.so" @"./libboringssl.cr.so.rsp"
obj/third_party/boringssl/boringssl/a_gentm.o(.debug_info+0x4a): error: undefined reference to 'asn1_generalizedtime_to_tm.min'
obj/third_party/boringssl/boringssl/a_gentm.o(.debug_info+0x5b): error: undefined reference to 'asn1_generalizedtime_to_tm.max'
obj/third_party/boringssl/boringssl/a_utctm.o(.debug_info+0x4a): error: undefined reference to 'asn1_utctime_to_tm.min'
obj/third_party/boringssl/boringssl/a_utctm.o(.debug_info+0x5b): error: undefined reference to 'asn1_utctime_to_tm.max'
obj/third_party/boringssl/boringssl/hexdump.o(.debug_info+0x34): error: undefined reference to 'hexbyte.hextable'
obj/third_party/boringssl/boringssl/kronecker.o(.debug_info+0x4a): error: undefined reference to 'BN_kronecker.tab'
obj/third_party/boringssl/boringssl/pem_lib.o(.debug_info+0x46): error: undefined reference to 'PEM_dek_info.map'
obj/third_party/boringssl/boringssl/a_strex.o(.debug_info+0x49): error: undefined reference to 'do_hex_dump.hexdig'
obj/third_party/boringssl/boringssl/x509_obj.o(.debug_info+0x4a): error: undefined reference to 'X509_NAME_oneline.hex'
obj/third_party/boringssl/boringssl/v3_utl.o(.debug_info+0x4c): error: undefined reference to 'hex_to_string.hexdig'
obj/third_party/boringssl/boringssl/ssl_lib.o(.debug_info+0x24b): error: undefined reference to 'cbb_add_hex.hextable'
obj/third_party/boringssl/boringssl/t1_lib.o(.debug_info+0x4c): error: undefined reference to 'tls1_choose_signature_algorithm.kDefaultPeerAlgorithms'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.


It seems to be related to debug info somehow (but building with is_debug=true doesn't change anything).

Just these flags seem to be sufficient:
$ gn gen out/Release --args='target_os="android" is_clang=true'

Setting symbol_level=0 or 1 makes the error go away.

Seems there's a missing piece since dfalcantara@ says he sees it with symbol_level=1.

pcc: This is a pretty wild guess, but could this be related to this change somehow?

------------------------------------------------------------------------
r281284 | pcc | 2016-09-12 18:12:59 -0700 (Mon, 12 Sep 2016) | 10 lines

DebugInfo: New metadata representation for global variables.

This patch reverses the edge from DIGlobalVariable to GlobalVariable.
This will allow us to more easily preserve debug info metadata when
manipulating global variables.

Fixes PR30362. A program for upgrading test cases is attached to that
bug.

Differential Revision: http://reviews.llvm.org/D20147
------------------------------------------------------------------------

I can't reproduce with dfalcantara's flags:

$ gn gen out/Release --args='target_cpu="arm" run_findbugs=true target_os="android" symbol_level=1 is_debug=true is_clang=true' && ninja -C out/Release libboringssl.cr.so
Done. Made 11372 targets from 995 files in 1809ms
ninja: Entering directory `out/Release'
[317/317] SOLINK ./libboringssl.cr.so

dfalcantara: what target are you building?

D'oh, I added the symbol_level=1 after seeing another thread earlier today and promptly forgot about it.  I'm building chrome_public_apk.  If I take that flag back out, things go back to failing as expected.

It does appear to be some other commit. I couldn't reproduce with a checkout of r281285 (which is r281284 + clang update).

We're having the same problem with our Android Clang bot.

https://build.chromium.org/p/tryserver.webrtc/builders/android_clang_dbg/builds/16326

ehmaldonado: Add symbol_level=1 to it, that'll work around this problem and it'll make your bot faster too.

One-file repro:

thakis@thakis:~/src/chrome/src$ cat test.cc
int f(int i) {
  static const int max[9] = { 99, 99, 12, 31, 23, 59, 59, 12, 59 };
  return max[i];
}
thakis@thakis:~/src/chrome/src$ cat test.c
#include <string.h>
#include <time.h>

int asn1_generalizedtime_to_tm(struct tm *tm, int l, char* a)
{
    static const int min[9] = { 0, 0, 1, 1, 0, 0, 0, 0, 0 };
    static const int max[9] = { 99, 99, 12, 31, 23, 59, 59, 12, 59 };
    int n, i, o;

    o = 0;
    /*
     * GENERALIZEDTIME is similar to UTCTIME except the year is represented
     * as YYYY. This stuff treats everything as a two digit field so make
     * first two fields 00 to 99
     */
    if (l < 13)
        goto err;
    for (i = 0; i < 7; i++) {
        if ((i == 6) && ((a[o] == 'Z') || (a[o] == '+') || (a[o] == '-'))) {
            i++;
            if (tm)
                tm->tm_sec = 0;
            break;
        }
        if ((a[o] < '0') || (a[o] > '9'))
            goto err;
        n = a[o] - '0';
        if (++o > l)
            goto err;

        if ((a[o] < '0') || (a[o] > '9'))
            goto err;
        n = (n * 10) + a[o] - '0';
        if (++o > l)
            goto err;

        if ((n < min[i]) || (n > max[i]))
            goto err;
    }
 err:
    return (0);
}
thakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang -DV8_DEPRECATION_WARNINGS -DENABLE_NOTIFICATIONS -DENABLE_BROWSER_CDMS -DENABLE_PRINTING=1 -DENABLE_BASIC_PRINTING=1 -DENABLE_SPELLCHECK=1 -DUSE_BROWSER_SPELLCHECKER=1 -DUSE_OPENSSL_CERTS=1 -DNO_TCMALLOC -DUSE_EXTERNAL_POPUP_MENU=1 -DENABLE_WEBRTC=1 -DDISABLE_NACL -DENABLE_SUPERVISED_USERS=1 -DSAFE_BROWSING_DB_REMOTE -DCHROMIUM_BUILD -DENABLE_MEDIA_ROUTER=1 -DENABLE_WEBVR -DFIELDTRIAL_TESTING_ENABLED -DCR_CLANG_REVISION=281889-1 -D_FILE_OFFSET_BITS=64 -DANDROID -DHAVE_SYS_UIO_H -DANDROID_NDK_VERSION=r12b -DCOMPONENT_BUILD -D__GNU_SOURCE=1 -D__compiler_offsetof=__builtin_offsetof -Dnan=__builtin_nan -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DWTF_USE_DYNAMIC_ANNOTATIONS=1 -DBORINGSSL_IMPLEMENTATION -DBORINGSSL_NO_STATIC_INITIALIZER -DOPENSSL_SMALL -D_XOPEN_SOURCE=700 -DBORINGSSL_SHARED_LIBRARY -I. -Igen -Ithird_party/boringssl/src/include -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -funwind-tables -fPIC -pipe -fcolor-diagnostics -ffunction-sections -fno-short-enums --target=arm-linux-androideabi -march=armv7-a -mfloat-abi=softfp -mthumb -mtune=generic-armv7-a -mfpu=neon -Os -fdata-sections -ffunction-sections -fno-omit-frame-pointer -gdwarf-3 -g2 --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Werror -Wall -Wno-unused-variable -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-covered-switch-default -Wno-deprecated-register -Wno-unneeded-internal-declaration -Wno-inconsistent-missing-override -Wno-shift-negative-value -Wno-undefined-var-template -Wno-nonportable-include-path -Wno-address-of-packed-member -std=c99 -c test.c -o test.o
thakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang++ -shared -Wl,--fatal-warnings -fPIC -Wl,-z,noexecstack -Wl,-z,now -Wl,-z,relro -Wl,-z,defs -Wl,--as-needed -fuse-ld=gold --gcc-toolchain=third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 -Wl,--icf=all -Wl,--build-id=sha1 -Wl,--no-undefined -Wl,--exclude-libs=libgcc.a -Wl,--exclude-libs=libc++_static.a -Wl,--exclude-libs=libvpx_assembly_arm.a --target=arm-linux-androideabi -Wl,--warn-shared-textrel -Wl,-O1 -nostdlib -Wl,--warn-shared-textrel --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm  -Lthird_party/android_tools/ndk/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a -o test.so test.o third_party/android_tools/ndk/platforms/android-16/arch-arm/usr/lib/crtbegin_so.o -lc++_shared -lc++abi -landroid_support -lunwind third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/libgcc.a -lc -ldl -lm third_party/android_tools/ndk/platforms/android-16/arch-arm/usr/lib/crtend_so.o
test.o(.debug_info+0x4a): error: undefined reference to 'asn1_generalizedtime_to_tm.min'
test.o(.debug_info+0x5b): error: undefined reference to 'asn1_generalizedtime_to_tm.max'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Shorter compile: third_party/llvm-build/Release+Asserts/bin/clang -DANDROID -D_DEBUG -fPIC -pipe -fcolor-diagnostics -ffunction-sections -fno-short-enums --target=arm-linux-androideabi -march=armv7-a -mfloat-abi=softfp -mthumb -mtune=generic-armv7-a -mfpu=neon -Os -fdata-sections -ffunction-sections -fno-omit-frame-pointer -gdwarf-3 -g2 --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -fvisibility=hidden -std=c99 -c test.c -o test.o

Needs -Wl,-z,defs on the link line to happen (at link time)

Shorter link line:

third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -fuse-ld=gold --gcc-toolchain=third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 --target=arm-linux-androideabi --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -o test.so test.o -Wl,-z,defs

Shorter:

thakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang --target=arm-linux-androideabi -Os -g2 --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -c test.c -o test.o

thakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang++ -shared -fPIC -fuse-ld=gold --gcc-toolchain=third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 --target=arm-linux-androideabi --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -o test.so test.o -Wl,-z,defs
test.o(.debug_info+0x48): error: undefined reference to 'asn1_generalizedtime_to_tm.min'
test.o(.debug_info+0x59): error: undefined reference to 'asn1_generalizedtime_to_tm.max'

Shorter:

thakis@thakis:~/src/chrome/src$ cat test.c

struct tm { int    tm_sec; };

int asn1_generalizedtime_to_tm(struct tm *tm, int l, char* a)
{
    static const int max[9] = { 99, 99, 12, 31, 23, 59, 59, 12, 59 };
    int n, i, o;

    o = 0;
    for (i = 0; i < 7; i++) {
        if (i == 6) {
            i++;
            if (tm)
                tm->tm_sec = 0;
        }
        n = a[i] - '0';

        if (n > max[i])
            break;
    }
    return (0);
}
thakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang --target=arm-linux-androideabi -Os -g2 -c test.c -o test.othakis@thakis:~/src/chrome/src$ third_party/llvm-build/Release+Asserts/bin/clang++ -shared --gcc-toolchain=third_party/android_tools/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 --target=arm-linux-androideabi --sysroot=third_party/android_tools/ndk/platforms/android-16/arch-arm -o test.so test.o -Wl,-z,defs
test.o(.debug_info+0x48): error: undefined reference to 'asn1_generalizedtime_to_tm.max'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Filed https://llvm.org/bugs/show_bug.cgi?id=30475 upstream, and reverted the roll for now.

Landed a revert for the culprit upstream in r282076.

Roll reverted, bad CL upstream reverted so it'll be good next time we roll. Fixed.

( bug 648979  tracks adding bot coverage for symbolized android builds)