Linux ChromeOS Buildspec Tests failed compile, "No matching compiler found in server" |
||||||
Issue descriptionNaCl failed to build. It looks like goma couldn't find an appropriate compiler. Infra issue? Seems similar to issue 599341. https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/Linux%20ChromeOS%20Buildspec%20Tests/builds/736 [1989/25857] ACTION modp_b64_nacl: build newlib x86-64 nlib_5a04e7e48c02b4804b2affa52d8f1c29 FAILED: gen/tc_newlib/lib64/libmodp_b64_nacl.a cd ../../third_party/modp_b64; python ../../native_client/build/build_nexe.py --root ../.. --product-dir ../../out/Release/xyz --config-name Release -t ../../native_client/toolchain/ --arch x86-64 --build newlib_nlib_clang --name ../../out/Release/gen/tc_newlib/lib64/libmodp_b64_nacl.a --objdir ../../out/Release/obj/third_party/modp_b64/modp_b64_nacl.gen/newlib-x86-64/modp_b64_nacl "--include-dirs=../../out/Release/gen/tc_newlib/include ../.. \"../../out/Release/gen\"" "--compile_flags=-m64 -O2 -g -Wall -fdiagnostics-show-option -Werror " --gomadir /mnt/data/b/build/slave/cache/cipd/goma "--defines=\"__STDC_LIMIT_MACROS=1\" \"__STDC_FORMAT_MACROS=1\" \"_GNU_SOURCE=1\" \"_POSIX_C_SOURCE=199506\" \"_XOPEN_SOURCE=600\" \"DYNAMIC_ANNOTATIONS_ENABLED=1\" \"DYNAMIC_ANNOTATIONS_PREFIX=NACL_\" \"NACL_BUILD_ARCH=x86\" V8_DEPRECATION_WARNINGS \"CLD_VERSION=2\" \"_FILE_OFFSET_BITS=64\" CHROMIUM_BUILD \"CR_CLANG_REVISION=264915-1\" UI_COMPOSITOR_IMAGE_TRANSPORT \"USE_AURA=1\" \"USE_ASH=1\" \"USE_PANGO=1\" \"USE_CAIRO=1\" \"USE_DEFAULT_RENDER_THEME=1\" \"USE_LIBJPEG_TURBO=1\" \"USE_X11=1\" \"IMAGE_LOADER_EXTENSION=1\" \"ENABLE_WEBRTC=1\" \"ENABLE_MEDIA_ROUTER=1\" USE_PROPRIETARY_CODECS ENABLE_PEPPER_CDMS ENABLE_NOTIFICATIONS \"ENABLE_TOPCHROME_MD=1\" \"ENABLE_WAYLAND_SERVER=1\" USE_UDEV FIELDTRIAL_TESTING_ENABLED \"ENABLE_TASK_MANAGER=1\" \"ENABLE_EXTENSIONS=1\" \"ENABLE_PDF=1\" \"ENABLE_PLUGINS=1\" \"ENABLE_SESSION_SERVICE=1\" \"ENABLE_THEMES=1\" \"ENABLE_AUTOFILL_DIALOG=1\" \"ENABLE_PRINTING=1\" \"ENABLE_PRINT_PREVIEW=1\" \"ENABLE_SPELLCHECK=1\" \"ENABLE_CAPTIVE_PORTAL_DETECTION=1\" \"ENABLE_APP_LIST=1\" \"ENABLE_SUPERVISED_USERS=1\" \"ENABLE_MDNS=1\" \"ENABLE_SERVICE_DISCOVERY=1\" V8_USE_EXTERNAL_STARTUP_DATA FULL_SAFE_BROWSING SAFE_BROWSING_CSD SAFE_BROWSING_DB_LOCAL \"USE_LIBPCI=1\" \"USE_OPENSSL=1\" \"USE_OPENSSL_CERTS=1\"" "--link_flags=-B../../out/Release/gen/tc_newlib/lib64 " "--source-list=../../out/gypfiles/third_party/modp_b64/newlib-x86-64.modp_b64_nacl.source_list.gypcmd" GOMA:clang++ compiling modp_b64.cc:*ERROR*: compiler_proxy:1236ms: No matching compiler found in server GOMA:clang++ compiling modp_b64.cc:*ERROR*: compiler_proxy:1236ms: ../../native_client/toolchain/linux_x86/pnacl_newlib/bin/x86_64-nacl-clang++ is disabled. FAILED with 1: /mnt/data/b/build/slave/cache/cipd/goma/gomacc ../../native_client/toolchain/linux_x86/pnacl_newlib/bin/x86_64-nacl-clang++ -c modp_b64.cc -o ../../out/Release/obj/third_party/modp_b64/modp_b64_nacl.gen/newlib-x86-64/modp_b64_nacl/modp_b64_70e59b52.o -MD -MF ../../out/Release/obj/third_party/modp_b64/modp_b64_nacl.gen/newlib-x86-64/modp_b64_nacl/modp_b64_70e59b52.d -m64 -O2 -g -Wall -fdiagnostics-show-option -Werror -D__STDC_LIMIT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D_GNU_SOURCE=1 -D_POSIX_C_SOURCE=199506 -D_XOPEN_SOURCE=600 -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DDYNAMIC_ANNOTATIONS_PREFIX=NACL_ -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D_FILE_OFFSET_BITS=64 -DCHROMIUM_BUILD -DCR_CLANG_REVISION=264915-1 -DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_ASH=1 -DUSE_PANGO=1 -DUSE_CAIRO=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJPEG_TURBO=1 -DUSE_X11=1 -DIMAGE_LOADER_EXTENSION=1 -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DUSE_PROPRIETARY_CODECS -DENABLE_PEPPER_CDMS -DENABLE_NOTIFICATIONS -DENABLE_TOPCHROME_MD=1 -DENABLE_WAYLAND_SERVER=1 -DUSE_UDEV -DFIELDTRIAL_TESTING_ENABLED -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PDF=1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_PRINTING=1 -DENABLE_PRINT_PREVIEW=1 -DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_MDNS=1 -DENABLE_SERVICE_DISCOVERY=1 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DUSE_OPENSSL_CERTS=1 -DNACL_WINDOWS=0 -DNACL_OSX=0 -DNACL_LINUX=0 -DNACL_ANDROID=0 -DNACL_BUILD_ARCH=x86 -DNACL_BUILD_SUBARCH=64 -I../../out/Release/gen/tc_newlib/include -I../.. -I../../out/Release/gen -DNDEBUG -std=gnu++0x -Wno-deprecated-register Compile options: ['-m64', '-O2', '-g', '-Wall', '-fdiagnostics-show-option', '-Werror', '-D__STDC_LIMIT_MACROS=1', '-D__STDC_FORMAT_MACROS=1', '-D_GNU_SOURCE=1', '-D_POSIX_C_SOURCE=199506', '-D_XOPEN_SOURCE=600', '-DDYNAMIC_ANNOTATIONS_ENABLED=1', '-DDYNAMIC_ANNOTATIONS_PREFIX=NACL_', '-DV8_DEPRECATION_WARNINGS', '-DCLD_VERSION=2', '-D_FILE_OFFSET_BITS=64', '-DCHROMIUM_BUILD', '-DCR_CLANG_REVISION=264915-1', '-DUI_COMPOSITOR_IMAGE_TRANSPORT', '-DUSE_AURA=1', '-DUSE_ASH=1', '-DUSE_PANGO=1', '-DUSE_CAIRO=1', '-DUSE_DEFAULT_RENDER_THEME=1', '-DUSE_LIBJPEG_TURBO=1', '-DUSE_X11=1', '-DIMAGE_LOADER_EXTENSION=1', '-DENABLE_WEBRTC=1', '-DENABLE_MEDIA_ROUTER=1', '-DUSE_PROPRIETARY_CODECS', '-DENABLE_PEPPER_CDMS', '-DENABLE_NOTIFICATIONS', '-DENABLE_TOPCHROME_MD=1', '-DENABLE_WAYLAND_SERVER=1', '-DUSE_UDEV', '-DFIELDTRIAL_TESTING_ENABLED', '-DENABLE_TASK_MANAGER=1', '-DENABLE_EXTENSIONS=1', '-DENABLE_PDF=1', '-DENABLE_PLUGINS=1', '-DENABLE_SESSION_SERVICE=1', '-DENABLE_THEMES=1', '-DENABLE_AUTOFILL_DIALOG=1', '-DENABLE_PRINTING=1', '-DENABLE_PRINT_PREVIEW=1', '-DENABLE_SPELLCHECK=1', '-DENABLE_CAPTIVE_PORTAL_DETECTION=1', '-DENABLE_APP_LIST=1', '-DENABLE_SUPERVISED_USERS=1', '-DENABLE_MDNS=1', '-DENABLE_SERVICE_DISCOVERY=1', '-DV8_USE_EXTERNAL_STARTUP_DATA', '-DFULL_SAFE_BROWSING', '-DSAFE_BROWSING_CSD', '-DSAFE_BROWSING_DB_LOCAL', '-DUSE_LIBPCI=1', '-DUSE_OPENSSL=1', '-DUSE_OPENSSL_CERTS=1', '-DNACL_WINDOWS=0', '-DNACL_OSX=0', '-DNACL_LINUX=0', '-DNACL_ANDROID=0', '-DNACL_BUILD_ARCH=x86', '-DNACL_BUILD_SUBARCH=64', '-I../../out/Release/gen/tc_newlib/include', '-I../..', '-I../../out/Release/gen', '-DNDEBUG'] Linker options: ['-B../../out/Release/gen/tc_newlib/lib64'] Traceback (most recent call last): File "../../native_client/build/build_nexe.py", line 845, in CompileProcess output_queue.put((filename, build.Compile(filename))) File "../../native_client/build/build_nexe.py", line 575, in Compile raise Error('FAILED with %d: %s' % (err, ' '.join(cmd_line))) Error: FAILED with 1: /mnt/data/b/build/slave/cache/cipd/goma/gomacc ../../native_client/toolchain/linux_x86/pnacl_newlib/bin/x86_64-nacl-clang++ -c modp_b64.cc -o ../../out/Release/obj/third_party/modp_b64/modp_b64_nacl.gen/newlib-x86-64/modp_b64_nacl/modp_b64_70e59b52.o -MD -MF ../../out/Release/obj/third_party/modp_b64/modp_b64_nacl.gen/newlib-x86-64/modp_b64_nacl/modp_b64_70e59b52.d -m64 -O2 -g -Wall -fdiagnostics-show-option -Werror -D__STDC_LIMIT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D_GNU_SOURCE=1 -D_POSIX_C_SOURCE=199506 -D_XOPEN_SOURCE=600 -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DDYNAMIC_ANNOTATIONS_PREFIX=NACL_ -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D_FILE_OFFSET_BITS=64 -DCHROMIUM_BUILD -DCR_CLANG_REVISION=264915-1 -DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_ASH=1 -DUSE_PANGO=1 -DUSE_CAIRO=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJPEG_TURBO=1 -DUSE_X11=1 -DIMAGE_LOADER_EXTENSION=1 -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DUSE_PROPRIETARY_CODECS -DENABLE_PEPPER_CDMS -DENABLE_NOTIFICATIONS -DENABLE_TOPCHROME_MD=1 -DENABLE_WAYLAND_SERVER=1 -DUSE_UDEV -DFIELDTRIAL_TESTING_ENABLED -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PDF=1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_PRINTING=1 -DENABLE_PRINT_PREVIEW=1 -DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_MDNS=1 -DENABLE_SERVICE_DISCOVERY=1 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DUSE_OPENSSL_CERTS=1 -DNACL_WINDOWS=0 -DNACL_OSX=0 -DNACL_LINUX=0 -DNACL_ANDROID=0 -DNACL_BUILD_ARCH=x86 -DNACL_BUILD_SUBARCH=64 -I../../out/Release/gen/tc_newlib/include -I../.. -I../../out/Release/gen -DNDEBUG -std=gnu++0x -Wno-deprecated-register
,
Jul 6 2016
Problem persists on a forced rebuild, so it is probably not flake. https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/Linux%20ChromeOS%20Buildspec%20Tests/builds/737 ukai, is this something your team could look at? Or does it sound like a build problem on our side? (Our other builders are green.)
,
Jul 6 2016
+llozano@ I confirmed that the builder is still using GYP (as it should be since it is a 51 branch build).
,
Jul 6 2016
Do you know which version of pnacl is used? Or, can we see the change list / commit, which made this build run? I will try to find the pnacl version used for this build. If you home-brewed pnacl or if you use too old pnacl, it should be problem in your side. If not, our side. I think.
,
Jul 6 2016
It's not a homebrew pnacl. This is one of our official builders, so it uses source from Chrome master. But I think it is trying to build Chrome 51, the last stable version. That branched on April 8. Would that be too old? stevenjb - I'm not sure how to find the pnacl version. Do you know?
,
Jul 6 2016
(The chrome official linux precise64 builder last build chrome 51 on July 1, so not too long ago. https://uberchromegw.corp.google.com/i/official.desktop.continuous/builders/precise64%20stable/builds/681 )
,
Jul 6 2016
No clue, sorry. llzano@ may know.
,
Jul 7 2016
The builder is building Chrome 51.0.2704.106 When I checkout chrome at that revision and look in the native_client files I see this: cat native_client/toolchain/linux_x86/pnacl_newlib/REV [GIT] https://chromium.googlesource.com/a/native_client/src/native_client.git: 2b6dfa6dd95e45d3873b60c64bec6d56c18d11a0 [GIT] https://chromium.googlesource.com/native_client/pnacl-compiler-rt.git: 9ded361c356ebc38a115e8134b8f7ca4f1d61eb0 [GIT] https://chromium.googlesource.com/native_client/pnacl-gcc.git: 574429118e460375c377728420c43aad4a3103dc [GIT] https://chromium.googlesource.com/native_client/pnacl-llvm.git: f6a5f463657a536ef3f55ce4b4704a7131e4f7ca [GIT] https://chromium.googlesource.com/native_client/pnacl-libcxxabi.git: de05b63efeb5df83abe927c08f330c35995c82a7 [GIT] https://chromium.googlesource.com/native_client/pnacl-llvm-testsuite.git: 6438a2b505fa29f82eb850aed1785db1738b971c [GIT] https://chromium.googlesource.com/native_client/pnacl-libcxx.git: 91a5433c0c5e891098987276f3c6e98fe5c1b86a [GIT] https://chromium.googlesource.com/native_client/pnacl-subzero.git: 45f7700f3d51ba04c1d82b04a58b2cfeeb830441 [GIT] https://chromium.googlesource.com/native_client/pnacl-clang.git: ce163fdd0f16b4481e5cf77a16d45e9b4dc8300e [GIT] https://chromium.googlesource.com/native_client/nacl-binutils.git: 1d8592cc9f02cc9aeaef992c296376a8fd4c8761 [GIT] https://chromium.googlesource.com/native_client/nacl-binutils.git: d5a23a319f685b7b853346eafe800d8ba8a69bac [GIT] https://chromium.googlesource.com/native_client/nacl-newlib.git: 784956835fd318fa64e513ead7774d897386a7be The pnacl uprev is from Sat Apr 2 09:39:43 2016 -0700, which is just before the M51 branch, so that seems right. These are the versions I see for pnacl-clang: jamescook@rubella2:/w/chrome/src/native_client/toolchain/linux_x86/pnacl_newlib/bin ((bdf73e7...))$ ./pnacl-clang --version clang version 3.7.0 (https://chromium.googlesource.com/a/native_client/pnacl-clang.git ce163fdd0f16b4481e5cf77a16d45e9b4dc8300e) (https://chromium.googlesource.com/a/native_client/pnacl-llvm.git f6a5f463657a536ef3f55ce4b4704a7131e4f7ca) nacl-version=2b6dfa6dd95e45d3873b60c64bec6d56c18d11a0 Target: le32-unknown-nacl Thread model: posix jamescook@rubella2:/w/chrome/src/native_client/toolchain/linux_x86/pnacl_newlib/bin ((bdf73e7...))$ ./pnacl-clang++ --version clang version 3.7.0 (https://chromium.googlesource.com/a/native_client/pnacl-clang.git ce163fdd0f16b4481e5cf77a16d45e9b4dc8300e) (https://chromium.googlesource.com/a/native_client/pnacl-llvm.git f6a5f463657a536ef3f55ce4b4704a7131e4f7ca) nacl-version=2b6dfa6dd95e45d3873b60c64bec6d56c18d11a0 Target: le32-unknown-nacl Thread model: posix Is that what you need?
,
Jul 7 2016
about #7. sorry, I don't know anything about the pnacl compiler.
,
Jul 7 2016
I think April 8 is old. To keep the compiler older than a month available, you need to keep using it, or we think you do not need it. By the way, I have revived the compiler in backend, and might become usable soon.
,
Jul 7 2016
stevenjb, do you know why this builder rolls back in time to earlier revisions? It seems like this will cause us trouble again in the future. yyanagisawa, thanks for reviving the compiler. Can you tell when was the last time it was used? I think our Linux stable builders should be using the same binary, and those were running last week.
,
Jul 7 2016
>> #11 As far as I guessed from the removal date, the last used date might be May 22 or so? https://goto.google.com/qubjd Also, I tried to see what compiler is used from logs but I could not. filed https://bugs.chromium.org/p/chromium/issues/detail?id=626223
,
Jul 7 2016
yyanagisawa, I think the compiler was removed on June 21st. https://groups.google.com/a/google.com/forum/#!original/goma-alerts/Ja-wwXZfOT0/F8jDCUxdAwAJ - the git hash of clang-3.7.0-unknown-pnacl_newlib-r2b6dfa6dd95e45d3873b60c64bec6d56 matches the version in comment #8 I looked back at the last 200 builds of this builder and this issue has not happened before. https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/Linux%20ChromeOS%20Buildspec%20Tests?numbuilds=300 I added docs to YAQS about how to find the nacl compiler version: https://yaqs.googleplex.com/eng/q/6425187769122816 I think this issue is fixed. If it happens again we can investigate more.
,
Jul 7 2016
Re comment #11 - the builder gets triggered for any update to the release branch, i.e. any time a release gets updated (e.g. 51, 52 or soon 53) it will run. So it isn't "rolling back" per se. I agree it is confusing, it would be better if we had a separate builder for Release vs. Beta vs. Canary. I'm sure I have a note somewhere to file a bug for that... or maybe I already did?
,
Aug 29 2016
,
Aug 29 2016
bulk verified |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by hinoka@chromium.org
, Jul 6 2016Labels: -Infra-Troopers