The Build System Periodically Fails to Find Mojom Files |
||||||||
Issue descriptionFiled by sheriff-o-matic@appspot.gserviceaccount.com on behalf of robliao@chromium.org compile failure on chromium/Win Periodically, these errors appear: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win/68730 [56903/72256] CXX obj/third_party/blink/renderer/bindings/modules/generated/event_modules_factory.obj FAILED: obj/third_party/blink/renderer/bindings/modules/generated/event_modules_factory.obj C:\b\swarming\w\ir\cache\goma\client/gomacc.exe ../../third_party/llvm-build/Release+Asserts/bin/clang-cl.exe /nologo /showIncludes -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\um -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\shared -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\winrt -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\ucrt -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\VC\Tools\MSVC\14.14.26428\include -imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\VC\Tools\MSVC\14.14.26428\atlmfc\include -DV8_DEPRECATION_WARNINGS -DUSE_AURA=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED "-DCR_CLANG_REVISION=\"334100-1\"" -D_HAS_NODISCARD -D_HAS_EXCEPTIONS=0 -D__STD_C -D_CRT_RAND_S -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -D_ATL_NO_OPENGL -D_WINDOWS -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -DPSAPI_VERSION=1 -DWIN32 -D_SECURE_ATL -D_USING_V110_SDK71_ -DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP -DWIN32_LEAN_AND_MEAN -DNOMINMAX -D_UNICODE -DUNICODE -DNTDDI_VERSION=0x0A000002 -D_WIN32_WINNT=0x0A00 -DWINVER=0x0A00 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DBLINK_MODULES_IMPLEMENTATION=1 -DBLINK_IMPLEMENTATION=1 -DINSIDE_BLINK -DWEBP_EXTERN=extern -DUSE_EGL -DENABLE_IPC_FUZZER -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=wchar_t -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_SUPPORT_GPU=1 "-DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\"" -DGR_GL_FUNCTION_TYPE=__stdcall -D__STD_C -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -DWTF_USE_WEBAUDIO_FFMPEG=1 -DWTF_USE_DEFAULT_RENDER_THEME=1 -DUSE_LIBJPEG_TURBO=1 -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DGTEST_RELATIVE_PATH -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_WIN -DABSL_ALLOCATOR_NOTHROW=1 -DNO_MAIN_THREAD_WRAPPING -DLIBXSLT_STATIC -I../.. -Igen -I../../third_party/libwebp/src -I../../third_party/wtl/include -I../../third_party/khronos -I../../gpu -I../../third_party/libyuv/include -I../../third_party/protobuf/src -I../../third_party/protobuf/src -Igen/protoc_out -I../../third_party/boringssl/src/include -I../../third_party/ced/src -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../skia/config -I../../skia/ext -I../../third_party/skia/include/c -I../../third_party/skia/include/config -I../../third_party/skia/include/core -I../../third_party/skia/include/effects -I../../third_party/skia/include/encode -I../../third_party/skia/include/gpu -I../../third_party/skia/include/images -I../../third_party/skia/include/lazy -I../../third_party/skia/include/pathops -I../../third_party/skia/include/pdf -I../../third_party/skia/include/pipe -I../../third_party/skia/include/ports -I../../third_party/skia/include/utils -I../../third_party/skia/src/gpu -I../../third_party/skia/src/sksl -I../../third_party/angle/include -I../../third_party/angle/src/common/third_party/base -Igen/angle -I../../third_party/blink/renderer/platform/wtf/os-win32 -I../../third_party/libjpeg_turbo -I../../v8/include -Igen/v8/include -I../../third_party/webrtc_overrides -I../../third_party/webrtc -I../../third_party/iccjpeg -I../../third_party/libpng -I../../third_party/zlib -I../../third_party/ots/include -I../../v8/include -Igen/v8/include -I../../third_party/libxml/src/include -I../../third_party/libxml/win32/include -I../../third_party/libxslt/src -I../../third_party/snappy/src -I../../third_party/snappy/win32 /utf-8 /X -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -fcolor-diagnostics -fmerge-all-constants -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 -no-canonical-prefixes /Gy /FS /bigobj /d2FastFail /Zc:sizedDealloc- -fmsc-version=1911 -m32 /Brepro /W4 -Wimplicit-fallthrough -Wthread-safety /WX /wd4091 /wd4127 /wd4251 /wd4275 /wd4312 /wd4324 /wd4351 /wd4355 /wd4503 /wd4589 /wd4611 /wd4100 /wd4121 /wd4244 /wd4505 /wd4510 /wd4512 /wd4610 /wd4838 /wd4995 /wd4996 /wd4456 /wd4457 /wd4458 /wd4459 /wd4200 /wd4201 /wd4204 /wd4221 /wd4018 /wd4245 /wd4267 /wd4305 /wd4389 /wd4702 /wd4701 /wd4703 /wd4661 /wd4706 /wd4715 /wd4267 -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-covered-switch-default -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-address-of-packed-member -Wno-nonportable-include-path -Wno-user-defined-warnings -Wno-unused-lambda-capture -Wno-null-pointer-arithmetic -Wno-enum-compare-switch -Wno-ignored-pragma-optimize /O1 /Ob2 /Oy- /Zc:inline /Gw /Oi /MT -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang enforce-in-thirdparty-webkit -Xclang -plugin-arg-find-bad-constructs -Xclang check-enum-max-value -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare /wd4305 /wd4324 /wd4714 /wd4800 /wd4996 -Xclang -add-plugin -Xclang blink-gc-plugin -Wglobal-constructors -Wno-inconsistent-missing-override /wd4344 /wd4706 -imsvc ../../third_party/abseil-cpp -DLIBXML_STATIC= /TP /wd4577 /GR- /c gen/third_party/blink/renderer/modules/event_modules_factory.cc /Foobj/third_party/blink/renderer/bindings/modules/generated/event_modules_factory.obj /Fd"obj/third_party/blink/renderer/bindings/modules/generated_cc.pdb" In file included from gen/third_party/blink/renderer/modules/event_modules_factory.cc:26: In file included from ../..\third_party/blink/renderer/modules/gamepad/gamepad_event.h:9: In file included from ../..\third_party/blink/renderer/modules/gamepad/gamepad.h:32: ../..\third_party/blink/renderer/modules/gamepad/gamepad_haptic_actuator.h(9,10): fatal error: 'device/gamepad/public/mojom/gamepad.mojom-blink.h' file not found #include "device/gamepad/public/mojom/gamepad.mojom-blink.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win/68639 [65709/72088] CXX obj/chrome/browser/test_support_ui/password_manager_test_base.obj FAILED: obj/chrome/browser/test_support_ui/password_manager_test_base.obj C:\b\swarming\w\ir\cache\goma\client/gomacc.exe ../../third_party/llvm-build/Release+Asserts/bin/clang-cl.exe /nologo /showIncludes "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\um" "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\shared" "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\winrt" "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\win_sdk\Include\10.0.17134.0\ucrt" "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\VC\Tools\MSVC\14.14.26428\include" "-imsvc..\..\..\..\win_toolchain\vs_files\3bc0ec615cf20ee342f3bc29bc991b5ad66d8d2c\VC\Tools\MSVC\14.14.26428\atlmfc\include" -DV8_DEPRECATION_WARNINGS -DUSE_AURA=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED -D"CR_CLANG_REVISION=\"334100-1\"" -D_HAS_NODISCARD -D_HAS_EXCEPTIONS=0 -D__STD_C -D_CRT_RAND_S -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -D_ATL_NO_OPENGL -D_WINDOWS -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -DPSAPI_VERSION=1 -DWIN32 -D_SECURE_ATL -D_USING_V110_SDK71_ -DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP -DWIN32_LEAN_AND_MEAN -DNOMINMAX -D_UNICODE -DUNICODE -DNTDDI_VERSION=0x0A000002 -D_WIN32_WINNT=0x0A00 -DWINVER=0x0A00 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DGTEST_API_= -DGTEST_HAS_POSIX_RE=0 -DGTEST_LANG_CXX11=1 -DGTEST_HAS_TR1_TUPLE=0 -DWEBP_EXTERN=extern -DTOOLKIT_VIEWS=1 -DUSE_EGL -DU_USING_ICU_NAMESPACE=0 -DU_ENABLE_DYLOAD=0 -DU_STATIC_IMPLEMENTATION -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_FILE -DUCHAR_TYPE=wchar_t -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DUNIT_TEST -DENABLE_IPC_FUZZER -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_SUPPORT_GPU=1 -D"SK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\"" -DGR_GL_FUNCTION_TYPE=__stdcall -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DGTEST_RELATIVE_PATH -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_WIN -DABSL_ALLOCATOR_NOTHROW=1 -DNO_MAIN_THREAD_WRAPPING -DV8_USE_EXTERNAL_STARTUP_DATA -DLEVELDB_PLATFORM_CHROMIUM=1 -DDeleteFile=DeleteFileW -DOS_WIN -I../.. -Igen -I../../third_party/googletest/custom -I../../third_party/googletest/src/googletest/include -I../../third_party/libwebp/src -I../../third_party/wtl/include -I../../third_party/khronos -I../../gpu -I../../third_party/libyuv/include -I../../third_party/ced/src -I../../third_party/icu/source/common -I../../third_party/icu/source/i18n -I../../third_party/boringssl/src/include -I../../third_party/protobuf/src -Igen/protoc_out -I../../third_party/protobuf/src -I../../third_party/googletest/custom -I../../third_party/googletest/src/googlemock/include -Igen/third_party/metrics_proto -I../../skia/config -I../../skia/ext -I../../third_party/skia/include/c -I../../third_party/skia/include/config -I../../third_party/skia/include/core -I../../third_party/skia/include/effects -I../../third_party/skia/include/encode -I../../third_party/skia/include/gpu -I../../third_party/skia/include/images -I../../third_party/skia/include/lazy -I../../third_party/skia/include/pathops -I../../third_party/skia/include/pdf -I../../third_party/skia/include/pipe -I../../third_party/skia/include/ports -I../../third_party/skia/include/utils -I../../third_party/skia/src/gpu -I../../third_party/skia/src/sksl -I../../v8/include -Igen/v8/include -I../../third_party/webrtc_overrides -I../../third_party/webrtc -I../../third_party/leveldatabase -I../../third_party/leveldatabase/src -I../../third_party/leveldatabase/src/include -I../../third_party/libwebm/source /utf-8 /X -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -fcolor-diagnostics -fmerge-all-constants -Xclang -mllvm -Xclang -instcombine-lower-dbg-declare=0 /Gy /FS /bigobj /d2FastFail /Zc:sizedDealloc- -fmsc-version=1911 -m32 /Brepro /W4 -Wimplicit-fallthrough -Wthread-safety /WX /wd4091 /wd4127 /wd4251 /wd4275 /wd4312 /wd4324 /wd4351 /wd4355 /wd4503 /wd4589 /wd4611 /wd4100 /wd4121 /wd4244 /wd4505 /wd4510 /wd4512 /wd4610 /wd4838 /wd4995 /wd4996 /wd4456 /wd4457 /wd4458 /wd4459 /wd4200 /wd4201 /wd4204 /wd4221 /wd4267 -Wno-missing-field-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-covered-switch-default -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-address-of-packed-member -Wno-nonportable-include-path -Wno-user-defined-warnings -Wno-unused-lambda-capture -Wno-null-pointer-arithmetic -Wno-enum-compare-switch -Wno-ignored-pragma-optimize /O1 /Ob2 /Oy- /Zc:inline /Gw /Oi /MT -Xclang -add-plugin -Xclang find-bad-constructs -Xclang -plugin-arg-find-bad-constructs -Xclang enforce-in-thirdparty-webkit -Xclang -plugin-arg-find-bad-constructs -Xclang check-enum-max-value -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare /wd4800 -Wno-inconsistent-missing-override -Wno-inconsistent-missing-override -imsvc ../../third_party/abseil-cpp /TP /wd4577 /GR- /c ../../chrome/browser/password_manager/password_manager_test_base.cc /Foobj/chrome/browser/test_support_ui/password_manager_test_base.obj /Fd"obj/chrome/browser/test_support_ui_cc.pdb" In file included from ../../chrome/browser/password_manager/password_manager_test_base.cc:18: ../..\chrome/browser/password_manager/chrome_password_manager_client.h(15,10): fatal error: 'components/autofill/content/common/autofill_driver.mojom.h' file not found #include "components/autofill/content/common/autofill_driver.mojom.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated.
,
Jun 15 2018
,
Jun 15 2018
these are almost certainly build flakes caused by bugs in deps lists. in this case, I think //third_party/blink/renderer/bindings/modules:event_modules should but doesn't depend on //third_party/blink/renderer/modules/gamepad which correctly depends on //device/gamepad/public/mojom:mojom_blink
,
Jun 15 2018
#3 is re your first link. I think I fixed your second link w/ https://chromium-review.googlesource.com/c/chromium/src/+/1099493
,
Jun 15 2018
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20x64/66495 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Mac/42702 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win/68776 https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Win%20x64/23910 failed at nearly the same time likely due to this.
,
Jun 15 2018
#5: those all appear to be due to https://chromium.googlesource.com/chromium/src/+/abe9813173cdc333082607e495b853234a79954d
,
Jun 15 2018
Indeed, but it's not clear how abe9813173cdc333082607e495b853234a79954d. That change simply included a header which included a chain that included a mojom file. Surely each mojom file does not need to be explicitly included by transitivity.
,
Jun 15 2018
#7: given that those headers are generated by the build, the mojom targets do need to be included in a target's deps (directly or indirectly).
,
Jun 15 2018
Another related failure:
../../third_party/blink/renderer/platform/scheduler/public/frame_scheduler.h:12:10: fatal error: 'third_party/blink/public/mojom/loader/pause_subresource_loading_handle.mojom-blink.h' file not found
#include "third_party/blink/public/mojom/loader/pause_subresource_loading_handle.mojom-blink.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
https://logs.chromium.org/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8943643434236746032%2F%2B%2Fsteps%2FFirst_build%2F0%2Fstdout
Since more than one devs made the same error of including a generated file without depending on the target that generates it, could we add a check in GN, that would make failures more deterministic?
,
Jun 15 2018
components/autofill/content/common/autofill_driver.mojom.h seems a generated header file. From my experience dealing with flaky builds, it is most likely caused by misconfigured indirect dependency in GN file.
,
Jun 15 2018
#9: I think gn already supports that, but there's nontrivial work in enabling it in chromium, iirc.
,
Jun 15 2018
Correct. This isn't something we can automatically catch IIRC, but I'm not sure how much of this is due to `gn check` not being enabled vs. other things. @rockot - can you take a look and remind me what the deal is here?
,
Jun 15 2018
It's always going to be caused by underspecified dependencies. mojom dependencies on mojoms are fine because they're enforced by the bindings generator itself, but there are two legitimate classes of failure here: a. Places where GN check is disabled and a source file includes a mojom header without a dependency on the right mojom target b. Places where people have circumvented circular dependency checks by not depending on real mojom targets, but on e.g. intermediate action targets like [1]. In fact that example that I just randomly found in code search is pretty egregious, considering that it's a "source set" with no sources... [1] https://cs.chromium.org/chromium/src/third_party/blink/renderer/platform/blob/BUILD.gn?rcl=f27798f4413b6f5e01b3c07aba49b9717e3b0cee&l=16
,
Jun 15 2018
#13: funny that you should choose that example; I'm working on that in https://chromium-review.googlesource.com/c/chromium/src/+/1100037 as it also caused a build flake earlier this week. There seem to be some interesting duplicate symbol issues to work though...
,
Jun 15 2018
,
Jun 15 2018
Ok. Removing from the sheriff queue for now, I don't think this is something we can expect a sheriff to follow up on at this point if there's no active failure.
,
Jun 15 2018
One way to have a check for those issues without relying on gn check was proposed in https://bugs.chromium.org/p/chromium/issues/detail?id=655123. It's generally possible to have ninja list all cases where a header gets used by a .cc, but there was no guarantee the .h is always generated first. It would take pushing a (stalled) patch to ninja, or making a separate tool out of the patch linked there.
,
Oct 17
,
Oct 17
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by robliao@chromium.org
, Jun 15 2018Components: Internals>Mojo>Bindings Infra>Client>Chrome
Labels: OS-Windows
Summary: The Build System Periodically Fails to Find Mojom Files (was: compile failure on chromium/Win)