New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 915731 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 19
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug


Sign in to add a comment

commit_id not regenerated by build after ANGLE BUILD.gn change

Project Member Reported by jmad...@chromium.org, Dec 17

Issue description

This change landed in ANGLE:

https://crrev.com/c/1370725

It changed a few targets in ANGLE's BUILD.gn. In particular it made one dependency a 'data_deps' instead of 'deps'. It passed the ANGLE CQ, rolled into Chrome here: https://crrev.com/c/1379518 and proceeded to immediately break win-rel:

https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/win-rel/8006

[2482/65589] RC obj/third_party/angle/libEGL/libEGL.res
FAILED: obj/third_party/angle/libEGL/libEGL.res 
C:/b/swarming/w/ir/cache/vpython/03001a/Scripts/python.exe ../../build/toolchain/win/tool_wrapper.py rc-wrapper environment.x64 rc.exe /nologo -DLIBEGL_IMPLEMENTATION -DEGLAPI= -DV8_DEPRECATION_WARNINGS -DUSE_AURA=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED "-DCR_CLANG_REVISION=\"346388-5\"" -D_HAS_NODISCARD -D_HAS_EXCEPTIONS=0 -D__STD_C -D_CRT_RAND_S -D_CRT_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_DEPRECATE -D_ATL_NO_OPENGL -D_WINDOWS -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS -DPSAPI_VERSION=1 -DWIN32 -D_SECURE_ATL -D_USING_V110_SDK71_ -DWINAPI_FAMILY=WINAPI_FAMILY_DESKTOP_APP -DWIN32_LEAN_AND_MEAN -DNOMINMAX -D_UNICODE -DUNICODE -DNTDDI_VERSION=0x0A000002 -D_WIN32_WINNT=0x0A00 -DWINVER=0x0A00 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DANGLE_IS_64_BIT_CPU "-DANGLE_EGL_LIBRARY_NAME=\"libEGL\"" "-DANGLE_GLESV2_LIBRARY_NAME=\"libGLESv2\"" -DANGLE_USE_EGL_LOADER -DGL_GLES_PROTOTYPES=1 -DEGL_EGL_PROTOTYPES=1 -DGL_GLEXT_PROTOTYPES -DEGL_EGLEXT_PROTOTYPES -Igen/angle -I../../third_party/angle/include -I../../third_party/angle/src /foobj/third_party/angle/libEGL/libEGL.res ../../third_party/angle/src/libEGL/libEGL.rc
In file included from <stdin>:11:
../../third_party/angle/src/libEGL\../common/version.h(10,10):  fatal error: 'id/commit.h' file not found
#include "id/commit.h"
         ^~~~~~~~~~~~~
1 error generated.

I tried to reproduce the failure locally and didn't have any luck. Doing a complete 'gn clean' and then building 'obj/third_party/angle/libEGL/libEGL.res' directly did reproduce the error. But I was unable to reproduce this failure by building any target. I tried libEGL, libEGL_static, angle_end2end_tests, they all seem smart enough to build the commit header when it is missing. The failing build seems to skip over the deps of ANGLE's libEGL entirely, and just build the sources. I was using very similar gn arguments as the builder.

Dirk or Nico, do you have any ideas how to reproduce this? Or how to fix it? At the moment it seems I can't land this change safely. If there's an underlying testing problem this also could affect other projects.

ANGLE's current BUILD.gn is here:
https://cs.chromium.org/chromium/src/third_party/angle/BUILD.gn
 
Looks like that build builds "all" target.
gn has some commands to look at targets relationships, maybe this file is referenced by something else?

Comment 2 Deleted

Looks like this file is supposed to be generated by https://cs.chromium.org/chromium/src/third_party/angle/BUILD.gn?type=cs&q=id%5C/commit&sq=package:chromium&g=0&l=388

I'm guessing https://cs.chromium.org/chromium/src/third_party/angle/src/libGLESv2.gni?type=cs&q=libEGL.res&sq=package:chromium&g=0&l=854 is what gets compiled into libEGL.res? It's compiled here https://cs.chromium.org/chromium/src/third_party/angle/BUILD.gn?type=cs&q=libegl_sources+&sq=package:chromium&g=0&l=794 and here https://cs.chromium.org/chromium/src/third_party/skia/third_party/angle2/BUILD.gn?q=libegl_sources+&sq=package:chromium&dr=C&l=131 (what's angle2? ignoring this branch for now...)

https://cs.chromium.org/chromium/src/third_party/angle/BUILD.gn?type=cs&q=libegl_sources+&sq=package:chromium&g=0&l=794 has

  configs += [
    ":commit_id_config",

but when I look at commit_id_config I think the intent is that clients depend on the "commit_id" action which then pushes the config to its dependees.

(If that's correct, I'd suggest putting visibility = [ ":commit_id" ] in the "commit_id_config" config to make sure that nobody else can make this mistake.)


libEGL_static already depends on libGLESv2_static which depends on libANGLE which does depend on :commit_id. To be honest I would've expected that to be enough, but maybe these deps need to be public_deps? In any case, doing the visiblity thing I suggested and making sure things using commit.h use a dep instead of a config should make things go.
Status: Started (was: Assigned)
Thanks very much Nico. I didn't know I could get so much information from "gn check". Working through the warnings now.
Seems something is wrong here that I don't fully understand. We're getting test failures on the ANGLE CQ that result from a missing dynamically loaded symbol from libGLESv2->libEGL. We suspect this is because we have a dirty build state not triggering a recompile of libEGL. See  issue angleproject:3030 . Yuly and I looked into this by debugging on the Android devices on the swarming infra. They are producing errors of the form:

AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZN3egl12ChooseConfigEPvPKiPS0_iPi" referenced by "/d
ata/app/org.chromium.native_test-Fd6YrZyFzN-sJ81NzQXcJw==/lib/arm64/libEGL.so

Also Yuly noticed a suspicious compile step on a try job that should have rebuilt hundreds of objects:

https://www.google.com/url?q=https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8926859325032697680/%2B/steps/compile__with_patch_/0/stdout&sa=D&source=hangouts&ust=1545168833625000&usg=AFQjCNH0TgvMAgY_TScDTHnWHR8wGzHQNQ

It seems like for some reason libEGL is not getting rebuilt when it should be. Will take a bit more investigation. Yuly also proposed asking the labs team to blow away the build caches.
Landed the following CL with the wrong bug tag:

The following revision refers to this bug:
  https://chromium.googlesource.com/angle/angle/+/720ca449767c38f48bebfc6a8c5821e43176c26d

commit 720ca449767c38f48bebfc6a8c5821e43176c26d
Author: Jamie Madill <jmadill@chromium.org>
Date: Tue Dec 18 15:25:40 2018

Pass GN header visibility check.

This fixes a few things:

* removes includes that weren't supposed to be present
* scopes some compiler code into white_box_perftests
* makes version.h/commit and angle_common id more visible
* roll zlib to a version that passes check

This should help prevent build problems from popping up in the
downstream Chromium build. We could also potentially look at
including gn check in our CQ recipe.

Bug: chromium:915429
Change-Id: I350f543e16de13c84eb2c43260f4966d47185114
Reviewed-on: https://chromium-review.googlesource.com/c/1380771
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>

[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/BUILD.gn
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/test_utils/angle_test_instantiate.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/samples/BUILD.gn
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/perf_tests/BitSetIteratorPerf.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/test_utils/angle_test_instantiate.h
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/perf_tests/ANGLEPerfTest.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/angle_perftests.gni
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/BUILD.gn
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/test_utils/angle_test_configs.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/perf_tests/CompilerPerf.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/DEPS
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/libGLESv2.gni
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/perf_tests/MultiviewPerf.cpp
[modify] https://crrev.com/720ca449767c38f48bebfc6a8c5821e43176c26d/src/tests/test_utils/angle_test_configs.h
I was able to repro the commit_id failure and fix it in the re-land. Unfortunately I haven't yet been able to root cause  issue angleproject:3030  . I'll continue looking at that in other issues.
Project Member

Comment 8 by bugdroid1@chromium.org, Dec 18

The following revision refers to this bug:
  https://chromium.googlesource.com/angle/angle/+/4638dc9def816e8ee86b09293b8b90519d4a5d9f

commit 4638dc9def816e8ee86b09293b8b90519d4a5d9f
Author: Jamie Madill <jmadill@chromium.org>
Date: Tue Dec 18 18:54:44 2018

Re-land "Load correct libGLESv2 on Linux and Mac."

Re-land fixes build to ensure commit_id is built before libEGL.

libEGL was implicitly loading libGLESv2 on startup. This is bad
because on platforms like Linux and Mac we could sometimes use the
incorrect rpath. This in turn meant we needed workarounds like using
"_angle" extensions to our shared objects to get the correct loading
behaviour.

Fix this by loading libGLESv2 dynamically in libEGL. We build the
loader automatically from egl.xml. The loader itself is lazily
initialized on every EGL entry point call. This is necessary because
on Linux, etc, there is no equivalent to Windows' DLLMain.

We also use an EGL.h with different generation options so we have the
proper function pointer types. A README is included for instructions
on how to regenerate EGL.h.

The entry point generation script is refactored into a helper class
that is used in the loader generator. Also adds the libGLESv2 versions
of the EGL entry points in the DEF file on Windows. This allows them to
be imported properly in 32-bit configurations.

Also fixes up some errors in ANGLE's entry point definitions. Also
includes a clang-format disable rule for the Khronos headers.

This CL will help us to run ANGLE tests against native drivers.

Bug:  angleproject:2871 
Bug:  chromium:915731 
Change-Id: I4192a938d1f4117cea1bf1399c98bda7ac25ddab
Reviewed-on: https://chromium-review.googlesource.com/c/1380511
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>

[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/entry_points_egl.h
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libEGL/libEGL.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/generate_entry_points.py
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libEGL/egl_loader_autogen.cpp
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES/.clang-format
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/gl_angle_ext.xml
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/run_code_generation_hashes.json
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/entry_points_egl_ext.h
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/KHR/.clang-format
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2.gni
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/registry_xml.py
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/proc_table_autogen.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/EGL/eglext_angle.h
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/EGL/egl.h
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/run_code_generation.py
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/tests/test_utils/ANGLETest.cpp
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES3/.clang-format
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/common/system_utils_win.cpp
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/EGL/.clang-format
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/egl_angle_ext.xml
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/gen_proc_table.py
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/util/posix/Posix_system_utils.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES2/gl2ext_explicit_context_autogen.inc
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/common/system_utils_posix.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/entry_points_egl_ext.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/common/system_utils_linux.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES3/gl31ext_explicit_context_autogen.inc
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/util/system_utils.h
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/entry_points_egl.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/util/windows/win32/Win32_system_utils.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/BUILD.gn
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/egl.xml
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libGLESv2/libGLESv2_autogen.def
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/common/system_utils.h
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/EGL/README.md
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES2/.clang-format
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/scripts/generate_loader.py
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/common/system_utils_mac.cpp
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/samples/BUILD.gn
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES/glext_explicit_context_autogen.inc
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/util/windows/winrt/WinRT_system_utils.cpp
[add] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/libEGL/egl_loader_autogen.h
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/include/GLES3/gl3ext_explicit_context_autogen.inc
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/tests/BUILD.gn
[modify] https://crrev.com/4638dc9def816e8ee86b09293b8b90519d4a5d9f/src/tests/gl_tests/ExplicitContextTest.cpp

Labels: Hotlist-PixelWrangler
Re-land CL should be rolling in soon. CC'ing pixel wrangler list to keep a heads up for build failures.
Project Member

Comment 10 by bugdroid1@chromium.org, Dec 18

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/e8e5b90e87948f6ee121cd03c1dd40ea2ce24523

commit e8e5b90e87948f6ee121cd03c1dd40ea2ce24523
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Tue Dec 18 23:05:59 2018

Roll src/third_party/angle 720ca449767c..4638dc9def81 (1 commits)

https://chromium.googlesource.com/angle/angle.git/+log/720ca449767c..4638dc9def81


git log 720ca449767c..4638dc9def81 --date=short --no-merges --format='%ad %ae %s'
2018-12-18 jmadill@chromium.org Re-land "Load correct libGLESv2 on Linux and Mac."


Created with:
  gclient setdep -r src/third_party/angle@4638dc9def81

The AutoRoll server is located here: https://autoroll.skia.org/r/angle-chromium-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:android_optional_gpu_tests_rel;luci.chromium.try:linux_optional_gpu_tests_rel;luci.chromium.try:mac_optional_gpu_tests_rel;luci.chromium.try:win_optional_gpu_tests_rel

BUG= chromium:915731 
TBR=geofflang@chromium.org

Change-Id: I91396c72efd335976a2db8765fe1064bb3e2e431
Reviewed-on: https://chromium-review.googlesource.com/c/1382763
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#617664}
[modify] https://crrev.com/e8e5b90e87948f6ee121cd03c1dd40ea2ce24523/DEPS

Status: Fixed (was: Started)
Blocking: angleproject:3052

Sign in to add a comment