New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 821871 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug



Sign in to add a comment

cygwin heap mismatch in buildbot build

Project Member Reported by brat...@opera.com, Mar 14 2018

Issue description

Not really sure where to file this weird buildbot error:

https://logs.chromium.org/v/?s=chromium%2Fbb%2Ftryserver.chromium.win%2Fwin10_chromium_x64_rel_ng%2F100922%2F%2B%2Frecipes%2Fsteps%2Fcompile__with_patch_%2F0%2Fstdout

FAILED: glibc_x64/libppapi_cpp_lib.dll glibc_x64/libppapi_cpp_lib.dll.TOC glibc_x64/lib.unstripped/libppapi_cpp_lib.dll 
C:/b/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe "../../build/toolchain/gcc_solink_wrapper.py" --readelf="../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-readelf.exe" --nm="../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-nm.exe" --strip=../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-strip.exe --sofile="glibc_x64/lib.unstripped/libppapi_cpp_lib.dll" --tocfile="glibc_x64/libppapi_cpp_lib.dll.TOC" --output="glibc_x64/libppapi_cpp_lib.dll"  -- ../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-g++.exe -shared -m64 -Werror -Wl,-O2 -Wl,--gc-sections -Wl,--fatal-warnings -o "glibc_x64/lib.unstripped/libppapi_cpp_lib.dll" -Wl,-soname="libppapi_cpp_lib.dll" @"glibc_x64/libppapi_cpp_lib.dll.rsp"
      3 [main] x86_64-nacl-g++ (3816) C:\b\c\b\win\src\native_client\toolchain\win_x86\nacl_x86_glibc\libexec\x86_64-nacl-g++.exe: *** fatal error - cygheap base mismatch detected - 0x1830970/0x1910970.
This problem is probably due to using incompatible versions of the cygwin DLL.
Search for cygwin1.dll using the Windows Start->Find/Search facility
and delete all but the most recent version.  The most recent version *should*
reside in x:\cygwin\bin, where 'x' is the drive on which you have
installed the cygwin distribution.  Rebooting is also suggested if you
are unable to find another cygwin DLL.
      1 [main] x86_64-nacl-g++ 4208 fork: child -1 - forked process 3816 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
      3 [main] x86_64-nacl-g++ (3480) C:\b\c\b\win\src\native_client\toolchain\win_x86\nacl_x86_glibc\libexec\x86_64-nacl-g++.exe: *** fatal error - cygheap base mismatch detected - 0x1830970/0x1960970.

 
Cc: jbudorick@chromium.org smut@chromium.org brucedaw...@chromium.org h...@chromium.org bradnelson@chromium.org scottmg@chromium.org thakis@chromium.org tikuta@chromium.org wfh@chromium.org
Components: -Infra Build
Strange ... adding a few others that might've seen this and/or have ideas. 

Maybe there's some version skew between this bot config and the cygwin version that NaCl depends on?

Comment 2 by s...@google.com, Mar 15 2018

cygwin isn't installed on Windows 10
I've seen this happen before when ASLR causes the address where cygwin wants its heap to go to be already taken. Rebooting is often the fix.

The message is mostly useless except if it gives good results in a Google search.

Since the problem seemed to go away I think it is reasonable to write it off as "cygwin is occasionally flaky on Windows".

Happened again:
https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win-asan/79
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket.appspot.com/8933198785476488832/+/steps/compile__with_patch_/0/logs/raw_io.output_failure_summary_/0

[16811/51747] LINK glibc_x64/ppapi_nacl_tests_glibc_x86_64.nexe
FAILED: glibc_x64/ppapi_nacl_tests_glibc_x86_64.nexe glibc_x64/exe.unstripped/ppapi_nacl_tests_glibc_x86_64.nexe
C:/b/swarming/w/ir/cache/vpython/5e8340/Scripts/python.exe "../../build/toolchain/gcc_link_wrapper.py" --output="glibc_x64/ppapi_nacl_tests_glibc_x86_64.nexe" --strip="../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-strip.exe" --unstripped-file="glibc_x64/exe.unstripped/ppapi_nacl_tests_glibc_x86_64.nexe" -- ../../native_client/toolchain/win_x86/nacl_x86_glibc/bin/x86_64-nacl-g++.exe -pthread -Wl,--fatal-warnings -m64 -Werror -Wl,-O2 -Wl,--gc-sections -o "glibc_x64/exe.unstripped/ppapi_nacl_tests_glibc_x86_64.nexe" -Wl,--start-group @"glibc_x64/ppapi_nacl_tests_glibc_x86_64.nexe.rsp" glibc_x64/libppapi_cpp_lib.dll -Wl,--end-group
3 [main] x86_64-nacl-g++ (5316) C:\b\swarming\w\ir\cache\builder\src\native_client\toolchain\win_x86\nacl_x86_glibc\libexec\x86_64-nacl-g++.exe: *** fatal error - cygheap base mismatch detected - 0x1B20970/0x1AB0970.
This problem is probably due to using incompatible versions of the cygwin DLL.
  Search for cygwin1.dll using the Windows Start->Find/Search facility
  and delete all but the most recent version.  The most recent version *should*
  reside in x:\cygwin\bin, where 'x' is the drive on which you have
  installed the cygwin distribution.  Rebooting is also suggested if you
  are unable to find another cygwin DLL.

What would it take to exorcise Cygwin fork emulation from the build? I suspect it may get flakier for new Windows versions.
Owner: dschuff@chromium.org
I do not know; dschuff@, any ideas?
Just ran across this again, sorry for missing it.

Currently I'm in the process (in what amounts to my %20 time) of migrating all of NaCl's build infrastructure to LUCI. I'm also taking this opportunity to think about our test matrix again and get something a bit more sustainable. For Chromium builds, we basically have a couple test suites which cover all of the the PPAPI APIs, across all the NaCl compilers (there are 4 of those), across all the OSes. Since we can check separately that the compilers generate the same code on different OSes, we can probably reduce that matrix (for example by only testing the gcc/glibc compiler on Linux). I'll look into that once I get the build infrastructure stable.
Status: Assigned (was: Untriaged)
This issue has an owner, a component and a priority, but is still listed as untriaged or unconfirmed. By definition, this bug is triaged. Changing status to "assigned". Please reach out to me if you disagree with how I've done this.

Sign in to add a comment