New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 889990 link

Starred by 3 users

Issue metadata

Status: ExternalDependency
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 2
Type: Bug

Blocked on:
issue 891181



Sign in to add a comment

Cannot link release binaries (dll/exe) on Windows after clang roll (when the F disk on the computer is not a normal disk)

Project Member Reported by brat...@opera.com, Sep 27

Issue description

Since the clang roll in 34893e97f49e129ceb545f4edb3758e4b1bee1e5 last week I've been unable to link any release binaries on Windows. 

(Commit subject: Roll clang 340925:342523.)

Simple verification test is:
git checkout 34893e97f49e129ceb545f4edb3758e4b1bee1e5
gclient sync
ninja -C out/Default bytecode_builtins_list_generator.exe
link failure

git checkout HEAD^
gclient sync
ninja -C out/Default bytecode_builtins_list_generator.exe

works.

The link failures have no meaningful error messages but are of the form:
FAILED: bytecode_builtins_list_generator.exe bytecode_builtins_list_generator.exe.pdb
ninja -t msvc -e environment.x64 -- ../../third_party/llvm-build/Release+Asserts/bin/lld-link.exe /nologo /OUT:./bytecode_builtins_list_generator.exe /PDB:./bytecode_builtins_list_generator.exe.pdb @./bytecode_builtins_list_generator.exe.rsp
lld-link: error: Cannot use debug info for 'libcmt.lib'
>>> failed to load reference 'f:\binaries.amd64ret\lib\amd64\libcmt.amd64.pdb': resource unavailable try again

lld-link: error: Cannot use debug info for 'libcmt.lib'
>>> failed to load reference 'f:\binaries.amd64ret\lib\amd64\libcmt.amd64.pdb': resource unavailable try again

[repeated with minor variations about 20 times]

There is nothing special with bytecode_builtins_list_generator.exe but it has few dependencies making it a quick test.

I tried to link the same binary with link.exe and got another error for libcmt.lib about a missing enclave method, but that might not be relevant.

I believe the libcmt.lib file and similar come from Visual C++, and I verified with processmonitor that it's from there lld-link does load the files.

I've reinstalled and upgraded Visual Studio to 2017, update 8.5 in case it would help, which it didn't. It's also reproducible in a brand new checkout.
This is one a Windows 7 computer.
Relevant environment variables set are:

set PATH=%PATH%;L:\src\depot_tools
set LANG=C
set DEPOT_TOOLS_WIN_TOOLCHAIN=0
set GYP_GENERATORS=ninja,msvs-ninja
set GYP_MSVS_VERSION=2017

(yes, I should remove the GYP_GENERATORS part)

args.gn is:

is_debug = false

I'll attach the rsp file for bytecode_builtins_list_generator.exe 


 
bytecode_builtins_list_generator.exe.rsp
1.3 KB Download
I get this too and I'm doing a debug build:

target_cpu = "x86"
is_component_build = true
is_debug = true
is_win_fastlink = true
enable_nacl = false
remove_webcore_debug_symbols = true
And my error is for a different library:

lld-link: error: Cannot use debug info for 'atls.lib'
>>> failed to load reference 'f:\binaries.x86ret\atlmfc\lib\i386\atls.i386.pdb': The parameter is incorrect.

bcwhite@ - do you also have set DEPOT_TOOLS_WIN_TOOLCHAIN=0 set? I'm pretty sure that is the crucial part. That means that you are using whatever version of VS is installed on your machine and it sounds like that must be incompatible.

That said, I cannot reproduce this. I have VS 2017 15.8.5 installed and with is_debug=false I can build bytecode_builtins_list_generator.exe.

bcwhite@ - can you confirm what OS you are using as well?

In my install of VS 2017 I see that libcmt.lib and libcmt.amd64.pdb are sitting together in the same directory:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\lib\x64

Is that the case for you? Perhaps lld-link is trying to use the absolute path to the PDB instead of following the relative path, but if that was the case then I would expect to see the problem on my machine and I don't, despite the fact that all of the .pdb mentions in libcmt.lib have a full path:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\lib\x64>strings libcmt.lib | find /i ".pdb"
f:\binaries.amd64ret\lib\amd64\libcmt.amd64.pdb
f:\binaries.amd64ret\lib\amd64\libcmt.amd64.pdb
...


I'm attaching a processmonitor log which includes two compilations. The first with the new (broken) clang, one with the older clang. The first one shows no pdb accesses at all, though it's possible it's missing from the log because there is no f:\ disk. The second loads the pdb files fine from the location next to the .lib files.

ProcessMonitorLog-lld-link-bytecode_builtins_list_generator.exe-twice.PML
4.6 MB Download
Cc: r...@chromium.org zturner@chromium.org
Does lld try to laod more stuff now?
It looks like lld is trying to load the same amount of stuff as before, but something has broken in its file-find algorithm so it can no longer find the PDBs. It looks in their absolute path instead of looking in the directory which contains the .lib file.

Without the PDBs there will no symbols for any of the libcmt.lib (CRT) files.

I'm sure some clever work with junctions could make it work, but we don't want to do that :-)

Can someone who can repro bisect lld? It builds pretty quickly (much quicker than clang).
From `svn log -r340925:342523 https://nico@llvm.org/svn/llvm-project/lld` these look most related:

------------------------------------------------------------------------
r341228 | aganea | 2018-08-31 13:41:58 -0400 (Fri, 31 Aug 2018) | 12 lines

[DebugInfo] Common behavior for error types

Following D50807, and heading towards D50664, this intermediary change does the following:

1. Upgrade all custom Error types in llvm/trunk/lib/DebugInfo/ to use the new StringError behavior (D50807).
2. Implement std::is_error_code_enum and make_error_code() for DebugInfo error enumerations.
3. Rename GenericError -> PDBError (the file will be renamed in a subsequent commit)
4. Update custom error messages to follow the same formatting: (\w\s*)+\.
5. Keep generic "file not found" (ENOENT) errors as they are in PDB code. Previously, there used to be a custom enumeration for that purpose.
6. Remove a few extraneous LF in log() implementations. Printing LF is a responsability at a higher level, not at the error level.



------------------------------------------------------------------------
r341403 | mstorsjo | 2018-09-04 16:56:56 -0400 (Tue, 04 Sep 2018) | 10 lines

[COFF] Allow exporting all symbols from system libraries specfied with -wholearchive:

When building a shared libc++.dll, it pulls in libc++abi.a statically
with the --wholearchive flag. If such a build is done with
--export-all-symbols, it's reasonable to assume that everything
from that library also should be exported with the same rules as normal
local object files, even though we normally avoid autoexporting things
from libc++abi.a in other cases when linking a DLL (user code).



------------------------------------------------------------------------
r341825 | aganea | 2018-09-10 09:51:21 -0400 (Mon, 10 Sep 2018) | 9 lines

[LLD][COFF] Cleanup error messages / add more coverage tests

- Log the reason for a PDB or precompiled-OBJ load failure
- Properly handle out-of-date PDB or precompiled-OBJ signature by displaying a corresponding error
- Slightly change behavior on PDB failure: any subsequent load attempt from another OBJ would result in the same error message being logged
- Slightly change behavior on PDB failure: retry with filename only if previous error was ENOENT ("no such file or directory")
- Tests: a. for native PDB errors; b. cover all the cases above

Differential Revision: https://reviews.llvm.org/D51559




Maaaaaybe

------------------------------------------------------------------------
r342003 | zturner | 2018-09-11 18:35:01 -0400 (Tue, 11 Sep 2018) | 1 line

[PDB] Write FPO Data to the PDB.

------------------------------------------------------------------------
r342080 | zturner | 2018-09-12 17:02:01 -0400 (Wed, 12 Sep 2018) | 26 lines

[PDB] Emit old fpo data to the PDB file.


(seem unlikely from the patch descriptions; didn't look at the patch)





A self-contained minimal repro that shows the problem would be great :-)
r341228 / r341825 sound like a reasonable theory from a distance. The system libs have a non-existent drive path for pdbs, and apparently we don't get ENOENT for that but "resource unavailable try again".

I pinged the author.
The computer has a card reader which seems to have hijacked F (and g, h, i) for itself, so somewhere in the system accessing it probably triggers a request to insert a floppy (well, sd card). 
Summary: Cannot link release binaries (dll/exe) on Windows after clang roll (when the F disk on the computer is not a normal disk) (was: Cannot link release binaries (dll/exe) on Windows after clang roll (DEPOT_TOOLS_WIN_TOOLCHAIN=0 ))
Disconnecting F from the card reader (leaving it unused), and linking works fine. So that is what triggers the problems, and it explains why suspiciously few saw the problem.
Nice debugging. I've tried to simulate that environment by using the subst command to point f:\ at a directory on my C: drive and to a thumb drive but my links keep stubbornly working. But, with luck the bug can be fixed using this information.

I just checked the PDB references in libcmt.pdb in the official toolchain and they are essentially the same - f:\binaries.amd64ret\lib\amd64\libcmt.pdb instead of f:\binaries.amd64ret\lib\amd64\libcmt.amd64.pdb (i.e.; the latest versions include .amd64 in the middle of the file name).

I'm the author of the patch which caused the issue (see here: https://reviews.llvm.org/D51559 - point 4.)

I'm attempting a fix: https://reviews.llvm.org/D52666 - could you possibly apply it locally over 34893e97f49e129ceb545f4edb3758e4b1bee1e5 and try again, with the same initial conditions?
I've not been able to build clang in Windows yet, but I can test an lld if someone else can build it and make it available.

Comment 18 Deleted

Patched lld-link.exe r342523 download (in zip archive):

https://drive.google.com/open?id=1VjomNAXXijiDCyVG7sIzC5Ch-3ErkmmA

FYI, it is a release build done with VS2017, assuming it matters.
Thanks. I tried it and it worked. It did link, no errors and I got a pdb generated.
Fix landed upstream in r343366. Chromium is currently on r343342.
Blockedon: 891181
Status: ExternalDependency (was: Unconfirmed)
The fix will come in with the next roll.

Sign in to add a comment