Win x64 Builder failure on chromium.perf |
|||||||||
Issue descriptionWin x64 Builder has failed the compilation step. https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%20x64%20Builder The error message seems to suggest some linking problem. But I see the main waterfall is entirely green. I am not sure what happens here. Is it a bot problem or some error is not caught by main waterfall? FAILED: load_library_perf_tests.exe load_library_perf_tests.exe.pdb C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-with-manifests environment.x64 True load_library_perf_tests.exe "C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:load_library_perf_tests.exe @load_library_perf_tests.exe.rsp" 1 mt.exe rc.exe "obj\chrome\load_library_perf_tests.load_library_perf_tests.exe.intermediate.manifest" obj\chrome\load_library_perf_tests.load_library_perf_tests.exe.generated.manifest ..\..\build\win\compatibility.manifest obj\content\content_browser.lib : fatal error LNK1107: invalid or corrupt file: cannot read at 0xEAE7739 Final: Total time = 1.997s Traceback (most recent call last): File "gyp-win-tool", line 323, in <module> sys.exit(main(sys.argv[1:])) File "gyp-win-tool", line 29, in main exit_code = executor.Dispatch(args) File "gyp-win-tool", line 71, in Dispatch return getattr(self, method)(*args[1:]) File "gyp-win-tool", line 179, in ExecLinkWithManifests subprocess.check_call(ldcmd + add_to_ld) File "C:\b\depot_tools\python276_bin\lib\subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:load_library_perf_tests.exe @load_library_perf_tests.exe.rsp load_library_perf_tests.exe.manifest.res' returned non-zero exit status 1107 [1604/1688] LINK_EMBED cc_perftests.exe LibDef: Total time = 0.000s OptRef: Total time = 0.078s OptRef: Total time = 0.015s OptIcf: Total time = 0.312s Pass 1: Interval #1, time = 118.046s Wait PDB close: Total time = 0.203s Wait type merge: Total time = 0.031s Pass 2: Interval #2, time = 0.780s Final: Total time = 118.826s [1605/1688] LINK_EMBED(DLL) chrome.dll FAILED: chrome.dll chrome.dll.lib chrome.dll.pdb C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-with-manifests environment.x64 True chrome.dll "C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /IMPLIB:chrome.dll.lib /DLL /OUT:chrome.dll @chrome.dll.rsp" 2 mt.exe rc.exe "obj\chrome\chrome_main_dll.chrome.dll.intermediate.manifest" obj\chrome\chrome_main_dll.chrome.dll.generated.manifest ..\..\chrome\app\chrome.dll.manifest obj\content\content_browser.lib : fatal error LNK1107: invalid or corrupt file: cannot read at 0x1E66FE64 Final: Total time = 51.652s Traceback (most recent call last): File "gyp-win-tool", line 323, in <module> sys.exit(main(sys.argv[1:])) File "gyp-win-tool", line 29, in main exit_code = executor.Dispatch(args) File "gyp-win-tool", line 71, in Dispatch return getattr(self, method)(*args[1:]) File "gyp-win-tool", line 179, in ExecLinkWithManifests subprocess.check_call(ldcmd + add_to_ld) File "C:\b\depot_tools\python276_bin\lib\subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /IMPLIB:chrome.dll.lib /DLL /OUT:chrome.dll @chrome.dll.rsp chrome.dll.manifest.res' returned non-zero exit status 1107 [1606/1688] LINK_EMBED performance_browser_tests.exe FAILED: performance_browser_tests.exe performance_browser_tests.exe.pdb C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-with-manifests environment.x64 True performance_browser_tests.exe "C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:performance_browser_tests.exe @performance_browser_tests.exe.rsp" 1 mt.exe rc.exe "obj\chrome\performance_browser_tests.performance_browser_tests.exe.intermediate.manifest" obj\chrome\performance_browser_tests.performance_browser_tests.exe.generated.manifest ..\..\build\win\compatibility.manifest obj\content\content_browser.lib : fatal error LNK1107: invalid or corrupt file: cannot read at 0x1E66FE64 Final: Total time = 198.745s Traceback (most recent call last): File "gyp-win-tool", line 323, in <module> sys.exit(main(sys.argv[1:])) File "gyp-win-tool", line 29, in main exit_code = executor.Dispatch(args) File "gyp-win-tool", line 71, in Dispatch return getattr(self, method)(*args[1:]) File "gyp-win-tool", line 179, in ExecLinkWithManifests subprocess.check_call(ldcmd + add_to_ld) File "C:\b\depot_tools\python276_bin\lib\subprocess.py", line 540, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'C:\b\depot_tools\python276_bin\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:performance_browser_tests.exe @performance_browser_tests.exe.rsp performance_browser_tests.exe.manifest.res' returned non-zero exit status 1107 [1607/1688] LINK_EMBED(DLL) chrome_child.dll LibDef: Total time = 0.000s OptRef: Total time = 0.343s OptRef: Total time = 0.156s OptIcf: Total time = 3.791s Pass 1: Interval #1, time = 1147.605s Wait PDB close: Total time = 1.107s Wait type merge: Total time = 0.032s Pass 2: Interval #2, time = 10.094s Final: Total time = 1157.699s ninja: build stopped: subcommand failed.
,
Jun 3 2016
The clobber build didn't seem to help. Here's the blamelist at the point when it turned red for reference in case we lose history: https://chromium.googlesource.com/chromium/src/+log/4680358215ff217eb6b0073c4be2243738369957%5E..b4c497067ca404764c5d67df607e894d3118bfae?pretty=fuller
,
Jun 5 2016
,
Jun 6 2016
Only one of the CLs in that range looks like it could have anything to do with this, and it's https://chromium.googlesource.com/chromium/src/+/ffb89045a543e91c09f2cf2c168a823e7b943f27%5E%21/#F0. I'm not familiar with the files being changed to know if it could be related. Anyone have an idea? Something could have changed in build that wouldn't be caught in the blamelist, too. Since both slaves in the builder pool are seeing the same issue it doesn't appear to be machine-specific, though.
,
Jun 6 2016
This is keep failing.
,
Jun 6 2016
,
Jun 6 2016
+sebmarchand, could your CL cause a breakage? See comment 4
,
Jun 6 2016
It's similar to 599186, we need to use msvs_shard on the content_browser target... I'm on it.
,
Jun 6 2016
,
Jun 6 2016
Do we have an ETA on the fix for this bug? It's a pretty huge one.
,
Jun 6 2016
I was waiting for an owner approval on https://codereview.chromium.org/2046573003/ , I'll TBR it.
,
Jun 6 2016
I've also opened crbug.com/617723 to avoid this kind of issue in the future.
,
Jun 6 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c9dd766b9c61d5a4186ce8d3c1bbf830ad27b2f2 commit c9dd766b9c61d5a4186ce8d3c1bbf830ad27b2f2 Author: sebmarchand <sebmarchand@chromium.org> Date: Mon Jun 06 20:34:13 2016 Shard content_browser to avoid MSVS 2 GB limitation. A change to enable WPO for the x64 Win official builds caused content_browserto expand beyond 2 GB which caused these errors: obj\content\content_browser.lib : fatal error LNK1107: invalid or corrupt file: cannot read at 0xEAE7739 msvs_shard is a way of breaking up a library to avoid this limitation. This issue doesn't affect the GN builds (because they use a sourceset concept instead of these huge static libraries). TBR=creis@chromium.org BUG= 616946 Review-Url: https://codereview.chromium.org/2046573003 Cr-Commit-Position: refs/heads/master@{#398112} [modify] https://crrev.com/c9dd766b9c61d5a4186ce8d3c1bbf830ad27b2f2/content/content.gyp
,
Jun 7 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2e304ec9ca19c8113db6b49405d6b2dd6d8cf2cd commit 2e304ec9ca19c8113db6b49405d6b2dd6d8cf2cd Author: sebmarchand <sebmarchand@chromium.org> Date: Tue Jun 07 12:41:48 2016 Revert of Use /LTCG:Incremental to speed up the non-clobber official builds. (patchset #2 id:20001 of https://codereview.chromium.org/2045683002/ ) Reason for revert: I'm seeing some compiler issue on the Win perf builders (https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%20x64%20Builder/builds/9316/steps/compile/logs/stdio) : fatal error C1001: An internal error has occurred in the compiler BUG= 616946 Original issue's description: > Use /LTCG:Incremental to speed up the non-clobber official builds. > > /LTCG:Incremental is different from /INCREMENTAL, it speeds-up the link time of the WPO build without affecting the code quality and the performance, see https://blogs.msdn.microsoft.com/vcblog/2014/11/12/speeding-up-the-incremental-developer-build-scenario/ for more details on this. > > Committed: https://crrev.com/98fccebb0a5f6f1307ed1e598418fa472a8e1800 > Cr-Commit-Position: refs/heads/master@{#398153} TBR=brucedawson@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review-Url: https://codereview.chromium.org/2035313007 Cr-Commit-Position: refs/heads/master@{#398279} [modify] https://crrev.com/2e304ec9ca19c8113db6b49405d6b2dd6d8cf2cd/build/common.gypi [modify] https://crrev.com/2e304ec9ca19c8113db6b49405d6b2dd6d8cf2cd/build/config/compiler/BUILD.gn
,
Jun 7 2016
This bug should be fixed now, the problem that we have now is that these bots take a long time to build, I'll start a new thread to discuss this.
,
Jun 7 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/b4d6cba233fd3e2e1a7b70f5f0f5387ebf0039d1 commit b4d6cba233fd3e2e1a7b70f5f0f5387ebf0039d1 Author: Sebastien Marchand <sebmarchand@chromium.org> Date: Tue Jun 07 17:59:55 2016 Revert of Use /LTCG:Incremental to speed up the non-clobber official builds. (patchset #2 id:20001 of https://codereview.chromium.org/2045683002/ ) Reason for revert: I'm seeing some compiler issue on the Win perf builders (https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%20x64%20Builder/builds/9316/steps/compile/logs/stdio) : fatal error C1001: An internal error has occurred in the compiler BUG= 616946 Original issue's description: > Use /LTCG:Incremental to speed up the non-clobber official builds. > > /LTCG:Incremental is different from /INCREMENTAL, it speeds-up the link time of the WPO build without affecting the code quality and the performance, see https://blogs.msdn.microsoft.com/vcblog/2014/11/12/speeding-up-the-incremental-developer-build-scenario/ for more details on this. > > Committed: https://crrev.com/98fccebb0a5f6f1307ed1e598418fa472a8e1800 > Cr-Commit-Position: refs/heads/master@{#398153} TBR=brucedawson@chromium.org NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review-Url: https://codereview.chromium.org/2035313007 Cr-Commit-Position: refs/heads/master@{#398279} (cherry picked from commit 2e304ec9ca19c8113db6b49405d6b2dd6d8cf2cd) Review URL: https://codereview.chromium.org/2044943002 . Cr-Commit-Position: refs/branch-heads/2761@{#5} Cr-Branched-From: afd97ff8f20acb90a6b495776e385411371651f0-refs/heads/master@{#398174} [modify] https://crrev.com/b4d6cba233fd3e2e1a7b70f5f0f5387ebf0039d1/build/common.gypi [modify] https://crrev.com/b4d6cba233fd3e2e1a7b70f5f0f5387ebf0039d1/build/config/compiler/BUILD.gn |
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by eakuefner@chromium.org
, Jun 3 2016