All windows bots failing all builds: ensure git tooling on windows |
||||
Issue descriptionALL of the windows tryservers are failing ALL builds on a git issue: https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/350385 https://build.chromium.org/p/tryserver.chromium.win/builders/win_optional_gpu_tests_rel/builds/6033 https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/334991 https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_compile_dbg_ng/builds/315190 https://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/139096 INFO:root:running ['E:\\b\\rr\\tmp21sx3f\\rw\\checkout\\scripts\\slave\\.recipe_deps\\depot_tools\\cipd.bat', 'ensure', '-list', 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpt7qmpj', '-root', 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpizx4tr', '-verbose'] Exception calling "Invoke" with "0" argument(s): "A parameter cannot be found t hat matches parameter name 'First'." At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps 1:11 char:29 + return $defaultFn.Invoke <<<< () + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : CatchFromBaseAdapterMethodInvoke Bootstrapping cipd client for windows-386... From https://chrome-infra-packages.appspot.com/client?platform=windows-386&vers ion= Exception calling "DownloadFile" with "2" argument(s): "The remote server retur ned an error: (400) Bad Request." At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps 1:47 char:21 + $wc.DownloadFile <<<< ($url, $client) + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : DotNetMethodException The term 'E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\ .cipd_client.exe' is not recognized as the name of a cmdlet, function, script f ile, or operable program. Check the spelling of the name, or if a path was incl uded, verify that the path is correct and try again. At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps 1:50 char:7 + $_ = & <<<< $client selfupdate -version "$cipdClientVer" + CategoryInfo : ObjectNotFound: (E:\b\rr\tmp21sx...cipd_client.e xe:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException The term 'E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\ .cipd_client.exe' is not recognized as the name of a cmdlet, function, script f ile, or operable program. Check the spelling of the name, or if a path was incl uded, verify that the path is correct and try again. At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps 1:56 char:2 + & <<<< $client @args + CategoryInfo : ObjectNotFound: (E:\b\rr\tmp21sx...cipd_client.e xe:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException INFO:root:running 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpizx4tr\\git-installer.exe -y -InstallPath="E:\\\\b\\\\rr\\\\tmp21sx3f\\\\rw\\\\checkout\\\\scripts\\\\slave\\\\.recipe_deps\\\\depot_tools\\\\git-2.10.0-64_bin" -Directory="E:\\\\b\\\\rr\\\\tmp21sx3f\\\\rw\\\\checkout\\\\scripts\\\\slave\\\\.recipe_deps\\\\depot_tools\\\\git-2.10.0-64_bin"' Traceback (most recent call last): File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 193, in <module> sys.exit(main(sys.argv[1:])) File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 185, in main install_git(args, git_version, git_directory) File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 115, in install_git '-Directory="%s"' % git_directory.replace('\\', '\\\\'), File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 26, in _check_call subprocess.check_call(argv, **kwargs) File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 535, in check_call retcode = call(*popenargs, **kwargs) File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 522, in call return Popen(*popenargs, **kwargs).wait() File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 709, in __init__ errread, errwrite) File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 957, in _execute_child startupinfo) WindowsError: [Error 2] The system cannot find the file specified
,
Dec 15 2016
,
Dec 15 2016
Rolling back CL.
,
Dec 15 2016
Actually, I'm not sure if rolling back is safe (since the CL was intended to fix a different unsafe issue).
,
Dec 15 2016
I reverted anyway; extra downloads in update_scripts is better than these failures.
,
Dec 15 2016
Revert is here: https://chromium-review.googlesource.com/c/420320/
,
Dec 15 2016
This should take effect... annny minute now...
,
Dec 15 2016
Bleh. cipd bootstrap had powershell 2.0 incompatibility. CL to fix: https://chromium-review.googlesource.com/c/420408/
,
Dec 15 2016
(Revert fixed the immediate problem, though the bots are still in a non-critical-right-now-funny-state, which that reverted CL was intended to fix)
,
Dec 15 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/229ac2bb535ca62b00a91778e07833175f0b761a commit 229ac2bb535ca62b00a91778e07833175f0b761a Author: Robbie Iannucci <iannucci@chromium.org> Date: Thu Dec 15 02:49:42 2016 Revert "Revert "Stop adhoc cipd client bootstrap."" This reverts commit 70f351e7843cb689bf9f3520d4d6183b138591f2. Reason for revert: CIPD bootstrap is now PS 2.0 compatible. BUG= 674363 ,673940 Original change's description: > Revert "Stop adhoc cipd client bootstrap." > > This reverts commit c6569aab121ed8e2859438fddf03dda81831252b. > > Reason for revert: This caused crbug.com/674363 , needs further investigation. > > Original change's description: > > Stop adhoc cipd client bootstrap. > > > > This was used before to get the git installer, but now depot_tools manages its > > own copy of CIPD. > > > > BUG=673940 > > > > Change-Id: I1077ff7eb27d9ca191dc0092b7cacaaa69c40449 > > Reviewed-on: https://chromium-review.googlesource.com/420303 > > Commit-Queue: Robbie Iannucci <iannucci@chromium.org> > > Reviewed-by: Aaron Gable <agable@chromium.org> > > Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> > > > > TBR=iannucci@chromium.org,agable@chromium.org,vadimsh@chromium.org,dnj@chromium.org,sebmarchand@chromium.org,chromium-reviews@chromium.org > BUG=673940 > NOPRESUBMIT=true > NOTREECHECKS=true > NOTRY=true > > Change-Id: I7aa5561ad0df67831987cd8d53a84105d42d9663 > Reviewed-on: https://chromium-review.googlesource.com/420320 > Commit-Queue: Robbie Iannucci <iannucci@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> > TBR=iannucci@chromium.org,agable@chromium.org,vadimsh@chromium.org,dnj@chromium.org,chromium-reviews@chromium.org,sebmarchand@chromium.org BUG=673940 NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Change-Id: I822312b1507003ee07930ef7527e12b997457673 Reviewed-on: https://chromium-review.googlesource.com/420321 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/229ac2bb535ca62b00a91778e07833175f0b761a/bootstrap/win/git_bootstrap.py
,
Dec 15 2016
Oh, here was the PS2.0 compatibility CL: be3daff577bbe9518b34989ebb5147216d35e60b
,
Dec 15 2016
The PS2.0 compat cl + revert-revert was picked up in https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/350503 and looks good. I'll include this bug in the postmortem for bug 673940
,
Dec 15 2016
Still seeing errors in luci/swarming builds, e.g. https://luci-milo.appspot.com/swarming/task/33196b9378090610
,
Dec 15 2016
Looks like we need to bump depot_tools pin in infra.git recipes, because recipe autoroller is stuck: it can't finish windows try job (due to this exact bug issue). I'll do that now.
,
Dec 15 2016
The CL (had to land manually): https://chromium.googlesource.com/infra/infra/+/af008f315c7cfebf5dc364818c34311883a77095 Looks like it helped: https://luci-milo.appspot.com/swarming/task/3319a9709ce10d10 |
||||
►
Sign in to add a comment |
||||
Comment 1 by kainino@chromium.org
, Dec 15 2016