New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 674363 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug



Sign in to add a comment

All windows bots failing all builds: ensure git tooling on windows

Project Member Reported by kainino@chromium.org, Dec 15 2016

Issue description

ALL of the windows tryservers are failing ALL builds on a git issue:

https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/350385
https://build.chromium.org/p/tryserver.chromium.win/builders/win_optional_gpu_tests_rel/builds/6033
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_x64_rel_ng/builds/334991
https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_compile_dbg_ng/builds/315190
https://build.chromium.org/p/tryserver.chromium.win/builders/win_clang/builds/139096

INFO:root:running ['E:\\b\\rr\\tmp21sx3f\\rw\\checkout\\scripts\\slave\\.recipe_deps\\depot_tools\\cipd.bat', 'ensure', '-list', 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpt7qmpj', '-root', 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpizx4tr', '-verbose']
Exception calling "Invoke" with "0" argument(s): "A parameter cannot be found t
hat matches parameter name 'First'."
At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps
1:11 char:29
+     return $defaultFn.Invoke <<<< ()
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : CatchFromBaseAdapterMethodInvoke
 
Bootstrapping cipd client for windows-386...
From https://chrome-infra-packages.appspot.com/client?platform=windows-386&vers
ion=
Exception calling "DownloadFile" with "2" argument(s): "The remote server retur
ned an error: (400) Bad Request."
At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps
1:47 char:21
+     $wc.DownloadFile <<<< ($url, $client)
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException
 
The term 'E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\
.cipd_client.exe' is not recognized as the name of a cmdlet, function, script f
ile, or operable program. Check the spelling of the name, or if a path was incl
uded, verify that the path is correct and try again.
At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps
1:50 char:7
+ $_ = & <<<<  $client selfupdate -version "$cipdClientVer"
    + CategoryInfo          : ObjectNotFound: (E:\b\rr\tmp21sx...cipd_client.e 
   xe:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
 
The term 'E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\
.cipd_client.exe' is not recognized as the name of a cmdlet, function, script f
ile, or operable program. Check the spelling of the name, or if a path was incl
uded, verify that the path is correct and try again.
At E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\cipd.ps
1:56 char:2
+ & <<<<  $client @args
    + CategoryInfo          : ObjectNotFound: (E:\b\rr\tmp21sx...cipd_client.e 
   xe:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
 
INFO:root:running 'c:\\users\\chrome~2\\appdata\\local\\temp\\tmpizx4tr\\git-installer.exe -y -InstallPath="E:\\\\b\\\\rr\\\\tmp21sx3f\\\\rw\\\\checkout\\\\scripts\\\\slave\\\\.recipe_deps\\\\depot_tools\\\\git-2.10.0-64_bin" -Directory="E:\\\\b\\\\rr\\\\tmp21sx3f\\\\rw\\\\checkout\\\\scripts\\\\slave\\\\.recipe_deps\\\\depot_tools\\\\git-2.10.0-64_bin"'
Traceback (most recent call last):
  File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 193, in <module>
    sys.exit(main(sys.argv[1:]))
  File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 185, in main
    install_git(args, git_version, git_directory)
  File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 115, in install_git
    '-Directory="%s"' % git_directory.replace('\\', '\\\\'),
  File "E:\b\rr\tmp21sx3f\rw\checkout\scripts\slave\.recipe_deps\depot_tools\bootstrap\win\git_bootstrap.py", line 26, in _check_call
    subprocess.check_call(argv, **kwargs)
  File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 535, in check_call
    retcode = call(*popenargs, **kwargs)
  File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 709, in __init__
    errread, errwrite)
  File "E:\b\depot_tools\python276_bin\lib\subprocess.py", line 957, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
 
Components: Infra>Platform>CIPD
Or a CIPD error.

Comment 2 by tkent@chromium.org, Dec 15 2016

Cc: iannucci@chromium.org

Comment 3 by iannu...@google.com, Dec 15 2016

Rolling back CL.

Comment 4 by iannu...@google.com, Dec 15 2016

Owner: iannucci@chromium.org
Status: Assigned (was: Untriaged)
Actually, I'm not sure if rolling back is safe (since the CL was intended to fix a different unsafe issue). 

Comment 5 by iannu...@google.com, Dec 15 2016

I reverted anyway; extra downloads in update_scripts is better than these failures.

Comment 7 by iannu...@google.com, Dec 15 2016

This should take effect... annny minute now...

Comment 8 by iannu...@google.com, Dec 15 2016

Bleh. cipd bootstrap had powershell 2.0 incompatibility. CL to fix: https://chromium-review.googlesource.com/c/420408/

Comment 9 by iannu...@google.com, Dec 15 2016

(Revert fixed the immediate problem, though the bots are still in a non-critical-right-now-funny-state, which that reverted CL was intended to fix)
Project Member

Comment 10 by bugdroid1@chromium.org, Dec 15 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/229ac2bb535ca62b00a91778e07833175f0b761a

commit 229ac2bb535ca62b00a91778e07833175f0b761a
Author: Robbie Iannucci <iannucci@chromium.org>
Date: Thu Dec 15 02:49:42 2016

Revert "Revert "Stop adhoc cipd client bootstrap.""

This reverts commit 70f351e7843cb689bf9f3520d4d6183b138591f2.

Reason for revert: CIPD bootstrap is now PS 2.0 compatible.

BUG= 674363 ,673940

Original change's description:
> Revert "Stop adhoc cipd client bootstrap."
> 
> This reverts commit c6569aab121ed8e2859438fddf03dda81831252b.
> 
> Reason for revert: This caused  crbug.com/674363 , needs further investigation.
> 
> Original change's description:
> > Stop adhoc cipd client bootstrap.
> > 
> > This was used before to get the git installer, but now depot_tools manages its
> > own copy of CIPD.
> > 
> > BUG=673940
> > 
> > Change-Id: I1077ff7eb27d9ca191dc0092b7cacaaa69c40449
> > Reviewed-on: https://chromium-review.googlesource.com/420303
> > Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> > Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
> > 
> 
> TBR=iannucci@chromium.org,agable@chromium.org,vadimsh@chromium.org,dnj@chromium.org,sebmarchand@chromium.org,chromium-reviews@chromium.org
> BUG=673940
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> 
> Change-Id: I7aa5561ad0df67831987cd8d53a84105d42d9663
> Reviewed-on: https://chromium-review.googlesource.com/420320
> Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> 

TBR=iannucci@chromium.org,agable@chromium.org,vadimsh@chromium.org,dnj@chromium.org,chromium-reviews@chromium.org,sebmarchand@chromium.org
BUG=673940
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Change-Id: I822312b1507003ee07930ef7527e12b997457673
Reviewed-on: https://chromium-review.googlesource.com/420321
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/229ac2bb535ca62b00a91778e07833175f0b761a/bootstrap/win/git_bootstrap.py

Oh, here was the PS2.0 compatibility CL: be3daff577bbe9518b34989ebb5147216d35e60b
Status: Fixed (was: Assigned)
The PS2.0 compat cl + revert-revert was picked up in https://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_ng/builds/350503 and looks good.

I'll include this bug in the postmortem for bug 673940
Still seeing errors in luci/swarming builds, e.g. https://luci-milo.appspot.com/swarming/task/33196b9378090610
Looks like we need to bump depot_tools pin in infra.git recipes, because recipe autoroller is stuck: it can't finish windows try job (due to this exact bug issue). 

I'll do that now.

Sign in to add a comment