The remaining output, minus timestamps.

----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "virtualenv.py", line 2328, in <module>
    main()
  File "virtualenv.py", line 713, in main
    symlink=options.symlink)
  File "virtualenv.py", line 945, in create_environment
    download=download,
  File "virtualenv.py", line 901, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "virtualenv.py", line 797, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/user/.vpython-root/929139/bin/python - setuptools pip wheel failed with error code 1
original error: exit status 1

goroutine 1:
#0 go.chromium.org/luci/vpython/venv/venv.go:577 - venv.(*Env).installVirtualEnv()
  reason: failed to create VirtualEnv
#1 go.chromium.org/luci/vpython/venv/venv.go:498 - venv.(*Env).createLocked.func1()
  reason: failed to install VirtualEnv

#2 go.chromium.org/luci/common/system/filesystem/tempdir.go:55 - filesystem.(*TempDir).With()
#3 go.chromium.org/luci/vpython/venv/venv.go:75 - venv.withTempDir()
#4 go.chromium.org/luci/vpython/venv/venv.go:474 - venv.(*Env).createLocked()
#5 go.chromium.org/luci/vpython/venv/venv.go:243 - venv.(*Env).ensure.func1()
  reason: failed to create new VirtualEnv

#6 go.chromium.org/luci/vpython/venv/venv.go:841 - venv.mustReleaseLock()
#7 go.chromium.org/luci/vpython/venv/venv.go:229 - venv.(*Env).ensure()
#8 go.chromium.org/luci/vpython/venv/venv.go:126 - venv.With()
  reason: failed to create empty probe environment

#9 go.chromium.org/luci/vpython/run.go:72 - vpython.Run()
#10 go.chromium.org/luci/vpython/application/application.go:261 - application.(*application).mainImpl()
#11 go.chromium.org/luci/vpython/application/application.go:349 - application.(*Config).Main.func1()
#12 go.chromium.org/luci/vpython/application/support.go:45 - application.run()
#13 go.chromium.org/luci/vpython/application/application.go:348 - application.(*Config).Main()
#14 vpython/main.go:92 - main.mainImpl()
#15 vpython/main.go:98 - main.main()
#16 runtime/proc.go:185 - runtime.main()
#17 runtime/asm_amd64.s:2337 - runtime.goexit()
]

The issue seems to be out of TE-scope as it is related to infra and vpython. Hence, adding label TE-NeedsTriageHelp for further investigation from dev team.

Thanks...!!

Yeah, this is an infrastructure tool that broke. I'll remove all these non-infra labels and move it to the right queue.

Same problem, still. Tried pdfium today.

$ download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s pdfium/buildtools/linux64/gn.sha1
0> Failed to fetch file gs://chromium-gn/b19596a0f7a5f849eca9d47da557020159db2021 for pdfium/buildtools/linux64/gn, skipping.
Process output:
New python executable in /home/user/.vpython-root/e2f97c/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /home/user/.vpython-root/e2f97c/bin/python - setuptools pip wheel:
  Collecting setuptools
Collecting pip
  Could not find a version that satisfies the requirement pip (from versions: )
No matching distribution found for pip
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "virtualenv.py", line 2328, in <module>
    main()
  File "virtualenv.py", line 713, in main
    symlink=options.symlink)
  File "virtualenv.py", line 945, in create_environment
    download=download,
  File "virtualenv.py", line 901, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "virtualenv.py", line 797, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/user/.vpython-root/e2f97c/bin/python - setuptools pip wheel failed with error code 1

Please try running "vpython" directly and see if you get a similar error message. If you do, run "vpython -log-level debug" and paste the output.

[options.go:155] Unable to resolve specification path. Using default specification.
[cipd.go:97] Resolving CIPD packages in root [/tmp/vpython_cipd299553567]:
[cipd.go:105]     Unresolved package: {infra/python/virtualenv version:15.1.0 %!s(int=0)}
[client.go:834] cipd: tag cache hit for infra/python/virtualenv:version:15.1.0 - d7fe4ca0377e894777e4f0a4d10defc600baf537
[cipd.go:128] Resolved package to: infra/python/virtualenv:d7fe4ca0377e894777e4f0a4d10defc600baf537 {"package":"infra/python/virtualenv", "version":"version:15.1.0"}
[probe.go:219] Candidate directory does not exist [/opt/intel/compilers_and_libraries_2016.2.181/linux/mpirt/bin/intel64_lin]; skipping...
[probe.go:219] Candidate directory does not exist [/usr/local/sbin]; skipping...
[probe.go:87] Detected Python version "2.7.12" from probe candidate [/usr/local/bin/python]
[config.go:192] Resolved system Python runtime (2.7.12 @ da1b9cff1f7aed83174c29df280aaec778b408cf52ffc85314627dc8e7dce122): /usr/local/bin/python
[venv.go:238] VirtualEnv is not complete.                 {"error":"environment is not complete"}
[venv.go:442] Deleting existing VirtualEnv: /home/user/.vpython-root/e2f97c
[venv.go:452] Using virtual environment root: /home/user/.vpython-root/e2f97c
[venv.go:538] Loading 1 package(s) into: /tmp/vpython_bootstrap611220978/packages
[client.go:280] In root:
[client.go:286]   to install:
[client.go:288]     infra/python/virtualenv:d7fe4ca0377e894777e4f0a4d10defc600baf537
[client.go:688] cipd: using instance cache at "/home/user/.vpython_cipd_cache/instances"
[client.go:1248] cipd: instance cache hit for infra/python/virtualenv:d7fe4ca0377e894777e4f0a4d10defc600baf537
[deployer.go:168] Deploying infra/python/virtualenv:d7fe4ca0377e894777e4f0a4d10defc600baf537 into /tmp/vpython_bootstrap611220978/packages(/)
[reader.go:297] cipd: about to extract 2.0 Mb (46 files)
[reader.go:336] cipd: extracting - 0%
[reader.go:336] cipd: extracting - 100%
[deployer.go:298] Successfully deployed infra/python/virtualenv:d7fe4ca0377e894777e4f0a4d10defc600baf537
[client.go:1435] All changes applied.
[venv.go:563] Creating VirtualEnv at: /home/user/.vpython-root/e2f97c
[venv.go:844] Running Python command (cwd=/tmp/vpython_bootstrap611220978/packages/virtualenv-15.1.0): /usr/local/bin/python -B -E -s virtualenv.py --no-download /home/user/.vpython-root/e2f97c
New python executable in /home/user/.vpython-root/e2f97c/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /home/user/.vpython-root/e2f97c/bin/python - setuptools pip wheel:
  Collecting setuptools
Collecting pip
  Could not find a version that satisfies the requirement pip (from versions: )
No matching distribution found for pip
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "virtualenv.py", line 2328, in <module>
    main()
  File "virtualenv.py", line 713, in main
    symlink=options.symlink)
  File "virtualenv.py", line 945, in create_environment
    download=download,
  File "virtualenv.py", line 901, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "virtualenv.py", line 797, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/user/.vpython-root/e2f97c/bin/python - setuptools pip wheel failed with error code 1
[annotate.go:242] original error: exit status 1
[annotate.go:242] 
[annotate.go:242] goroutine 1:
[annotate.go:242] #0 go.chromium.org/luci/vpython/venv/venv.go:574 - venv.(*Env).installVirtualEnv()
[annotate.go:242]   reason: failed to create VirtualEnv
[annotate.go:242] 
[annotate.go:242] #1 go.chromium.org/luci/vpython/venv/venv.go:490 - venv.(*Env).createLocked.func1()
[annotate.go:242]   reason: failed to install VirtualEnv
[annotate.go:242] 
[annotate.go:242] #2 go.chromium.org/luci/common/system/filesystem/tempdir.go:55 - filesystem.(*TempDir).With()
[annotate.go:242] #3 go.chromium.org/luci/vpython/venv/venv.go:72 - venv.withTempDir()
[annotate.go:242] #4 go.chromium.org/luci/vpython/venv/venv.go:466 - venv.(*Env).createLocked()
[annotate.go:242] #5 go.chromium.org/luci/vpython/venv/venv.go:241 - venv.(*Env).ensure.func1()
[annotate.go:242]   reason: failed to create new VirtualEnv
[annotate.go:242] 
[annotate.go:242] #6 go.chromium.org/luci/vpython/venv/venv.go:885 - venv.mustReleaseLock()
[annotate.go:242] #7 go.chromium.org/luci/vpython/venv/venv.go:227 - venv.(*Env).ensure()
[annotate.go:242] #8 go.chromium.org/luci/vpython/venv/venv.go:297 - venv.(*Env).withImpl()
[annotate.go:242] #9 go.chromium.org/luci/vpython/venv/venv.go:137 - venv.With()
[annotate.go:242] #10 go.chromium.org/luci/vpython/run.go:63 - vpython.Run()
[annotate.go:242] #11 go.chromium.org/luci/vpython/application/application.go:266 - application.(*application).mainImpl()
[annotate.go:242] #12 go.chromium.org/luci/vpython/application/application.go:354 - application.(*Config).Main.func1()
[annotate.go:242] #13 go.chromium.org/luci/vpython/application/support.go:47 - application.run()
[annotate.go:242] #14 go.chromium.org/luci/vpython/application/application.go:353 - application.(*Config).Main()
[annotate.go:242] #15 vpython/main.go:92 - main.mainImpl()
[annotate.go:242] #16 vpython/main.go:98 - main.main()
[annotate.go:242] #17 runtime/proc.go:185 - runtime.main()
[annotate.go:242] #18 runtime/asm_amd64.s:2337 - runtime.goexit()

​I think I found the problem. I have this in /etc/pip.conf.

no-binary = cffi,lxml,pip

Removing pip makes it progress further.

Collecting cffi==1.10.0 (from -r
/tmp/vpython_bootstrap153376911/requirements.txt (line 4))
  Could not find a version that satisfies the requirement cffi==1.10.0
(from -r /tmp/vpython_bootstrap153376911/requirements.txt (line 4)) (from
versions: )

Removing cffi then fixes it.

So a fix could be to install a local pip.conf, or to pass --only-binary for
each package.

https://pip.pypa.io/en/stable/user_guide/#config-file
https://pip.pypa.io/en/stable/user_guide/#config-precedence

Thanks for taking the time to track this down and including information/links. I was able to quickly reproduce this and have created a patch along the lines of your suggestion: https://chromium-review.googlesource.com/#/c/infra/luci/luci-go/+/866294

Once that lands and rolls into production, this particular problem should be fixed.

Thanks again for your help and reporting! Global pip config ... it's always something ><

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/b7bf5fecb3c0bec70979c4b577cfb31e15009002

commit b7bf5fecb3c0bec70979c4b577cfb31e15009002
Author: Dan Jacques <dnj@chromium.org>
Date: Sun Jan 14 19:11:33 2018

[vpython] Better pip isolation, fixups.

Add directives to better isolate "pip" from user/system configuration.
While "pip" cannot be fully isolated, this removes additional variables
and lays groundwork for quickly adding additional isolation if needed.

Log a helpful error message when something fails during VirtualEnv
setup.

Also, clean up Interpreter's command generation.

Bug: 786617
Test: local
Change-Id: I50960280a2968a7f4b71663dd9113939f857731f
- Ran with global "pip" config, reproduced issue and confirmed solution.
Reviewed-on: https://chromium-review.googlesource.com/866294
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/application/application.go
[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/python/interpreter.go
[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/python/interpreter_test.go
[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/venv/venv.go
[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/venv/venv_resources_test.go
[modify] https://crrev.com/b7bf5fecb3c0bec70979c4b577cfb31e15009002/vpython/venv/venv_test.go

​Verified!​

Thanks for following up with verification!