New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 771277 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Oct 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

gclient runhooks in V8 failed on Gentoo Linux

Project Member Reported by tzik@chromium.org, Oct 3 2017

Issue description

Chrome Version: ToT
OS: Gentoo Linux

What steps will reproduce the problem?
(1) Run `fetch --no-history v8` on Gentoo Linux.

What is the expected result?
V8 sources should be fetched successfully.

What happens instead?
Failed to fetch gn binary when running hooks. The error message was:
> ________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-gn -s v8/buildtools/linux64/gn.sha1' in '/home/tzik/toolchain/pkg/v8'
> Failed to fetch file gs://chromium-gn/15cc6942069e0463e11752fcdf2244dbeec78a43 for v8/buildtools/linux64/gn. [Err: ]

Please use labels and text to provide additional information.

download_from_google_storage uses gsutil.py in depot_tools, and gsutil.py fails to run silently returning 127 as the exit value.
The failure seems to come from vpython. Running vpython in depot_tools dir with a debug option, it emits:
> vpython -log-level debug -vpython-spec gsutil.vpython -- external_bin/gsutil/gsutil_4.13/gsutil/gsutil

> [D2017-10-04T03:36:18.516216+09:00 59059 0 cipd.go:112] Resolving CIPD packages in root [/tmp/vpython_cipd267732909]:
(snip)
> [D2017-10-04T03:36:18.546214+09:00 59059 0 venv.go:554] Creating VirtualEnv at: /home/tzik/.vpython-root/edde42
> [I2017-10-04T03:36:18.546245+09:00 59059 0 venv.go:765] Running Python command (cwd=/tmp/vpython_bootstrap867157800/packages/virtualenv-15.1.0): /usr/bin/python-exec2c -B -E -s virtualenv.py --no-download /home/tzik/.vpython-root/edde42
> -B: unable to find executable in PATH.

Where, /usr/bin/python-exec2c is wrong. It should be just "python", but vpython [1] resolves it to python-exec2c, which is the symlink target of /usr/bin/python in Gentoo.
As python-exec2c changes its behavior depending on argv[0], the symlink resolution probably doesn't work as intended in this case.

[1] https://cs.chromium.org/chromium/infra/go/src/go.chromium.org/luci/vpython/python/interpreter.go?sq=package:chromium&dr=CSs&l=57

For graphics-related bugs, please copy/paste the contents of the about:gpu
page at the end of this report.

 

Comment 1 by tzik@chromium.org, Oct 3 2017

A workaround for this is prepending /usr/lib/python-exec/python2.7 to PATH, so that vpython can find the interpreter binary directly without python-exec2c, per a discussion on https://chromium-review.googlesource.com/c/infra/luci/luci-go/+/697350

Comment 2 by tzik@chromium.org, Oct 4 2017

Components: Infra>Client>V8 Infra>SDK
Unfortunately I don't have a Gentoo Linux machine to test this, but since you've spent considerable effort debugging this, do you have a proposed solution? Can you submit a CL that would fix it on Gentoo?

To make changes to vpython in LUCI repo:
  git clone https://github.com/luci/luci-go
  cd luci-go/vpython/python/
  ... hack ...
  git commit -a
  git cl upload

To make to depot_tools:
  fetch infra
  cd infra/depot_tools
  ... hack ...
  git commit -a
  git cl upload
I assume this is not a V8-specific problem at all, but would fail when fetching any other repos too, right? E.g. fetch chromium.
Cc: iannucci@chromium.org
Components: -Infra>Client>V8
Agreed, not V8-specific. Infra>SDK is a good enough component for this.

Comment 6 by tzik@chromium.org, Oct 4 2017

It potentially matters to Chromium, but currently not failing there, as chromium uses pinned depot_tools that is not yet broken.
Project Member

Comment 7 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/c874f8ac98a7f1e2065922fc3663573145498c39

commit c874f8ac98a7f1e2065922fc3663573145498c39
Author: tzik <tzik@chromium.org>
Date: Wed Oct 04 17:25:29 2017

Remove symlink evaluation on the path normalization of python interpreter

/usr/bin/python on Gentoo is a symlink to python-exec2c, and it changes
its behavior depending on argv[0]. So, resolving the symlink here causes
failure on the python invocation.

Bug:  771277 
Change-Id: Ifeda73780f2d353c9702d008f1f3231339caad54
Reviewed-on: https://chromium-review.googlesource.com/697350
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/c874f8ac98a7f1e2065922fc3663573145498c39/vpython/python/interpreter.go

Project Member

Comment 8 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb

commit bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb
Author: Dan Jacques <dnj@chromium.org>
Date: Wed Oct 04 17:45:37 2017

Roll luci-go.

infra/go/src/go.chromium.org/luci:
c874f8ac Remove symlink evaluation on the path normalization of python interpreter
d6879e33 [buildbucket] add an idiomatic Build type
677234be [logdog] Improve Makefile a bit.
a11820b6 [logdog] Actually allow PubSub to hit /auth/pubsub/* on backend module.
04a6c8e6 scheduler: Replace task.Trigger with proto message.
8456783d [logdog] Route AuthDB change PubSub notifications to "backend" module.
8b7cf7bc [milo] fix build query limit
e9702ca2 [milo] Add "extra" field to Console to absorb new properties.
e27c01ed scheduler: Teach noop tasks to emit triggers.
09633ae8 scheduler: Use snake_case for proto fields in internal/*.proto.
28d4e01a scheduler: Implement 'ForceInvocation' using v2 data structures.
d8f1c7fb [common|gitiles] Add protoized form of Git log data.
d24a7328 [milo] Change /console/<project> to show console previews.
e1679a46 [milo] exclude PendingBuildStates from response
5e968a5c tq: Add a testing helper.
79f513e2 [milo] Buildbot.GetBuildbotBuildsJSON: remove cursor

TBR=iannucci@chromium.org
BUG= chromium:771277 
TEST=None

Change-Id: Id8f864d7d0e9382a4b84488ccc558cfcd16013c7
Reviewed-on: https://chromium-review.googlesource.com/700427
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb/DEPS

Project Member

Comment 9 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/84fa7158e582624b4e4d09da8ac35bfcf2e5e21a

commit 84fa7158e582624b4e4d09da8ac35bfcf2e5e21a
Author: Dan Jacques <dnj@chromium.org>
Date: Wed Oct 04 18:09:10 2017

[cipd_bootstrap_v2] Roll vpython canary/staging.

Roll vpython:
canary: bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb
staging: 0577e51184b2da24260cbb01763eba51915b9755

TBR=iannucci@chromium.org
BUG= chromium:771277 
TEST=None

Change-Id: I6b28e796e6925b3b6d65e16a8800a951a261e1e0
Reviewed-on: https://chromium-review.googlesource.com/700428
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/84fa7158e582624b4e4d09da8ac35bfcf2e5e21a/scripts/slave/cipd_bootstrap_v2.py

Project Member

Comment 10 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/f9ca741581276986c1d9110d1a281eb9dcc99572

commit f9ca741581276986c1d9110d1a281eb9dcc99572
Author: Dan Jacques <dnj@chromium.org>
Date: Wed Oct 04 20:06:16 2017

[vpython] Roll stable.

Roll "vpython" stable to canary:
bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb

TBR=iannucci@chromium.org
BUG= chromium:771277 
TEST=canary

Change-Id: Idf45d36ce22557cbcc44831f93d54b92bdd22cfe
Reviewed-on: https://chromium-review.googlesource.com/701175
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/f9ca741581276986c1d9110d1a281eb9dcc99572/scripts/slave/cipd_bootstrap_v2.py

Project Member

Comment 11 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/a58f1dacf1574708c916ece68b7edf3a27251010

commit a58f1dacf1574708c916ece68b7edf3a27251010
Author: Dan Jacques <dnj@chromium.org>
Date: Wed Oct 04 20:40:39 2017

[vpython] Roll CIPD version.

Roll CIPD version to:
bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb

TBR=iannucci@chromium.org
BUG= chromium:771277 
TEST=canary

Change-Id: I4fb064dcad7072a6a4f728f6c6384a8a65ad56a3
Reviewed-on: https://chromium-review.googlesource.com/701434
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/a58f1dacf1574708c916ece68b7edf3a27251010/cipd_manifest.txt

Project Member

Comment 12 by bugdroid1@chromium.org, Oct 4 2017

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/0ce03d3101add5d3dfb3f898d342d2f1d961d7e5

commit 0ce03d3101add5d3dfb3f898d342d2f1d961d7e5
Author: Dan Jacques <dnj@google.com>
Date: Wed Oct 04 20:54:27 2017

Comment 13 by tzik@chromium.org, Oct 5 2017

Status: Fixed (was: Untriaged)
Thanks! It works fine for me now.
Project Member

Comment 14 by bugdroid1@chromium.org, Oct 5 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/41288f5ae4dd53e7d93e29340751e25b9cefe24c

commit 41288f5ae4dd53e7d93e29340751e25b9cefe24c
Author: depot-tools-roller@chromium.org <depot-tools-roller@chromium.org>
Date: Thu Oct 05 10:49:11 2017

Roll src/third_party/depot_tools/ b2e961b11..b5807979e (72 commits; 5 trivial rolls)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/b2e961b1171d..b5807979e8a9

$ git log b2e961b11..b5807979e --date=short --no-merges --format='%ad %ae %s'
2017-10-04 xiaoyin.l Use HTTPS links in CheckAuthorizedAuthor
2017-10-04 dnj [vpython] Roll CIPD version.
2017-10-04 phajdan.jr gclient config: add support for custom vars
2017-10-03 phajdan.jr gclient: make predefined variables native
2017-10-03 mmoss Allow gclient solution URLs to be |None|.
2017-08-28 agable Simplify git-cl-diff for Gerrit
2017-10-03 phajdan.jr gclient flatten: properly quote conditionals
2017-09-28 phajdan.jr gclient: add support for native boolean variables
2017-09-29 phajdan.jr gclient flatten: emit conditions for hooks
2017-09-29 siggi Update fetch config with new Syzygy location.
2017-09-29 agable Remove rietveld commit queue command line client
2017-09-29 agable Remove rietveld git cherry-pick-upload command
2017-09-29 borenet roll-dep: Add comments indicating that --no-log needs to keep working
2017-09-28 mmoss Expose new gclient.sync() arg in gclient.checkout().
2017-09-07 brucedawson Fix autoninja to allow compiling one source file
2017-09-29 agable Remove defunct Rietveld end-to-end tests
2017-09-28 dnj [depot_tools] Add "ensure_bootstrap" script.
2017-09-28 agable bot_update: use patch repo instead of project if it is mapped
2017-09-27 mmoss Add ability to pass extra flags to gclient.sync().
2017-09-27 dnj Roll "vpython".
2017-09-26 agrieve Fix CheckPatchFormatted() when using inherit-review-settings-ok
2017-09-26 mmoss Remove unused gclient.sync() 'with_branch_heads' arg.
2017-09-25 phajdan.jr gclient: evaluate variables before passing them to GN
2017-09-26 phajdan.jr gclient: expose target_os
2017-09-26 yyanagisawa git-cl upload: adjust code-review score on TBR.
2017-09-25 emso Adds progress argument option to fetch for git checkout
2017-09-18 rharrison Add support for PDFium to my_activity.py
2017-09-18 rmistry Add skia_buildbot config
2017-09-14 mmoss Add some differentiating details to gerrit step names.
2017-09-15 ehmaldonado WebRTC: Remove hack in bot_update.py.
2017-09-13 agable git-cl: Parse new change urls containing project paths
2017-09-11 ehmaldonado WebRTC: Update source-of-truth on 'fetch webrtc'
2017-09-13 iannucci Roll vpython.
2017-09-13 athom Add gsutil.vpython spec to recipe bundles
2017-09-13 ehmaldonado Revert "Fix checkout.py issues when p.patchlevel > 1."
2017-09-12 ehmaldonado Fix checkout.py issues when p.patchlevel > 1.
2017-09-12 iannucci Roll led and vpython
2017-09-12 phajdan.jr gclient flatten: improve reporting of DEPS files
2017-09-11 iannucci Add s390x support for cipd client bootstrap script.
2017-09-11 agable git-cl upload: Send mail when starting CQ
2017-09-11 bjaideep gclient: Add aix to host_os list
2017-09-11 agable Add GCE instructions to git cl creds check
2017-09-11 ehmaldonado WebRTC: Update source-of-truth for 'git-cl'
2017-09-07 dcheng Use dateutil.parser for parsing dates.
2017-09-11 tikuta Update ninja to v1.8.2
2017-09-07 dnj [gsutil] run through "vpython" (2)
2017-09-07 mbjorge Revert "[gsutil] run through "vpython""
2017-09-07 dnj [gsutil] run through "vpython"
2017-09-07 phajdan.jr Revert "update_depot_tools: make cipd_bin_setup failures fatal"
2017-09-06 phajdan.jr cipd: also support wget as fetch command
2017-09-06 phajdan.jr update_depot_tools: make cipd_bin_setup failures fatal
2017-09-05 phajdan.jr gclient: fetch arbitrary refs
2017-09-06 phajdan.jr Move downstream recipe trybots to production
2017-09-06 brucedawson Revert "Fix autoninja.bat to not swallow ^^ sequences"
2017-09-06 agable Revert "Capture ctrl-c in presubmit multiprocessing pool"
2017-09-05 brucedawson Fix autoninja.bat to not swallow ^^ sequences
2017-09-06 sebmarchand Add a script to make it easier to build a single source file with ninja
2017-09-05 agable git-cl set_close: Don't fail on branches with no issue
2017-09-05 agable Remove gerrit-over-ssh code from my_activity.py
2017-09-05 agable Capture ctrl-c in presubmit multiprocessing pool
2017-08-24 agable Don't send email from git-cl-description
2017-09-04 whesse Dart: Update the config for the "fetch dart" command.
2017-09-05 tikuta Add Nico and Scott to OWNERS for ninja related things
2017-09-02 dpranke Add a --batch mode to `git-cl owners`.
2017-09-04 tikuta Update ninja to v1.8.0.
2017-09-01 iannucci Update led tool to c9c1865b81113f02fd618259624170f59e2c832e.
2017-08-31 dnj [vpython] Update version.

Created with:
  roll-dep src/third_party/depot_tools
BUG= 771277 ,769369, 764087 , 741001 ,None,758725,741001


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=phajdan.jr@chromium.org

Change-Id: Icb9ca53560ec8ee470e60fea5ba7fc23a764fb34
Reviewed-on: https://chromium-review.googlesource.com/701321
Reviewed-by: depot-tools-roller . <depot-tools-roller@chromium.org>
Commit-Queue: depot-tools-roller . <depot-tools-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#506697}
[modify] https://crrev.com/41288f5ae4dd53e7d93e29340751e25b9cefe24c/DEPS

Comment 15 Deleted

Sign in to add a comment