vpython does not know how to infer CIPD platform "linux_aarch64" |
|||||
Issue descriptionAfter the change to vpython for depot_tools, the ARMv8 (arm64) builder at https://luci-milo.appspot.com/buildbot/client.dart/target-arm64-vm-linux-release-be/ has been failing with a message: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ] when running gsutil from depot_tools. The full error is: USER: chrome-bot VPYTHON_VIRTUALENV_ROOT: /b/c/vpython [E2017-09-13T04:48:22.392114-07:00 4887 0 annotate.go:236] goroutine 1: [E2017-09-13T04:48:22.392178-07:00 4887 0 annotate.go:236] #0 vpython/pep425.go:238 - main.getPEP425CIPDTemplateForTag() [E2017-09-13T04:48:22.392217-07:00 4887 0 annotate.go:236] reason: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ] [E2017-09-13T04:48:22.392298-07:00 4887 0 annotate.go:236] [E2017-09-13T04:48:22.392368-07:00 4887 0 annotate.go:236] #1 vpython/main.go:37 - main.glob..func1() [E2017-09-13T04:48:22.392466-07:00 4887 0 annotate.go:236] #2 go.chromium.org/luci/vpython/cipd/cipd.go:72 - cipd.(*PackageLoader).Resolve() [E2017-09-13T04:48:22.392510-07:00 4887 0 annotate.go:236] reason: failed to get CIPD template arguments [E2017-09-13T04:48:22.392541-07:00 4887 0 annotate.go:236] [E2017-09-13T04:48:22.392570-07:00 4887 0 annotate.go:236] #3 go.chromium.org/luci/vpython/venv/config.go:184 - venv.(*Config).makeEnv() [E2017-09-13T04:48:22.392600-07:00 4887 0 annotate.go:236] reason: failed to resolve packages [E2017-09-13T04:48:22.392630-07:00 4887 0 annotate.go:236] [E2017-09-13T04:48:22.392683-07:00 4887 0 annotate.go:236] #4 go.chromium.org/luci/vpython/venv/venv.go:136 - venv.With() [E2017-09-13T04:48:22.392715-07:00 4887 0 annotate.go:236] #5 go.chromium.org/luci/vpython/run.go:115 - vpython.Run()
,
Sep 13 2017
,
Sep 13 2017
,
Sep 13 2017
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-go.git/+/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced commit 0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced Author: Dan Jacques <dnj@google.com> Date: Wed Sep 13 17:05:14 2017 [vpython] Support "aarch64" platform. Add awareness and mapping for "aarch64" platform. BUG= chromium:764710 TEST=unit Change-Id: I65b5d072a0b55e37947a372ba47e2ff5d2767c30 Reviewed-on: https://chromium-review.googlesource.com/665161 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: William Hesse <whesse@google.com> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced/vpython/cipd/pep425.go [modify] https://crrev.com/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced/vpython/cipd/pep425_test.go
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/fb827c61d6f945aaa31f200cacf2ccb7df91aac3 commit fb827c61d6f945aaa31f200cacf2ccb7df91aac3 Author: Robert Iannucci <iannucci@chromium.org> Date: Wed Sep 13 17:38:54 2017 Roll luci-go. infra/go/src/go.chromium.org/luci: 0b4a49dd [vpython] Support "aarch64" platform. 825ff468 scheduler: use Gerrit scope for querying Gitiles. 845ece6b scheduler: fix wrong refs path. bbecbedf scheduler: remove no longer useful repetitive logging. 1180a0c9 scheduler: add and remove TODOs for tandrii@. 990da3f7 [scheduler] set revision on buildbucket tasks triggered by gitiles. 18260b33 scheduler: Add jobID argument to GetVisibleInvocationsByNonce. 49fdf15b [scheduler] gitiles task now emits triggers. 90631bee scheduler: Extract controller instantiation into withController function. ca3bdbbc [scheduler] propagate accumulated triggers to each task launcher. 9ddaedf1 scheduler: Extract Job entity update into a separate function. 76dea500 [scheduler] teach engine to propagate triggers. TBR=dnj@chromium.org, vadimsh@chromium.org Bug: 764710 Change-Id: Iedb4207c65b1c202e60cc67184ae43cea694ee36 Reviewed-on: https://chromium-review.googlesource.com/664602 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/fb827c61d6f945aaa31f200cacf2ccb7df91aac3/DEPS
,
Sep 13 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6 commit 2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6 Author: Robert Iannucci <iannucci@chromium.org> Date: Wed Sep 13 18:14:27 2017 Roll vpython. This picks up luci-go @ 0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced which adds awareness of the "linux-aarch64" platform/architecture to vpython. TBR=dnj@chromium.org, vadimsh@chromium.org Bug: 764710 Change-Id: Ib2098c89c64d042879ddb8d281513616d49087ef Reviewed-on: https://chromium-review.googlesource.com/665305 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6/cipd_manifest.txt
,
Sep 20 2017
The error is still occurring on the builder build32-b3, see log https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fclient.dart%2Ftarget-arm64-vm-linux-release-be%2F1825%2F%2B%2Frecipes%2Fsteps%2Fgsutil_download_tarball%2F0%2Fstdout Which says: E2017-09-20T02:00:39.590061-07:00 20987 0 annotate.go:236] goroutine 1: [E2017-09-20T02:00:39.590121-07:00 20987 0 annotate.go:236] #0 vpython/pep425.go:238 - main.getPEP425CIPDTemplateForTag() [E2017-09-20T02:00:39.590159-07:00 20987 0 annotate.go:236] reason: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ] [E2017-09-20T02:00:39.590235-07:00 20987 0 annotate.go:236] [E2017-09-20T02:00:39.590303-07:00 20987 0 annotate.go:236] #1 vpython/main.go:37 - main.glob..func1() [E2017-09-20T02:00:39.590399-07:00 20987 0 annotate.go:236] #2 go.chromium.org/luci/vpython/cipd/cipd.go:72 - cipd.(*PackageLoader).Resolve() [E2017-09-20T02:00:39.590443-07:00 20987 0 annotate.go:236] reason: failed to get CIPD template arguments [E2017-09-20T02:00:39.590474-07:00 20987 0 annotate.go:236] [E2017-09-20T02:00:39.590502-07:00 20987 0 annotate.go:236] #3 go.chromium.org/luci/vpython/venv/config.go:184 - venv.(*Config).makeEnv() [E2017-09-20T02:00:39.590532-07:00 20987 0 annotate.go:236] reason: failed to resolve packages [E2017-09-20T02:00:39.590578-07:00 20987 0 annotate.go:236] [E2017-09-20T02:00:39.590614-07:00 20987 0 annotate.go:236] #4 go.chromium.org/luci/vpython/venv/venv.go:136 - venv.With() [E2017-09-20T02:00:39.590644-07:00 20987 0 annotate.go:236] #5 go.chromium.org/luci/vpython/run.go:115 - vpython.Run() [E2017-09-20T02:00:39.590692-07:00 20987 0 annotate.go:236] #6 go.chromium.org/luci/vpython/application/application.go:266 - application.(*application).mainImpl() [E2017-09-20T02:00:39.590728-07:00 20987 0 annotate.go:236] #7 go.chromium.org/luci/vpython/application/application.go:343 - application.(*Config).Main.func1()
,
Sep 20 2017
We need to actually roll "vpython" here: https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/cipd_bootstrap_v2.py
,
Sep 20 2017
,
Sep 20 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/55e90753c017f0b540839c61560331469734befb commit 55e90753c017f0b540839c61560331469734befb Author: Dan Jacques <dnj@google.com> Date: Wed Sep 20 19:36:27 2017 [cipd_bootstrap_v2] Bump vpython version. BUG= chromium:764710 TEST=None - Canary seems stable. R=iannucci@chromium.org, nodir@chromium.org Change-Id: I6cf73c94364a69cb6371f68f9a585acd8d177812 Reviewed-on: https://chromium-review.googlesource.com/675703 Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/55e90753c017f0b540839c61560331469734befb/scripts/slave/cipd_bootstrap_v2.py
,
Sep 22 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/e49aeab38a510a8335ac7e72075a126424f0efb4 commit e49aeab38a510a8335ac7e72075a126424f0efb4 Author: Dan Jacques <dnj@chromium.org> Date: Fri Sep 22 18:53:11 2017 [cipd_bootstrap_v2] Bump staging/prod vpython. Roll staging vpython to production. Bump staging vpython to f431894262f24aa6107a0022eacb5c8964f4fd48. TBR=iannucci@chromium.org BUG= chromium:764710 TEST=None Change-Id: Ifa0c94e4c56c5e70696230cd9691457d17efc9a2 Reviewed-on: https://chromium-review.googlesource.com/679056 Reviewed-by: Daniel Jacques <dnj@chromium.org> Commit-Queue: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/e49aeab38a510a8335ac7e72075a126424f0efb4/scripts/slave/cipd_bootstrap_v2.py
,
Sep 23 2017
Is this done?
,
Sep 23 2017
The build is still failing, but for a different reason. This problem is solved.
,
Sep 25 2017
The reason the build is still failing is that cipd and update_depot_tools do not work for aarch64. I will file a new issue for this, with the information below. Leaving this issue closed. The build is still failing for 2 reasons: The cipd script does not accept an architecture aarch64, and if it is modified to accept that architecture, there is no cipd package for aarch64: cipd ensure returns: Bootstrapping cipd client for linux-aarch64 from https://chrome-infra-packages.appspot.com/client?platform=linux-aarch64&version=git_revision:0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1... curl: (22) The requested URL returned error: 404 Not Found The full details of debugging, to get to this error, are: The build is failing with a vpython failure: /usr/bin/python -u /b/build/scripts/slave/.recipe_deps/depot_tools/gsutil.py -- cp gs://dart-cross-compiled-binaries/cross_build_arm64_release_415a6e2a751d25f26ae9fa862318cf5645f7adc5.tar.bz2 cross_build_arm64_release_415a6e2a751d25f26ae9fa862318cf5645f7adc5.tar.bz2 produces the error: /b/depot_tools/vpython: line 12: /b/depot_tools/.cipd_bin/vpython: No such file or directory and this is because the vpython script runs "cipd_bin_setup", which returns "UNKNOWN Machine architecture: aarch64" The cipd_bin_setup bash function runs cipd ensure -log-level warning -ensure-file cipd_manifest.txt -root .cipd_bin (my reproduction command line is run in /b/depot_tools, so I don't show the paths to the args) When I change the cipd script to accept a machine type aarch64, by changing line 37 from arm*) to arm*|aarch*) I get the error Bootstrapping cipd client for linux-aarch64 from https://chrome-infra-packages.appspot.com/client?platform=linux-aarch64&version=git_revision:0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1... curl: (22) The requested URL returned error: 404 Not Found This also happens when running update_depot_tools.
,
Sep 25 2017
That last failure is because the CIPD service doesn't support the architecture, and thus cannot download a client for it.
,
Sep 25 2017
New issue filed for the fact the CIPD does not support aarch64. Please comment further there. https://bugs.chromium.org/p/chromium/issues/detail?id=768318 |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by whesse@google.com
, Sep 13 2017