New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 764710 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 741001



Sign in to add a comment

vpython does not know how to infer CIPD platform "linux_aarch64"

Project Member Reported by whesse@google.com, Sep 13 2017

Issue description

After the change to vpython for depot_tools, the ARMv8 (arm64)
builder at https://luci-milo.appspot.com/buildbot/client.dart/target-arm64-vm-linux-release-be/ has been failing with a message:

failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ]

when running gsutil from depot_tools.  The full error is:


 USER: chrome-bot
 VPYTHON_VIRTUALENV_ROOT: /b/c/vpython

[E2017-09-13T04:48:22.392114-07:00 4887 0 annotate.go:236] goroutine 1:
[E2017-09-13T04:48:22.392178-07:00 4887 0 annotate.go:236] #0 vpython/pep425.go:238 - main.getPEP425CIPDTemplateForTag()
[E2017-09-13T04:48:22.392217-07:00 4887 0 annotate.go:236]   reason: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ]
[E2017-09-13T04:48:22.392298-07:00 4887 0 annotate.go:236] 
[E2017-09-13T04:48:22.392368-07:00 4887 0 annotate.go:236] #1 vpython/main.go:37 - main.glob..func1()
[E2017-09-13T04:48:22.392466-07:00 4887 0 annotate.go:236] #2 go.chromium.org/luci/vpython/cipd/cipd.go:72 - cipd.(*PackageLoader).Resolve()
[E2017-09-13T04:48:22.392510-07:00 4887 0 annotate.go:236]   reason: failed to get CIPD template arguments
[E2017-09-13T04:48:22.392541-07:00 4887 0 annotate.go:236] 
[E2017-09-13T04:48:22.392570-07:00 4887 0 annotate.go:236] #3 go.chromium.org/luci/vpython/venv/config.go:184 - venv.(*Config).makeEnv()
[E2017-09-13T04:48:22.392600-07:00 4887 0 annotate.go:236]   reason: failed to resolve packages
[E2017-09-13T04:48:22.392630-07:00 4887 0 annotate.go:236] 
[E2017-09-13T04:48:22.392683-07:00 4887 0 annotate.go:236] #4 go.chromium.org/luci/vpython/venv/venv.go:136 - venv.With()
[E2017-09-13T04:48:22.392715-07:00 4887 0 annotate.go:236] #5 go.chromium.org/luci/vpython/run.go:115 - vpython.Run()

 

Comment 1 by whesse@google.com, Sep 13 2017

Labels: -Pri-3 Pri-2

Comment 2 by whesse@google.com, Sep 13 2017

Blocking: 741001

Comment 4 by d...@chromium.org, Sep 13 2017

Summary: vpython does not know how to infer CIPD platform "linux_aarch64" (was: CIPD package for arm64 python not found)
Project Member

Comment 5 by bugdroid1@chromium.org, Sep 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced

commit 0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced
Author: Dan Jacques <dnj@google.com>
Date: Wed Sep 13 17:05:14 2017

[vpython] Support "aarch64" platform.

Add awareness and mapping for "aarch64" platform.

BUG= chromium:764710 
TEST=unit

Change-Id: I65b5d072a0b55e37947a372ba47e2ff5d2767c30
Reviewed-on: https://chromium-review.googlesource.com/665161
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced/vpython/cipd/pep425.go
[modify] https://crrev.com/0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced/vpython/cipd/pep425_test.go

Project Member

Comment 6 by bugdroid1@chromium.org, Sep 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/fb827c61d6f945aaa31f200cacf2ccb7df91aac3

commit fb827c61d6f945aaa31f200cacf2ccb7df91aac3
Author: Robert Iannucci <iannucci@chromium.org>
Date: Wed Sep 13 17:38:54 2017

Roll luci-go.

infra/go/src/go.chromium.org/luci:
0b4a49dd [vpython] Support "aarch64" platform.
825ff468 scheduler: use Gerrit scope for querying Gitiles.
845ece6b scheduler: fix wrong refs path.
bbecbedf scheduler: remove no longer useful repetitive logging.
1180a0c9 scheduler: add and remove TODOs for tandrii@.
990da3f7 [scheduler] set revision on buildbucket tasks triggered by gitiles.
18260b33 scheduler: Add jobID argument to GetVisibleInvocationsByNonce.
49fdf15b [scheduler] gitiles task now emits triggers.
90631bee scheduler: Extract controller instantiation into withController function.
ca3bdbbc [scheduler] propagate accumulated triggers to each task launcher.
9ddaedf1 scheduler: Extract Job entity update into a separate function.
76dea500 [scheduler] teach engine to propagate triggers.

TBR=dnj@chromium.org, vadimsh@chromium.org

Bug:  764710 
Change-Id: Iedb4207c65b1c202e60cc67184ae43cea694ee36
Reviewed-on: https://chromium-review.googlesource.com/664602
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/fb827c61d6f945aaa31f200cacf2ccb7df91aac3/DEPS

Project Member

Comment 7 by bugdroid1@chromium.org, Sep 13 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6

commit 2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6
Author: Robert Iannucci <iannucci@chromium.org>
Date: Wed Sep 13 18:14:27 2017

Roll vpython.

This picks up luci-go @ 0b4a49dd79cb1e7ef737a7ebea37c63c50f21ced
which adds awareness of the "linux-aarch64" platform/architecture
to vpython.

TBR=dnj@chromium.org, vadimsh@chromium.org

Bug:  764710 
Change-Id: Ib2098c89c64d042879ddb8d281513616d49087ef
Reviewed-on: https://chromium-review.googlesource.com/665305
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/2b619cc7119ba419fc2d15d1dd2aa55ffea7d0f6/cipd_manifest.txt

Comment 8 by whesse@google.com, Sep 20 2017

The error is still occurring on the builder build32-b3, see log 
https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fclient.dart%2Ftarget-arm64-vm-linux-release-be%2F1825%2F%2B%2Frecipes%2Fsteps%2Fgsutil_download_tarball%2F0%2Fstdout

Which says:

E2017-09-20T02:00:39.590061-07:00 20987 0 annotate.go:236] goroutine 1:
[E2017-09-20T02:00:39.590121-07:00 20987 0 annotate.go:236] #0 vpython/pep425.go:238 - main.getPEP425CIPDTemplateForTag()
[E2017-09-20T02:00:39.590159-07:00 20987 0 annotate.go:236]   reason: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_aarch64" ]
[E2017-09-20T02:00:39.590235-07:00 20987 0 annotate.go:236] 
[E2017-09-20T02:00:39.590303-07:00 20987 0 annotate.go:236] #1 vpython/main.go:37 - main.glob..func1()
[E2017-09-20T02:00:39.590399-07:00 20987 0 annotate.go:236] #2 go.chromium.org/luci/vpython/cipd/cipd.go:72 - cipd.(*PackageLoader).Resolve()
[E2017-09-20T02:00:39.590443-07:00 20987 0 annotate.go:236]   reason: failed to get CIPD template arguments
[E2017-09-20T02:00:39.590474-07:00 20987 0 annotate.go:236] 
[E2017-09-20T02:00:39.590502-07:00 20987 0 annotate.go:236] #3 go.chromium.org/luci/vpython/venv/config.go:184 - venv.(*Config).makeEnv()
[E2017-09-20T02:00:39.590532-07:00 20987 0 annotate.go:236]   reason: failed to resolve packages
[E2017-09-20T02:00:39.590578-07:00 20987 0 annotate.go:236] 
[E2017-09-20T02:00:39.590614-07:00 20987 0 annotate.go:236] #4 go.chromium.org/luci/vpython/venv/venv.go:136 - venv.With()
[E2017-09-20T02:00:39.590644-07:00 20987 0 annotate.go:236] #5 go.chromium.org/luci/vpython/run.go:115 - vpython.Run()
[E2017-09-20T02:00:39.590692-07:00 20987 0 annotate.go:236] #6 go.chromium.org/luci/vpython/application/application.go:266 - application.(*application).mainImpl()
[E2017-09-20T02:00:39.590728-07:00 20987 0 annotate.go:236] #7 go.chromium.org/luci/vpython/application/application.go:343 - application.(*Config).Main.func1()

Project Member

Comment 11 by bugdroid1@chromium.org, Sep 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/55e90753c017f0b540839c61560331469734befb

commit 55e90753c017f0b540839c61560331469734befb
Author: Dan Jacques <dnj@google.com>
Date: Wed Sep 20 19:36:27 2017

[cipd_bootstrap_v2] Bump vpython version.

BUG= chromium:764710 
TEST=None
  - Canary seems stable.

R=iannucci@chromium.org, nodir@chromium.org

Change-Id: I6cf73c94364a69cb6371f68f9a585acd8d177812
Reviewed-on: https://chromium-review.googlesource.com/675703
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/55e90753c017f0b540839c61560331469734befb/scripts/slave/cipd_bootstrap_v2.py

Project Member

Comment 12 by bugdroid1@chromium.org, Sep 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/e49aeab38a510a8335ac7e72075a126424f0efb4

commit e49aeab38a510a8335ac7e72075a126424f0efb4
Author: Dan Jacques <dnj@chromium.org>
Date: Fri Sep 22 18:53:11 2017

[cipd_bootstrap_v2] Bump staging/prod vpython.

Roll staging vpython to production.

Bump staging vpython to f431894262f24aa6107a0022eacb5c8964f4fd48.

TBR=iannucci@chromium.org
BUG= chromium:764710 
TEST=None

Change-Id: Ifa0c94e4c56c5e70696230cd9691457d17efc9a2
Reviewed-on: https://chromium-review.googlesource.com/679056
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/e49aeab38a510a8335ac7e72075a126424f0efb4/scripts/slave/cipd_bootstrap_v2.py

Comment 13 by s...@google.com, Sep 23 2017

Cc: -d...@chromium.org
Owner: d...@chromium.org
Status: Started (was: Untriaged)
Is this done?

Comment 14 by d...@chromium.org, Sep 23 2017

Status: Fixed (was: Started)
The build is still failing, but for a different reason. This problem is solved.

Comment 15 by whesse@google.com, Sep 25 2017

The reason the build is still failing is that cipd and update_depot_tools do
not work for aarch64.  I will file a new issue for this, with the information below.  Leaving this issue closed.

The build is still failing for 2 reasons: The cipd script does not accept an architecture aarch64, and if it is modified to accept that architecture, there is no cipd package for aarch64: cipd ensure returns:

Bootstrapping cipd client for linux-aarch64 from https://chrome-infra-packages.appspot.com/client?platform=linux-aarch64&version=git_revision:0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1...
curl: (22) The requested URL returned error: 404 Not Found

The full details of debugging, to get to this error, are:

The build is failing with a vpython failure: /usr/bin/python -u /b/build/scripts/slave/.recipe_deps/depot_tools/gsutil.py -- cp gs://dart-cross-compiled-binaries/cross_build_arm64_release_415a6e2a751d25f26ae9fa862318cf5645f7adc5.tar.bz2 cross_build_arm64_release_415a6e2a751d25f26ae9fa862318cf5645f7adc5.tar.bz2

produces the error:
/b/depot_tools/vpython: line 12: /b/depot_tools/.cipd_bin/vpython: No such file or directory

and this is because the vpython script runs "cipd_bin_setup", which returns "UNKNOWN Machine architecture: aarch64"

The cipd_bin_setup bash function runs

cipd ensure -log-level warning -ensure-file cipd_manifest.txt -root .cipd_bin
(my reproduction command line is run in /b/depot_tools, so I don't show the paths to the args)

When I change the cipd script to accept a machine type aarch64, by changing line 37 from 
arm*)
to
arm*|aarch*)
I get the error 
Bootstrapping cipd client for linux-aarch64 from https://chrome-infra-packages.appspot.com/client?platform=linux-aarch64&version=git_revision:0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1...
curl: (22) The requested URL returned error: 404 Not Found

This also happens when running update_depot_tools.

Comment 16 by d...@chromium.org, Sep 25 2017

That last failure is because the CIPD service doesn't support the architecture, and thus cannot download a client for it.

Comment 17 by whesse@google.com, Sep 25 2017

New issue filed for the fact the CIPD does not support aarch64.  Please comment further there.  https://bugs.chromium.org/p/chromium/issues/detail?id=768318

Sign in to add a comment