run cipd will give "UNKNOWN Machine architecture: s390x"

Moved issue v8:6813 to now be  issue chromium:764087 .

Ok, let's take a step back.

From what I can gather:
 * You're using depot_tools on an IBM s390 server?
 * What are you using depot_tools for?

I'm not aware that this is a platform that depot_tools is intended to support.

Hello,

Yes, we are using depot_tools on a s390 server. We have been using depot_tools to download and update v8 and v8 related packages.

I guess I'm asking; why are you using gsutil from depot_tools and not one from e.g. the gcloud SDK? Is it because this is being called during `gclient sync`?

ah, yes, I see this is part of download_from_google_storage.

yes, you are right. :)

So it looks like go supports this architecture out of the box. We already have the mechanism in-place to support cross compilation, so I'll add it to our target architectures there.

That said, I don't believe that we (the chromium operations team) officially support s390x as a platform :). Hopefully this 'just works' for now though.

CLs:
  * https://chromium-review.googlesource.com/c/infra/infra/+/661783
  * https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/661787

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1

commit 0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1
Author: Robert Iannucci <iannucci@chromium.org>
Date: Mon Sep 11 23:02:25 2017

Add s390x support for infra binaries.

This is best-effort only; s390x is still not a supported platform.

However, since we already have cross compilation support and s390x is a
supported GOARCH, this is easy to add.

R=dnj@chromium.org, vadimsh@chromium.org

Bug:  764087 
Change-Id: I115d3a154cfbb364b110dff0fceffb5d80e68575
Reviewed-on: https://chromium-review.googlesource.com/661783
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1/recipes/recipes/infra_continuous.py
[modify] https://crrev.com/0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1/build/build.py
[modify] https://crrev.com/0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1/recipes/README.recipes.md
[modify] https://crrev.com/0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1/recipes/recipes/infra_continuous.expected/infra-cross-compile.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/ff2bf09daf6b2150d55d0290c480ef728655c2cb

commit ff2bf09daf6b2150d55d0290c480ef728655c2cb
Author: Robert Iannucci <iannucci@chromium.org>
Date: Mon Sep 11 23:36:59 2017

Add s390x support for cipd client bootstrap script.

This is best-effort only; s390x is still not a supported platform.

CIPD pin bump contains no client changes; just points to a version
which is built for s390x as well our normal platforms.

R=dnj@chromium.org, vadimsh@chromium.org

Bug:  764087 
Change-Id: I777246712eea1b39036f3b14b88cbd41c7cf0507
Reviewed-on: https://chromium-review.googlesource.com/661787
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/ff2bf09daf6b2150d55d0290c480ef728655c2cb/cipd
[modify] https://crrev.com/ff2bf09daf6b2150d55d0290c480ef728655c2cb/cipd_client_version

Hey so this is (probably?) fixed now; I would go ahead and update depot_tools and see if to works.

Hello, thank you very much for helping! Now the error message looks like this:

johnyan@sancus:/sandbox/johnyan/v8-master/v8>gclient sync
Bootstrapping cipd client for linux-s390x from https://chrome-infra-packages.appspot.com/client?platform=linux-s390x&version=git_revision:0aa5e4d06ec4d3b90980fcb0d0b7c34a5e65b9e1...
Error: resolving package: failed to resolve package version (line 6): package "infra/tools/luci/vpython/linux-s390x" doesn't have instance with version "git_revision:33c91153e35ed9b42acdb2b16a78c38b3cf424db".
Syncing projects: 100% (22/22), done.

________ running '/usr/bin/python v8/gypfiles/landmines.py' in '/sandbox/johnyan/v8-master'

________ running 'download_from_google_storage --no_resume --platform=win32 --no_auth --bucket chromium-clang-format -s v8/buildtools/win/clang-format.exe.sha1' in '/sandbox/johnyan/v8-master'
The current platform doesn't match "win32", skipping.

________ running 'download_from_google_storage --no_resume --platform=darwin --no_auth --bucket chromium-clang-format -s v8/buildtools/mac/clang-format.sha1' in '/sandbox/johnyan/v8-master'
The current platform doesn't match "darwin", skipping.

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s v8/buildtools/linux64/clang-format.sha1' in '/sandbox/johnyan/v8-master'
Failed to fetch file gs://chromium-clang-format/5349d1954e17f6ccafb6e6663b0f13cdb2bb33c8 for v8/buildtools/linux64/clang-format. [Err: /home/johnyan/depot_tools/vpython: line 12: /home/johnyan/depot_tools/.cipd_bin/vpython: No such file or directory
]
0> Failed to fetch file gs://chromium-clang-format/5349d1954e17f6ccafb6e6663b0f13cdb2bb33c8 for v8/buildtools/linux64/clang-format, skipping. [Err: /home/johnyan/depot_tools/vpython: line 12: /home/johnyan/depot_tools/.cipd_bin/vpython: No such file or directory
]
Downloading 1 files took 7.761223 second(s)
Error: Command 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s v8/buildtools/linux64/clang-format.sha1' returned non-zero exit status 1 in /sandbox/johnyan/v8-master

ah, right, I forgot to roll the vpython pin as well.

Ok, please try again

Hello,

I am now getting this error:

________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s v8/buildtools/linux64/clang-format.sha1' in '/sandbox/johnyan/v8-master'
Failed to fetch file gs://chromium-clang-format/5349d1954e17f6ccafb6e6663b0f13cdb2bb33c8 for v8/buildtools/linux64/clang-format. [Err: [E2017-09-12T15:48:13.797044-04:00 5252 0 annotate.go:236] goroutine 1:
[E2017-09-12T15:48:13.797092-04:00 5252 0 annotate.go:236] #0 vpython/pep425.go:238 - main.getPEP425CIPDTemplateForTag()
[E2017-09-12T15:48:13.797102-04:00 5252 0 annotate.go:236]   reason: failed to infer CIPD platform for tag [python:"cp27" abi:"cp27mu" platform:"linux_s390x" ]
[E2017-09-12T15:48:13.797110-04:00 5252 0 annotate.go:236]
[E2017-09-12T15:48:13.797117-04:00 5252 0 annotate.go:236] #1 vpython/main.go:37 - main.glob..func1()
[E2017-09-12T15:48:13.797129-04:00 5252 0 annotate.go:236] #2 go.chromium.org/luci/vpython/cipd/cipd.go:72 - cipd.(*PackageLoader).Resolve()
[E2017-09-12T15:48:13.797143-04:00 5252 0 annotate.go:236]   reason: failed to get CIPD template arguments
[E2017-09-12T15:48:13.797150-04:00 5252 0 annotate.go:236]
[E2017-09-12T15:48:13.797158-04:00 5252 0 annotate.go:236] #3 go.chromium.org/luci/vpython/venv/config.go:184 - venv.(*Config).makeEnv()
[E2017-09-12T15:48:13.797165-04:00 5252 0 annotate.go:236]   reason: failed to resolve packages
[E2017-09-12T15:48:13.797173-04:00 5252 0 annotate.go:236]
[E2017-09-12T15:48:13.797180-04:00 5252 0 annotate.go:236] #4 go.chromium.org/luci/vpython/venv/venv.go:136 - venv.With()
[E2017-09-12T15:48:13.797187-04:00 5252 0 annotate.go:236] #5 go.chromium.org/luci/vpython/run.go:115 - vpython.Run()
[E2017-09-12T15:48:13.797194-04:00 5252 0 annotate.go:236] #6 go.chromium.org/luci/vpython/application/application.go:266 - application.(*application).mainImpl()
[E2017-09-12T15:48:13.797202-04:00 5252 0 annotate.go:236] #7 go.chromium.org/luci/vpython/application/application.go:343 - application.(*Config).Main.func1()
[E2017-09-12T15:48:13.797231-04:00 5252 0 annotate.go:236] #8 go.chromium.org/luci/vpython/application/support.go:36 - application.run()
[E2017-09-12T15:48:13.797239-04:00 5252 0 annotate.go:236] #9 go.chromium.org/luci/vpython/application/application.go:344 - application.(*Config).Main()
[E2017-09-12T15:48:13.797253-04:00 5252 0 annotate.go:236] #10 vpython/main.go:80 - main.mainImpl()
[E2017-09-12T15:48:13.797265-04:00 5252 0 annotate.go:236] #11 vpython/main.go:86 - main.main()
[E2017-09-12T15:48:13.797272-04:00 5252 0 annotate.go:236] #12 runtime/proc.go:185 - runtime.main()
[E2017-09-12T15:48:13.797284-04:00 5252 0 annotate.go:236] #13 runtime/asm_s390x.s:994 - runtime.goexit()
]

I checked https://github.com/luci/luci-go/blob/master/vpython/cipd/pep425.go
Maybe need to add s390x to PlatformForPEP425Tag.

Thanks again!
John

ah, yep, that's the next bump :).

However, even if we add it there, we'd still have to begin cross-compiling all of our python wheels for s390x too. We use the dockcross project for this (https://github.com/dockcross/dockcross). If you can get a s390x cross compilation environment working in dockcross, we can import those images and add them to our python wheel cross-compilation stuff.

That said, some of the packages (like cryptography) are notoriously finicky. If the process to support s390x varies from e.g. linux-x64, we'll probably need to look for a different solution.

Hi John, 

If you can, please share your priority for this work and whether there's another path you can take to sidestep this issue. Until earlier today, the foundation team was not aware this was a supported HW platform and we need a bit of time to prioritize the work. Your feedback will help us figure this out and plan next steps accordingly. 

Thanks!

Hello,

Thank you both for helping!

We need to fix this issue in order to continuously support and maintain V8 on s390/ppc platform, which we have been doing since around 2014, because Node.js is using V8 as its core JS engine. Given that this issue breaks gclient sync, it's very important for us. We will try cross compilation first to see how it goes, because the Node.js team is also trying that. I will get back to you when I have a clue.

Thanks again!
John

Hello,

I finally got a working dockcross for s390. The pull request is opened at https://github.com/dockcross/dockcross/pull/162.

Thanks!

Hello, the pull request for dockcross has been merged to master. Could you please give it another try to get vpython and cipd binaries for s390? Thanks.

Robbie (or Dan), it looks like John was able to do the work suggested in comment 19, is this back on our side now?

Hi, just put a reminder on this one.

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/1af25a30cf1d41ed6170e61f001caab1ad80e920

commit 1af25a30cf1d41ed6170e61f001caab1ad80e920
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Sep 28 01:37:32 2017

[vpython] Implement bypass mode.

Implement the ability for a caller to instruct "vpython" to completely
bypass the VirtualEnv setup/execution and run the provided Python
command directly.

This will be used to allow unsupported architectures to function.

BUG= chromium:764087 
TEST=local
R=iannucci@chromium.org

Change-Id: I2dfdf18fd92db71264b94248f0f8848d8c2be95c
Reviewed-on: https://chromium-review.googlesource.com/688580
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/1af25a30cf1d41ed6170e61f001caab1ad80e920/vpython/application/application.go
[modify] https://crrev.com/1af25a30cf1d41ed6170e61f001caab1ad80e920/vpython/python/find.go
[modify] https://crrev.com/1af25a30cf1d41ed6170e61f001caab1ad80e920/vpython/venv/config.go

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/0577e51184b2da24260cbb01763eba51915b9755

commit 0577e51184b2da24260cbb01763eba51915b9755
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Sep 28 01:55:34 2017

[vpython] Implement environment bypass.

Implement environment bypass mode in "vpython". If a user sets the
VPYTHON_BYPASS environment variable to a specific sentinel value,
"vpython" will skip its VirtualEnv setup and run Python directly.

BUG= chromium:764087 
TEST=local,unit
  - Ran locally, observed things working.
  - Unit test for bypass mode succeeds.

Change-Id: Ic45691876ab4aedcb19f57dc61ca20a27cace24e
Reviewed-on: https://chromium-review.googlesource.com/688716
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[add] https://crrev.com/0577e51184b2da24260cbb01763eba51915b9755/go/src/infra/tools/vpython/test_data/test_bypass.py
[modify] https://crrev.com/0577e51184b2da24260cbb01763eba51915b9755/go/src/infra/tools/vpython/main.go
[modify] https://crrev.com/0577e51184b2da24260cbb01763eba51915b9755/go/src/infra/tools/vpython/main_test.go
[modify] https://crrev.com/0577e51184b2da24260cbb01763eba51915b9755/DEPS
[add] https://crrev.com/0577e51184b2da24260cbb01763eba51915b9755/go/src/infra/tools/vpython/test_data/test_bypass.output

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/4145082558e22927a2dc85c8459533b106e1051f

commit 4145082558e22927a2dc85c8459533b106e1051f
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Sep 28 04:02:18 2017

Roll "vpython".

Roll "vpython" to git_revision:0577e51184b2da24260cbb01763eba51915b9755

TBR=iannucci@chromium.org
BUG= chromium:764087 
TEST=None

Change-Id: I117a0c2d368303805abb6426e3b9e4cbb8997b91
Reviewed-on: https://chromium-review.googlesource.com/688584
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/4145082558e22927a2dc85c8459533b106e1051f/cipd_manifest.txt

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/ca062d13fcf3b339e6ede38f6f7a5e04ddf87990

commit ca062d13fcf3b339e6ede38f6f7a5e04ddf87990
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Sep 28 04:34:11 2017

[vpython] Roll canary revision.

Roll vpython canary revision to:
0577e51184b2da24260cbb01763eba51915b9755

TBR=iannucci@chromium.org
BUG= chromium:764087 
TEST=None

Change-Id: I90c7b12f7d46330f488b072cab70b166c7930c0f
Reviewed-on: https://chromium-review.googlesource.com/688719
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/ca062d13fcf3b339e6ede38f6f7a5e04ddf87990/scripts/slave/cipd_bootstrap_v2.py

This change: https://chromium.googlesource.com/infra/infra/+/0577e51184b2da24260cbb01763eba51915b9755/go/src/infra/tools/vpython/main.go#28

Should let you do the following:
$ export VPYTHON_BYPASS="manually managed python not supported by chrome operations"

After that, the s390 "vpython" build will not try and download wheels or create a VirtualEnv, and you're on your own with respect to package support. Sound good?

(note that the python packages shouldn't be too exotic; everything should be obtainable via PyPI. We just use vpython to manage these packages explicitly for devs and bots on the platforms we maintain.)

looks good to me. Thank you for helping!

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/41288f5ae4dd53e7d93e29340751e25b9cefe24c

commit 41288f5ae4dd53e7d93e29340751e25b9cefe24c
Author: depot-tools-roller@chromium.org <depot-tools-roller@chromium.org>
Date: Thu Oct 05 10:49:11 2017

Roll src/third_party/depot_tools/ b2e961b11..b5807979e (72 commits; 5 trivial rolls)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/b2e961b1171d..b5807979e8a9

$ git log b2e961b11..b5807979e --date=short --no-merges --format='%ad %ae %s'
2017-10-04 xiaoyin.l Use HTTPS links in CheckAuthorizedAuthor
2017-10-04 dnj [vpython] Roll CIPD version.
2017-10-04 phajdan.jr gclient config: add support for custom vars
2017-10-03 phajdan.jr gclient: make predefined variables native
2017-10-03 mmoss Allow gclient solution URLs to be |None|.
2017-08-28 agable Simplify git-cl-diff for Gerrit
2017-10-03 phajdan.jr gclient flatten: properly quote conditionals
2017-09-28 phajdan.jr gclient: add support for native boolean variables
2017-09-29 phajdan.jr gclient flatten: emit conditions for hooks
2017-09-29 siggi Update fetch config with new Syzygy location.
2017-09-29 agable Remove rietveld commit queue command line client
2017-09-29 agable Remove rietveld git cherry-pick-upload command
2017-09-29 borenet roll-dep: Add comments indicating that --no-log needs to keep working
2017-09-28 mmoss Expose new gclient.sync() arg in gclient.checkout().
2017-09-07 brucedawson Fix autoninja to allow compiling one source file
2017-09-29 agable Remove defunct Rietveld end-to-end tests
2017-09-28 dnj [depot_tools] Add "ensure_bootstrap" script.
2017-09-28 agable bot_update: use patch repo instead of project if it is mapped
2017-09-27 mmoss Add ability to pass extra flags to gclient.sync().
2017-09-27 dnj Roll "vpython".
2017-09-26 agrieve Fix CheckPatchFormatted() when using inherit-review-settings-ok
2017-09-26 mmoss Remove unused gclient.sync() 'with_branch_heads' arg.
2017-09-25 phajdan.jr gclient: evaluate variables before passing them to GN
2017-09-26 phajdan.jr gclient: expose target_os
2017-09-26 yyanagisawa git-cl upload: adjust code-review score on TBR.
2017-09-25 emso Adds progress argument option to fetch for git checkout
2017-09-18 rharrison Add support for PDFium to my_activity.py
2017-09-18 rmistry Add skia_buildbot config
2017-09-14 mmoss Add some differentiating details to gerrit step names.
2017-09-15 ehmaldonado WebRTC: Remove hack in bot_update.py.
2017-09-13 agable git-cl: Parse new change urls containing project paths
2017-09-11 ehmaldonado WebRTC: Update source-of-truth on 'fetch webrtc'
2017-09-13 iannucci Roll vpython.
2017-09-13 athom Add gsutil.vpython spec to recipe bundles
2017-09-13 ehmaldonado Revert "Fix checkout.py issues when p.patchlevel > 1."
2017-09-12 ehmaldonado Fix checkout.py issues when p.patchlevel > 1.
2017-09-12 iannucci Roll led and vpython
2017-09-12 phajdan.jr gclient flatten: improve reporting of DEPS files
2017-09-11 iannucci Add s390x support for cipd client bootstrap script.
2017-09-11 agable git-cl upload: Send mail when starting CQ
2017-09-11 bjaideep gclient: Add aix to host_os list
2017-09-11 agable Add GCE instructions to git cl creds check
2017-09-11 ehmaldonado WebRTC: Update source-of-truth for 'git-cl'
2017-09-07 dcheng Use dateutil.parser for parsing dates.
2017-09-11 tikuta Update ninja to v1.8.2
2017-09-07 dnj [gsutil] run through "vpython" (2)
2017-09-07 mbjorge Revert "[gsutil] run through "vpython""
2017-09-07 dnj [gsutil] run through "vpython"
2017-09-07 phajdan.jr Revert "update_depot_tools: make cipd_bin_setup failures fatal"
2017-09-06 phajdan.jr cipd: also support wget as fetch command
2017-09-06 phajdan.jr update_depot_tools: make cipd_bin_setup failures fatal
2017-09-05 phajdan.jr gclient: fetch arbitrary refs
2017-09-06 phajdan.jr Move downstream recipe trybots to production
2017-09-06 brucedawson Revert "Fix autoninja.bat to not swallow ^^ sequences"
2017-09-06 agable Revert "Capture ctrl-c in presubmit multiprocessing pool"
2017-09-05 brucedawson Fix autoninja.bat to not swallow ^^ sequences
2017-09-06 sebmarchand Add a script to make it easier to build a single source file with ninja
2017-09-05 agable git-cl set_close: Don't fail on branches with no issue
2017-09-05 agable Remove gerrit-over-ssh code from my_activity.py
2017-09-05 agable Capture ctrl-c in presubmit multiprocessing pool
2017-08-24 agable Don't send email from git-cl-description
2017-09-04 whesse Dart: Update the config for the "fetch dart" command.
2017-09-05 tikuta Add Nico and Scott to OWNERS for ninja related things
2017-09-02 dpranke Add a --batch mode to `git-cl owners`.
2017-09-04 tikuta Update ninja to v1.8.0.
2017-09-01 iannucci Update led tool to c9c1865b81113f02fd618259624170f59e2c832e.
2017-08-31 dnj [vpython] Update version.

Created with:
  roll-dep src/third_party/depot_tools
BUG= 771277 ,769369, 764087 , 741001 ,None,758725,741001


Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, see:
http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium#TOC-Failures-due-to-DEPS-rolls


TBR=phajdan.jr@chromium.org

Change-Id: Icb9ca53560ec8ee470e60fea5ba7fc23a764fb34
Reviewed-on: https://chromium-review.googlesource.com/701321
Reviewed-by: depot-tools-roller . <depot-tools-roller@chromium.org>
Commit-Queue: depot-tools-roller . <depot-tools-roller@chromium.org>
Cr-Commit-Position: refs/heads/master@{#506697}
[modify] https://crrev.com/41288f5ae4dd53e7d93e29340751e25b9cefe24c/DEPS

Hello Ian, could you please also add ppc64 and ppc64le to build?
like this: https://chromium-review.googlesource.com/c/infra/infra/+/661783 and https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/661787

Thanks,
John

RE #36: You open a new bug for discussion and possible implementation of that request.

RE #37, open https://bugs.chromium.org/p/chromium/issues/detail?id=773857 for discussion.

Thanks,
John