"No module named Crypto.Hash.SHA256" when running dev appserver on Swarming |
|||
Issue description
This breaks Swarming's local_smoke_test.py since it can't use local Isolate and blocks some commits.
From isolate server log:
ERROR 2017-09-26 01:52:05,871 wsgi.py:263]
Traceback (most recent call last):
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/infra/luci/appengine/isolate/main_frontend.py", line 29, in <module>
import handlers_frontend
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/infra/luci/appengine/isolate/handlers_frontend.py", line 19, in <module>
import gcs
File "/b/s/w/ir/cache/builder/Luci_py_Presubmit/infra/luci/appengine/isolate/gcs.py", line 19, in <module>
import Crypto.Hash.SHA256 as SHA256
ImportError: No module named Crypto.Hash.SHA256
INFO 2017-09-26 01:52:05,874 module.py:821] default: "POST /api/isolateservice/v1/preupload HTTP/1.1" 500 -
Happened here in particular https://ci.chromium.org/swarming/task/38d4a9d3ac4bda10?server=chromium-swarm.appspot.com (logs obtained through SSH).
It runs
infra/python/cpython/${platform}:version:2.7.13.chromium11
infra/tools/luci/vpython/${platform}:git_revision:e3ad4ddcc78dc0023cb4bc6972407961af03bd73
,
Sep 26 2017
I'll try to add .vpython spec to luci-py and make sure it is picked up when running presubmit tests.
,
Sep 26 2017
I don't know much about the Crypto package. Why not use "hashlib"?
,
Sep 26 2017
PyCrypto is required to run GAE's dev_appserver because PyCrypto is used on GAE (and it's the only available crypto package there). local_smoke_test.py starts two dev appservers (with Isolate and Swarming) and runs smoke tests against them. While we can replace SHA256, we still need RSA crypto (it is used to produce signed GS URLs), and there are no replacement for it.
,
Sep 26 2017
Gotcha. So situation before was: all bots have pyCrypto installed on them at system level, so interpreter gets it for free. With bundle, this is no longer the case?
,
Sep 26 2017
Yes. I'm fine with either solution; vpython spec or just installing it everywhere. Thanks!
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/e46ff7d15e4a0e2b932cff52918c24dbb5c50521 commit e46ff7d15e4a0e2b932cff52918c24dbb5c50521 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Sep 26 19:18:42 2017 Build pycrypto wheels for x86 Linux and OSX. They are needed to run GAE dev appserver. We assume only x86 Linux and OSX are supported platforms for that. R=iannucci@chromium.org, dnj@chromium.org BUG= 768660 Change-Id: I22b508495a2968d9163764ad7f830073125bb667 Reviewed-on: https://chromium-review.googlesource.com/683799 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/e46ff7d15e4a0e2b932cff52918c24dbb5c50521/infra/tools/dockerbuild/wheel.py
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/7d69d57e42bfc7958971f62248f3fa39e56232c6 commit 7d69d57e42bfc7958971f62248f3fa39e56232c6 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Sep 26 22:22:44 2017 Add vpython spec with PyCrypto for running GAE devserver for smoke test. R=iannucci@chromium.org, dnj@chromium.org BUG= 768660 Change-Id: I117d2dd1b8195c7e95383f30f74c99e7698483ef Reviewed-on: https://chromium-review.googlesource.com/685508 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [add] https://crrev.com/7d69d57e42bfc7958971f62248f3fa39e56232c6/.vpython
,
Sep 26 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/bb2bef261c4fe7480c3cf19f42ee77e969ad6410 commit bb2bef261c4fe7480c3cf19f42ee77e969ad6410 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Tue Sep 26 22:47:11 2017 Enable venv when running presubmits for luci-py. R=iannucci@chromium.org BUG= 768660 Change-Id: I87f51fbc867ded5c658625816d682c6e54b8cbdd Reviewed-on: https://chromium-review.googlesource.com/685360 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/bb2bef261c4fe7480c3cf19f42ee77e969ad6410/scripts/slave/README.recipes.md [add] https://crrev.com/bb2bef261c4fe7480c3cf19f42ee77e969ad6410/scripts/slave/recipes/run_presubmit.expected/luci-py.json [modify] https://crrev.com/bb2bef261c4fe7480c3cf19f42ee77e969ad6410/scripts/slave/recipes/run_presubmit.py
,
Sep 26 2017
|
|||
►
Sign in to add a comment |
|||
Comment 1 by vadimsh@chromium.org
, Sep 26 2017