chromium_presubmit does not have psutil python module in presubmit step |
|||
Issue descriptionWhen I tried to update third_party library, I see presubmit error. https://chromium-review.googlesource.com/c/chromium/src/+/917930 https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/57342 ~~~ snip ~~~ /b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_instrumentation_test_run_test.py (0.39s) failed Traceback (most recent call last): File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_instrumentation_test_run_test.py", line 15, in <module> from pylib.local.device import local_device_instrumentation_test_run File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_instrumentation_test_run.py", line 31, in <module> from pylib.local.device import local_device_test_run File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_test_run.py", line 16, in <module> from devil.android.tools import device_recovery File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 11, in <module> import psutil ImportError: No module named psutil /b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_test_run_test.py (0.08s) failed Traceback (most recent call last): File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_test_run_test.py", line 12, in <module> from pylib.local.device import local_device_test_run File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/build/android/pylib/local/device/local_device_test_run.py", line 16, in <module> from devil.android.tools import device_recovery File "/b/swarming/w/ir/cache/builder/chromium_presubmit/src/third_party/catapult/devil/devil/android/tools/device_recovery.py", line 11, in <module> import psutil ImportError: No module named psutil Presubmit checks took 6.6s to calculate. step returned non-zero exit code: 1 But I think the error is not come from my change and suspecting this is related to vpython. iannucci@, do you know something?
,
Mar 14 2018
Pretty easy to repro locally, doing some diagnosis
,
Mar 14 2018
Looks like these have a shebang line to use 'python', which will pick up the .vpython environment from depot_tools. Switching it to 'vpython' fixes the issue. https://chromium-review.googlesource.com/c/chromium/src/+/961863
,
Mar 14 2018
I think we'll likely want to change presubmit_support in depot_tools to always use vpython when invoking sub-python stuff. I'll prep a CL for that as well.
,
Mar 14 2018
Here's a CL to make presubmit_support behave better in these sorts of scenarios: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/961865
,
Mar 14 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/6d9aeed366502f66f626059308328b9351e5ad46 commit 6d9aeed366502f66f626059308328b9351e5ad46 Author: Robert Iannucci <iannucci@chromium.org> Date: Wed Mar 14 07:15:18 2018 Use vpython for pylib tests. This way they can find psutil on dev machines as well as bots. R=skyostil@chromium.org, tikuta@chromium.org Bug: 821669 Change-Id: I7d53409ea9e62597f9edcea472dbec754d929687 Reviewed-on: https://chromium-review.googlesource.com/961863 Reviewed-by: John Budorick <jbudorick@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Cr-Commit-Position: refs/heads/master@{#543023} [modify] https://crrev.com/6d9aeed366502f66f626059308328b9351e5ad46/build/android/pylib/local/device/local_device_instrumentation_test_run_test.py [modify] https://crrev.com/6d9aeed366502f66f626059308328b9351e5ad46/build/android/pylib/local/device/local_device_test_run_test.py
,
Mar 14 2018
,
Mar 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/5025893cb6061b8a219750155cccd67b55079311 commit 5025893cb6061b8a219750155cccd67b55079311 Author: Robert Iannucci <iannucci@chromium.org> Date: Thu Mar 22 20:35:06 2018 [presubmit_support] Prevent depot_tools' virtualenv from leaking into other repos. Currently in LUCI `git cl` is invoked on chromium_presubmit using vpython. This means that it operates under the depot_tools .vpython environment (which is blank). Some features in presubmit_support allow invocation of python subprocesses, and previously they would use `sys.executable` (especially on windows, but occasionally on non-windows as well). In non-vpython environments, this just picks up the system python and whatever modules happen to be installed there. Some python unittests work around this on non-windows systems by having a shebang line which explicitly invokes vpython. This CL changes presubmit_support so that invocations of 'python', or executions of '.py' scripts will always end up invoking vpython. In the best case, this will pick up the invoked scripts' vpython environment. In the worst case, this will invoke the script under an empty vpython environment (aka "stock python"). R=dpranke@chromium.org, jbudorick@chromium.org, tandrii@chromium.org, tikuta@chromium.org Bug: 821669 Change-Id: I5d2d5dfd0364022d56833c2c8af4983553a29c7a Reviewed-on: https://chromium-review.googlesource.com/961865 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/5025893cb6061b8a219750155cccd67b55079311/presubmit_canned_checks.py [modify] https://crrev.com/5025893cb6061b8a219750155cccd67b55079311/presubmit_support.py [modify] https://crrev.com/5025893cb6061b8a219750155cccd67b55079311/tests/presubmit_unittest.py
,
Mar 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools/+/e8703543e3930885d28e561bcf2ab9638b7eeb45 commit e8703543e3930885d28e561bcf2ab9638b7eeb45 Author: Robert Iannucci <iannucci@chromium.org> Date: Thu Mar 22 20:54:37 2018 Trigger recipe roll to pick up 5025893cb6061b8a219750155cccd67b55079311. To make sure that the following CL actually rolls into downstream recipes: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/961865 TBR=nodir@chromium.org, vadimsh@chromium.org Bug: 821669 Change-Id: Ia530a619e355291a15d849e09c449fc82e31534b Reviewed-on: https://chromium-review.googlesource.com/976558 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/e8703543e3930885d28e561bcf2ab9638b7eeb45/recipes/trigger_recipe_roller.txt
,
Mar 23 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f311c119ab164b10b083c09fe4ade377cd348c18 commit f311c119ab164b10b083c09fe4ade377cd348c18 Author: depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Date: Thu Mar 22 22:49:15 2018 Roll src/third_party/depot_tools/ c621b21ad..88f9c40e0 (3 commits) https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/c621b21ad4bf..88f9c40e0cce $ git log c621b21ad..88f9c40e0 --date=short --no-merges --format='%ad %ae %s' 2018-03-22 ehmaldonado gclient eval: Expand vars while parsing DEPS files 2018-03-22 iannucci Trigger recipe roll to pick up 5025893cb6061b8a219750155cccd67b55079311. 2018-03-19 iannucci [presubmit_support] Prevent depot_tools' virtualenv from leaking into other repos. Created with: roll-dep src/third_party/depot_tools BUG= chromium:821199 , chromium:821669 , chromium:821669 The AutoRoll server is located here: https://depot-tools-chromium-roll.skia.org Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+/master/autoroll/README.md If the roll is causing failures, please contact the current sheriff, who should be CC'd on the roll, and stop the roller if necessary. TBR=agable@chromium.org Change-Id: I8471b021376c10b08dd3241f5083c1be87ea6c85 Reviewed-on: https://chromium-review.googlesource.com/976625 Reviewed-by: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Commit-Queue: depot-tools-chromium-autoroll <depot-tools-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com> Cr-Commit-Position: refs/heads/master@{#545276} [modify] https://crrev.com/f311c119ab164b10b083c09fe4ade377cd348c18/DEPS
,
Jun 15 2018
Coming back from leave; IIRC, this issue was fixed. Please reopen if not. |
|||
►
Sign in to add a comment |
|||
Comment 1 by iannucci@chromium.org
, Mar 14 2018