Use vpython for telemetry_perf_unittests |
|||||||||
Issue descriptionI'm having CQ trouble with two of the bots for https://chromium-review.googlesource.com/c/chromium/src/+/893539/16 - android_n5x_swarming_rel and win7_chromium_rel_ng Both bots are unable to find the module httplib2 (trace below) nednguyen@ thinks it has to do with the python version When I run the test on my linux machine, it looks for the module here: /usr/lib/python2.7/dist-packages/httplib2/ Is it possible to get that module or python version added to the failing bots? Traceback (most recent call last): <module> at /b/swarming/w/ir/tools/perf/record_wpr:24 sys.exit(record_wpr.Main(environment=config)) Main at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/record_wpr.py:285 _PrintAllBenchmarks(environment.top_level_dir, sys.stderr) _PrintAllBenchmarks at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/record_wpr.py:109 _PrintAllImpl(_GetSubclasses(base_dir, benchmark.Benchmark), 'benchmarks', _GetSubclasses at /b/swarming/w/ir/third_party/catapult/telemetry/telemetry/record_wpr.py:87 index_by_class_name=True) DiscoverClasses at /b/swarming/w/ir/third_party/catapult/common/py_utils/py_utils/discover.py:99 modules = DiscoverModules(start_dir, top_level_dir, pattern) DiscoverModules at /b/swarming/w/ir/third_party/catapult/common/py_utils/py_utils/discover.py:58 module = importlib.import_module(module_name) import_module at /usr/lib/python2.7/importlib/__init__.py:37 __import__(name) <module> at /b/swarming/w/ir/tools/perf/process_perf_results.py:10 from core import upload_results_to_perf_dashboard <module> at /b/swarming/w/ir/tools/perf/core/upload_results_to_perf_dashboard.py:18 from core import results_dashboard <module> at /b/swarming/w/ir/tools/perf/core/results_dashboard.py:14 import httplib2 ImportError: No module named httplib2
,
Feb 6 2018
I bump this to P1 because it's blocking Ashley from landing CL
,
Feb 6 2018
,
Feb 6 2018
You'll need to add infra/python/wheels/httplib2-py2_py3 to src/.vpython.
,
Feb 6 2018
-> ashleymarie, as #4 should be added in your CL
,
Feb 6 2018
I added the .vpython change but it still seems to fail the same test with the same error message https://chromium-review.googlesource.com/c/chromium/src/+/893539/23 https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/355830 Does the .vpython change need to be in a separate cl that gets checked in first? -> jbudorick, for the question
,
Feb 6 2018
The tryjob will pick up the vpython change immediately in the same CL. You can see the file in the input isolate for the failing task: (in the tarball 863809ed8fd43e474b8f71b264c3cc935c56dab2.tar) https://chromium-swarm.appspot.com/task?id=3b854c2c31748c10 I think the problem is that the task itself isn't using vpython. According to the first few log lines, it's running "Command: /usr/bin/python ../../testing/scripts/run_telemetry_as_googletest.py". /usr/bin/python is system python and isn't smart enough to read the .vpython spec.
,
Feb 7 2018
Ah, yeah, sorry. Misdiagnosis on my part. We should switch this to running in vpython. I won't have time to do so this week, though.
,
Feb 7 2018
,
Feb 7 2018
So to clarify this is a unittest that is invoking the script run_telemetry_as_googletest.py: https://cs.chromium.org/chromium/src/tools/perf/scripts_smoke_unittest.py?q=run_telemetry_benchmark_as_googletest&sq=package:chromium&dr=C&l=62 Therefore this is where we need to switch it to using vpython? Here is the CL where we had to do this on the recipe side when we ran into issues with http2: https://chromium-review.googlesource.com/c/chromium/tools/build/+/861911
,
Feb 7 2018
I think you can switch this to running vpython similar to how this script is doing it since you already added it to the .vpython script: https://chromium-review.googlesource.com/c/chromium/src/+/869512/14/testing/scripts/run_gpu_integration_test_as_googletest.py. It is a small change to scripts_smoke_unittest.py
,
Feb 7 2018
Anyone know who would be a good approver for the vpython change?
,
Feb 7 2018
,
Feb 9 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/500b8241fdb7b98b99d990a68c1807687f7eca6a commit 500b8241fdb7b98b99d990a68c1807687f7eca6a Author: Ashley Enstad <ashleymarie@chromium.org> Date: Fri Feb 09 19:40:57 2018 Adding vpython dependency for scripts_smoke_unittest tools/perf/core/results_dashboard imports httplib2 so we need to include that dependency when running tests that depend on results_dashboard. Bug: 758632 , 809563 Change-Id: Ic4ca05d9365ab80b95d1fa1886be07f0417844bb Reviewed-on: https://chromium-review.googlesource.com/908710 Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Emily Hanley <eyaich@chromium.org> Reviewed-by: Ned Nguyen <nednguyen@google.com> Cr-Commit-Position: refs/heads/master@{#535788} [modify] https://crrev.com/500b8241fdb7b98b99d990a68c1807687f7eca6a/.vpython [modify] https://crrev.com/500b8241fdb7b98b99d990a68c1807687f7eca6a/tools/perf/core/results_dashboard.py [modify] https://crrev.com/500b8241fdb7b98b99d990a68c1807687f7eca6a/tools/perf/scripts_smoke_unittest.py
,
Oct 18
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by ashleymarie@chromium.org
, Feb 6 2018