"swarming.py reproduce" crashes |
||||
Issue descriptionTrying to reproduce this locally: https://chromium-swarm.appspot.com/task?id=3af44789a4596410&refresh=10&show_raw=1 Running: python tools\swarming_client\swarming.py reproduce -S chromium-swarm.appspot.com 3af44789a4596410 Throws exception: Traceback (most recent call last): File "tools\swarming_client\swarming.py", line 1772, in <module> sys.exit(main(sys.argv[1:])) File "tools\swarming_client\swarming.py", line 1764, in main return dispatcher.execute(OptionParserSwarming(version=__version__), args) File "C:\Users\kbr\src\chrome\src\tools\swarming_client\third_party\depot_tools\subcommand.py", line 251, in execute return command(parser, args[1:]) File "tools\swarming_client\swarming.py", line 1582, in CMDreproduce env_prefixes = properties['env'] KeyError: 'env' Sending the crash report ... done. Report URL: https://chromium-swarm.appspot.com/restricted/ereporter2/errors/6117571914366976 Process exited due to exception 'env' There's a bug in the code which handles "env_prefixes". It fetches properties['env'] instead of properties['env_prefixes'].
,
Jan 9 2018
Note: after patching in this fix locally it still crashes:
Traceback (most recent call last):
File "tools\swarming_client\swarming.py", line 1772, in <module>
sys.exit(main(sys.argv[1:]))
File "tools\swarming_client\swarming.py", line 1764, in main
return dispatcher.execute(OptionParserSwarming(version=__version__), args)
File "C:\Users\kbr\src\chrome\src\tools\swarming_client\third_party\depot_tools\subcommand.py", line 251, in execute
return command(parser, args[1:])
File "tools\swarming_client\swarming.py", line 1584, in CMDreproduce
for key, paths in env_prefixes.iteritems():
AttributeError: 'list' object has no attribute 'iteritems'
Sending the crash report ... done.
Report URL: https://chromium-swarm.appspot.com/restricted/ereporter2/errors/5637427017809920
Process exited due to exception
'list' object has no attribute 'iteritems'
,
Jan 9 2018
yeah, fixing.
,
Jan 9 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/7f12a449ca611184098f6fc4bf9b8c09fae1eb62 commit 7f12a449ca611184098f6fc4bf9b8c09fae1eb62 Author: Marc-Antoine Ruel <maruel@chromium.org> Date: Tue Jan 09 19:11:12 2018 client: fix swarming.py reproduce with env_prefixes. - Remove encoding, it's unnecessary now that subprocess42 is SUPER SMART. :) - Add a small unit test. R=iannucci@chromium.org Bug: 800413 Change-Id: I4199c66146efb61bcb591714bae9350838da2f13 Reviewed-on: https://chromium-review.googlesource.com/857937 Reviewed-by: Robbie Iannucci <iannucci@chromium.org> Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> [modify] https://crrev.com/7f12a449ca611184098f6fc4bf9b8c09fae1eb62/client/swarming.py [modify] https://crrev.com/7f12a449ca611184098f6fc4bf9b8c09fae1eb62/client/tests/swarming_test.py
,
Jan 9 2018
Thanks for this fix but it's still not working for me. python tools\swarming_client\swarming.py reproduce -S chromium-swarm.appspot.com 3af44789a4596410 265 files remaining... 227 files remaining... 189 files remaining... 166 files remaining... 148 files remaining... 99 files remaining... Traceback (most recent call last): File "tools\swarming_client\swarming.py", line 1786, in <module> sys.exit(main(sys.argv[1:])) File "tools\swarming_client\swarming.py", line 1778, in main return dispatcher.execute(OptionParserSwarming(version=__version__), args) File "C:\Users\kbr\src\chrome\src\tools\swarming_client\third_party\depot_tools\subcommand.py", line 251, in execute return command(parser, args[1:]) File "tools\swarming_client\swarming.py", line 1651, in CMDreproduce return subprocess.call(command + extra_args, env=env, cwd=workdir) File "C:\python_27_amd64\files\lib\subprocess.py", line 493, in call return Popen(*popenargs, **kwargs).wait() File "C:\python_27_amd64\files\lib\subprocess.py", line 679, in __init__ errread, errwrite) File "C:\python_27_amd64\files\lib\subprocess.py", line 896, in _execute_child startupinfo) TypeError: environment can only contain strings Sending the crash report ... done. Report URL: https://chromium-swarm.appspot.com/restricted/ereporter2/errors/5673908839120896 Process exited due to exception environment can only contain strings Output of "set" in my Command Prompt (Google internal only in case it has sensitive information): https://drive.google.com/file/d/1aaqTnIXb1s0lNw8RY-xPBC-cSzDHIWow/view?usp=sharing
,
Jan 10 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/0795e6510d23a472da796420c93c1e63dbaf2923 commit 0795e6510d23a472da796420c93c1e63dbaf2923 Author: Marc-Antoine Ruel <maruel@chromium.org> Date: Wed Jan 10 19:28:23 2018 client: use subproces42 instead of subprocess subprocess has a lot of issues, one being lack of unicode support. The rest of the code uses subprocess42 but for an unknown reason, 'swarming.py reproduce' did not. Obviously, 'M-A from 2015' is at fault. R=iannucci@chromium.org Bug: 800413 Change-Id: I3f8b6e3e07bee4156ba217a6e930371b62d0bb3d Reviewed-on: https://chromium-review.googlesource.com/860260 Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/0795e6510d23a472da796420c93c1e63dbaf2923/client/swarming.py [modify] https://crrev.com/0795e6510d23a472da796420c93c1e63dbaf2923/client/tests/swarming_test.py
,
Jan 10 2018
Thanks for the fixes. Confirming that swarming.py reproduce now runs the test, though it doesn't pass because some of the arguments cause the test to try to write to a nonexistent directory:
[1/1] gpu_tests.maps_integration_test.MapsIntegrationTest.Maps_maps passed 12.0930s
INFO:root:Closing browser (pid=16360) ...
[16360:18556:0110/115100.598:INFO:chrome_cryptauth_service.cc(233)] Profile is not authenticated yet; waiting before starting CryptAuth managers.
[16360:18556:0110/115100.741:WARNING:url_request_context_getter.cc(43)] URLRequestContextGetter leaking due to no owning thread.
INFO:root:Successfully shut down browser cooperatively
INFO:root:Browser is closed.
1 test passed in 21.4s, 0 skipped, 0 failures.
INFO:root:Try printing formatted exception: <type 'exceptions.IOError'> [Errno 2] No such file or directory: 'out\\output.json' <traceback object at 0x00000000047AA408>
Traceback (most recent call last):
<module> at C:\Users\kbr\src\chrome\src\work\third_party\catapult\telemetry\telemetry\testing\run_browser_tests.py:359
ret_code = RunTests(sys.argv[1:])
RunTests at C:\Users\kbr\src\chrome\src\work\third_party\catapult\telemetry\telemetry\testing\run_browser_tests.py:328
ret, _, _ = runner.run()
run at C:\Users\kbr\src\chrome\src\work\third_party\catapult\third_party\typ\typ\runner.py:189
self._write(self.args.write_full_results_to, full_results)
_write at C:\Users\kbr\src\chrome\src\work\third_party\catapult\third_party\typ\typ\runner.py:656
self.host.write_text_file(path, json.dumps(obj, indent=2) + '\n')
write_text_file at C:\Users\kbr\src\chrome\src\work\third_party\catapult\third_party\typ\typ\host.py:179
return self._write(path, contents, mode='w')
_write at C:\Users\kbr\src\chrome\src\work\third_party\catapult\third_party\typ\typ\host.py:185
with open(path, mode) as f:
IOError: [Errno 2] No such file or directory: 'out\\output.json'
Locals:
contents : '{\n "version": 3, \n "interrupted": false, \n "path_delimiter": ".", \n "seconds_since_epoch": 1515613860, \n "num_failures_by_type": {\n "FAIL": 0, \n "PASS": 1, \n "SKIP": 0\n }, \n "tests": {\n "gpu_tests": {\n "maps_integration_test": {\n "MapsIntegrationTest": {\n "Maps_maps": {\n "actual": "PASS", \n "times": [\n 12.093\n ], \n "expected": "PASS"\n }\n }\n }\n }\n }\n}\n'
mode : 'w'
path : 'out\\output.json'
Closing as Verified -- if we think "swarming.py reproduce" needs more features I'll file a follow-on bug.
,
Jan 10 2018
,
Jan 11 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/72a97881dd6b6a6e0147d1cad984bdada22201c0 commit 72a97881dd6b6a6e0147d1cad984bdada22201c0 Author: Marc-Antoine Ruel <maruel@chromium.org> Date: Thu Jan 11 15:15:45 2018 Roll src/tools/swarming_client/ 36e0979a4..88229872d (3 commits) https://chromium.googlesource.com/infra/luci/client-py.git/+log/36e0979a4f2f..88229872dd17 $ git log 36e0979a4..88229872d --date=short --no-merges --format='%ad %ae %s' 2018-01-10 maruel client: fix incorrect 'fix' in 7efbe3ad731c875. 2018-01-10 maruel client: create the output directory directory when ${ISOLATED_OUTDIR} is used 2018-01-10 maruel client: use subproces42 instead of subprocess Created with: roll-dep src/tools/swarming_client R=kbr@chromium.org Bug: 800413 , 800899 Change-Id: Ic9372e8ba63fb5be66902bc9a4c6658f9ea307df Reviewed-on: https://chromium-review.googlesource.com/861123 Reviewed-by: Kenneth Russell <kbr@chromium.org> Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#528627} [modify] https://crrev.com/72a97881dd6b6a6e0147d1cad984bdada22201c0/DEPS |
||||
►
Sign in to add a comment |
||||
Comment 1 by kbr@chromium.org
, Jan 9 2018