Issue metadata
Sign in to add a comment
|
All shards of telemetry_perf_unittests failing with internal Swarming failure |
||||||||||||||||||||||
Issue descriptionIn this tryjob: https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/34953 of this CL which should be unrelated: https://chromium-review.googlesource.com/c/chromium/src/+/754019 All shards of telemetry_perf_unittests failed with an internal Swarming failure. The shard exited with code 0: https://chromium-swarm.appspot.com/task?id=399c909198029f10&refresh=10&show_raw=1 so there doesn't seem a good reason why it failed. Can someone help? Note I'm not seeing evidence of this elsewhere on the bot: https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/?limit=200 so it might be my patch, but I don't see how. Thanks.
,
Nov 3 2017
It looks like the collect step is failing with
Traceback (most recent call last):
File "C:\b\c\b\win\src\tools\swarming_client\swarming.py", line 1727, in <module>
sys.exit(main(sys.argv[1:]))
File "C:\b\c\b\win\src\tools\swarming_client\swarming.py", line 1719, in main
return dispatcher.execute(OptionParserSwarming(version=__version__), args)
File "C:\b\c\b\win\src\tools\swarming_client\third_party\depot_tools\subcommand.py", line 251, in execute
return command(parser, args[1:])
File "C:\b\c\b\win\src\tools\swarming_client\swarming.py", line 1324, in CMDcollect
options.perf)
File "C:\b\c\b\win\src\tools\swarming_client\swarming.py", line 768, in collect
"console" in task_output_stdout).encode(
File "C:\b\c\b\win\src\tools\swarming_client\swarming.py", line 717, in decorate_shard_output
metadata.get('output', '').rstrip(),
AttributeError: 'dict' object has no attribute 'rstrip'
,
Nov 3 2017
https://apis-explorer.appspot.com/apis-explorer/?base=https://chromium-swarm.appspot.com/_ah/api#p/swarming/v1/swarming.task.result?task_id=399c909198029f10&_h=1& shows what I believe the swarming.py script is doing to get the metadata object which is being referenced in the stack trace. It doesn't have an output element though.
,
Nov 3 2017
https://chromium-review.googlesource.com/c/infra/luci/luci-py/+/753846 should address https://bugs.chromium.org/p/chromium/issues/detail?id=781421#c2, not sure how to check if it's made it onto the bots though.
,
Nov 3 2017
The CL in #4 looks to be very related to the exception we're seeing.
,
Nov 3 2017
That CL landed this morning, and https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/34953 started at 1:30 PM. So it shouldn't be failing like that. I don't think that CL has been rolled out. Do we need to roll the swarming client in chromium src?
,
Nov 3 2017
yeah, the current swarming revision in chromium is https://chromium.googlesource.com/infra/luci/client-py/+/5da404cf35b6541f16d8bd6cc3e506df1fda8021
,
Nov 3 2017
I'll do a roll.
,
Nov 3 2017
https://chromium-review.googlesource.com/c/chromium/src/+/753138 is the roll. Should I NOTRY it? I don't know how much this is affecting the bot. The bot doesn't seem to be all purple.
,
Nov 3 2017
No.
,
Nov 3 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/1c1c8f97645c9608fac1f74401eddf162b7df2e0 commit 1c1c8f97645c9608fac1f74401eddf162b7df2e0 Author: Vadim Shtayura <vadimsh@chromium.org> Date: Fri Nov 03 22:39:06 2017 Fix handling of empty /stdout result. If /stdout returns an empty dict (which happens if there's no stdout), result['output'] becomes empty dict too, which later trips .rstrip(...) call. BUG= 781421 R=martiniss@chromium.org Change-Id: I9b0086b6686bdc5073fbe30452a2d72532119469 Reviewed-on: https://chromium-review.googlesource.com/754284 Reviewed-by: Stephen Martinis <martiniss@chromium.org> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/1c1c8f97645c9608fac1f74401eddf162b7df2e0/client/swarming.py
,
Nov 4 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/a075effafacc793a2902cf534fed98f0d4ffc5d6 commit a075effafacc793a2902cf534fed98f0d4ffc5d6 Author: Stephen Martinis <martiniss@chromium.org> Date: Sat Nov 04 01:31:13 2017 Roll src/tools/swarming_client/ 5da404cf3..6fd3c7b6e (2 commits) https://chromium.googlesource.com/infra/luci/client-py.git/+log/5da404cf35b6..6fd3c7b6eb7c $ git log 5da404cf3..6fd3c7b6e --date=short --no-merges --format='%ad %ae %s' 2017-11-03 vadimsh Fix handling of empty /stdout result. 2017-11-03 maruel swarming.py: fix regression in adaf484266, improve decorated ouput Created with: roll-dep src/tools/swarming_client R=vadimsh@chromium.org Bug: 781421 Change-Id: I8240c90e86a092e5deaf4d6b737ac95fea962c15 Reviewed-on: https://chromium-review.googlesource.com/754293 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Stephen Martinis <martiniss@chromium.org> Cr-Commit-Position: refs/heads/master@{#514006} [modify] https://crrev.com/a075effafacc793a2902cf534fed98f0d4ffc5d6/DEPS
,
Nov 7 2017
This seems to have been fixed, AFAIK. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by jbudorick@chromium.org
, Nov 3 2017