Change swarming collect recipe to not print stdout |
||||
Issue descriptionWhen a Swarming task's stdout is collected, it is effectively output twice, as the annotator reprints it another time. In practice we do not need the output anymore, this used to be so users wouldn't be confused in buildbot world but this is just overhead for LUCI tasks or reentrant task. In particular this can become a real issue when the total output is larger than 16Mb, which affects the task. AIs: - Add a new flag in swarming.py collect to not fetch stdout[1]. - Update recipe_module/swarming/api.py to use this new flag. [1] While at it, we may want to not return the task process exit code either.
,
Sep 7 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/adaf48426621c897d1bd094cd73906d7bfab3d50 commit adaf48426621c897d1bd094cd73906d7bfab3d50 Author: Tim 'mithro' Ansell <tansell@chromium.org> Date: Thu Sep 07 23:41:56 2017 swarming.py: Add --task-output-stdout option Enables the ability to control where the stdout of a task ends up going. The options are; * Don't download / get at all. * Only output to the json summary file. * Only output to the console. * Continue to output in all of the above (default). At the moment when using swarming on the build bots we end up with; - stdout in swarming - stdout in the collect_task output - stdout inside the summary.json file For Layout Tests where the stdout is *long* this makes recipes take significant time in just parsing the summary.json output. BUG= 731095 Change-Id: I0fa7228a5be21820b3089306f6e9c8646983159c Reviewed-on: https://chromium-review.googlesource.com/654020 Reviewed-by: Tim 'mithro' Ansell <tansell@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org> [modify] https://crrev.com/adaf48426621c897d1bd094cd73906d7bfab3d50/client/swarming.py [modify] https://crrev.com/adaf48426621c897d1bd094cd73906d7bfab3d50/client/tests/swarming_test.py [modify] https://crrev.com/adaf48426621c897d1bd094cd73906d7bfab3d50/client/tools/swarming_load_test_client.py
,
Sep 8 2017
With https://chromium-review.googlesource.com/654020 we now have a way to do this; swarming.py --task-output-stdout=(console|json|none|both) I'm planning on deploying this feature for Layout Tests on swarming soon.
,
Sep 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/519e4311ba5a076a7102b48e9efdbdcda2ed06e1 commit 519e4311ba5a076a7102b48e9efdbdcda2ed06e1 Author: Tim 'mithro' Ansell <tansell@chromium.org> Date: Fri Sep 08 02:26:19 2017 Roll src/tools/swarming_client/ 72b6a2dc6..5e8001d9a (7 commits) https://chromium.googlesource.com/infra/luci/client-py.git/+log/72b6a2dc6046..5e8001d9a710 $ git log 72b6a2dc6..5e8001d9a --date=short --no-merges --format='%ad %ae %s' 2017-09-08 tansell swarming.py: Add --task-output-stdout option 2017-09-07 justincohen tools/run_on_bots: Enable running a task in batches. 2017-08-31 tansell Removing arfile. 2017-08-31 tansell tools/run_on_bots: Enable giving a hash instead of script. 2017-08-30 aludwin Implement task updates in gRPC proxy 2017-08-29 justincohen Add support for no bots found in run_on_bots.py 2017-08-17 maruel More improvements to run_on_bots.py Created with: roll-dep src/tools/swarming_client R=maruel@chromium.org,dpranke@chromium.org BUG= 731095 Change-Id: I0d53fd7f17d3b24cc37a24f230a85c7b5b3797da Reviewed-on: https://chromium-review.googlesource.com/656401 Commit-Queue: Tim 'mithro' Ansell <tansell@chromium.org> Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Commit-Position: refs/heads/master@{#500489} [modify] https://crrev.com/519e4311ba5a076a7102b48e9efdbdcda2ed06e1/DEPS
,
Oct 13 2017
,
Jun 21 2018
|
||||
►
Sign in to add a comment |
||||
Comment 1 by rmis...@google.com
, Jun 8 2017