Make wait_for_tasks work with large numbers of tasks |
|||
Issue descriptionhttps://ci.chromium.org/swarming/task/4251facd41598f10?server=chrome-swarming.appspot.com fails because there are ~470 tasks it tries to collect. wait_for_finished_task_set.py currently tries to fetch all of these from the swarming server, which generates a URL that's over 10,000 characters long, which seems to break the swarming server. The script should be able to split up the requests, since it knows each individual task it's trying to fetch.
,
Jan 10
,
Jan 10
Thanks a lot for working on it Stephen!
,
Jan 11
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build/+/c52a9790db08f46051e02a8da12b77caf3f04a5b commit c52a9790db08f46051e02a8da12b77caf3f04a5b Author: Stephen Martinis <martiniss@chromium.org> Date: Fri Jan 11 18:24:16 2019 Batch tasks to wait for wait_for_finished_task_set.py currently requests task status from swarming for every task it's supposed to wait for. This is problematic if the script is supposed to wait for hundreds of tasks, as there are limits on URL length in app engine. This CL changes the script to batch the requests into sets of 300 tasks, which is small enough to work. Bug: 920724 Change-Id: Icb3d0169eb3c95c7ac05c2508a1e28b579f4f426 Reviewed-on: https://chromium-review.googlesource.com/c/1405618 Commit-Queue: Stephen Martinis <martiniss@chromium.org> Commit-Queue: Garrett Beaty <gbeaty@chromium.org> Auto-Submit: Stephen Martinis <martiniss@chromium.org> Reviewed-by: Garrett Beaty <gbeaty@chromium.org> Reviewed-by: Shuotao Gao <stgao@chromium.org> Reviewed-by: Yuke Liao <liaoyuke@chromium.org> [modify] https://crrev.com/c52a9790db08f46051e02a8da12b77caf3f04a5b/scripts/slave/recipe_modules/swarming/resources/wait_for_finished_task_set.py [modify] https://crrev.com/c52a9790db08f46051e02a8da12b77caf3f04a5b/scripts/slave/recipe_modules/swarming/unittests/wait_for_finished_task_set_unittest.py
,
Jan 14
|
|||
►
Sign in to add a comment |
|||
Comment 1 by martiniss@chromium.org
, Jan 10