Swarming: When polling an expired TaskToRun, cancel it immediately |
||||||
Issue descriptionWhen TaskToRun global index is stale, it's possible that _validate_task_async() has to loop over a lot of expired items. It should set the negative cache entry while at it, so that other polling loops are less affected, which should help scanning speed. i.e. call set_lookup_cache(task_key, False) while logging about expired task.
,
Feb 16 2018
Changing the title for a better idea. Basically, as the bots are polling, they will expire tasks concurrently, accelerating the rate of stale task cleanup.
,
Feb 21 2018
,
Feb 21 2018
,
Mar 20 2018
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/a98580c6c313b4e5677c87f5bbb1feaaa6f67c98 commit a98580c6c313b4e5677c87f5bbb1feaaa6f67c98 Author: Marc-Antoine Ruel <maruel@chromium.org> Date: Tue Mar 20 14:00:44 2018 Swarming: expire tasks as they are polled This means polling is slower, because bots poll handlers are "wasting" time expiring tasks inline, but in practice it's "faster" for three reasons: - Expired task is added to the negative cache, so other bots skip it. - Tasks are expired faster, which is necessary for TaskProperties fallback. - Reduce the load on the cron job, which can overflow since it's not expiring that many task per second, which is a problem when a lot of tasks expires at once or a build up occurs due to "force_bots_to_sleep_and_not_run_task" being used. Includes a unit test to confirm it actually works. R=vadimsh@chromium.org Bug: 812886 Change-Id: I2e15a39af0273982cf360a514b137ccb0ed9111c Reviewed-on: https://chromium-review.googlesource.com/927644 Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> [modify] https://crrev.com/a98580c6c313b4e5677c87f5bbb1feaaa6f67c98/appengine/swarming/server/task_scheduler.py [modify] https://crrev.com/a98580c6c313b4e5677c87f5bbb1feaaa6f67c98/appengine/swarming/server/task_scheduler_test.py [modify] https://crrev.com/a98580c6c313b4e5677c87f5bbb1feaaa6f67c98/appengine/swarming/server/task_to_run.py [modify] https://crrev.com/a98580c6c313b4e5677c87f5bbb1feaaa6f67c98/appengine/swarming/server/task_to_run_test.py
,
Mar 22 2018
,
Jun 2 2018
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by mar...@chromium.org
, Feb 15 2018