Swarming bot: make task_runner ping more often with exponential backoff |
||
Issue descriptiontask_running only ping on one of these conditions: - buffered stdout is large (100kb) - 10s elapsed and some stdout/stderr happened - 30s otherwise For short task, it'd be nice to update more frequently. This is especially getting in the way of local_smoke_test test_cancel_running, as the bot is not pinging often enough, slowing the test down. See _OutputBuffer in: https://cs.chromium.org/chromium/infra/luci/appengine/swarming/swarming_bot/bot_code/task_runner.py
,
Mar 26 2018
To be clear, it only affects on-going tasks. Once a task completes, there's no delay.
,
Dec 1
A quick comment: One potential optimization would be for bots to update more frequently only if there is an "active_viewer" watching the output. For the (common) case where no-one is polling/watching the logs the bot could write at the current rate (or even slower). I suspect case where someone is actively watching is fairly rare so it shouldn't impact the system load much. Nest used something like this (to poll devices at a faster rate for room temperature/other stats) when there is an active client, and allowing the devices to sleep longer otherwise. It would require some way to track if there is an "active_viewer", I don't know how complex that would be.
,
Dec 1
What would be a reasonable latency for you? Would 5 sec be fine?
,
Dec 5
+Eric for confirmation
,
Dec 6
I'm not the original requester (I found this ticket while reading swarming documentation). But, I'll put in my $0.02! In Jenkins it looks like the console is updated at about a once per second rate, and that feels "live/prompt". I wasn't able to quickly check bamboo real-time logs but IIRC it was about the same rate. Here is a little 10 sec clip (viewable if you are an FTE) showing the update rate. https://x20web.corp.google.com/users/el/eleenest/temp_shared/181205_jenkins_console_update_speed.mov So, being updated once per sec is "great". Updating once every 5 sec would probably still feel slow to a human viewer. But an improvement. Eric |
||
►
Sign in to add a comment |
||
Comment 1 by iannucci@chromium.org
, Mar 26 2018