Provision throttling workqueue fails to account for aborted tasks
Reported by
jrbarnette@chromium.org,
Sep 20 2017
|
||
Issue descriptionThe main processing loop in lib/workqueue/service.py keeps a running count of the total number of tasks that have started but not yet finished. It logs that number, and also reports a gauge metric to keep track. The running count fails to account for tasks that are aborted. So, every time a task aborts, the running count goes up by one permanently. The accounting needs to be fixed.
,
Sep 21 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/chromite/+/72c1b4d0673cd0fb967b4541b3efe349c266eccf commit 72c1b4d0673cd0fb967b4541b3efe349c266eccf Author: Richard Barnette <jrbarnette@chromium.org> Date: Thu Sep 21 08:08:04 2017 [workqueue] Fix running task count accounting. In the workqueue server, the number of running tasks was kept as a running count, and adjusted directly inside `ProcessRequests()`. Aborted tasks weren't accounted for, causing the count to slowly increase over time. This fixes things by moving the accounting to TaskManager. BUG= chromium:767171 TEST=unit tests Change-Id: I8b8f3e416ff312385037bf5445b04e0d397755bb Reviewed-on: https://chromium-review.googlesource.com/676185 Commit-Ready: Richard Barnette <jrbarnette@chromium.org> Tested-by: Richard Barnette <jrbarnette@chromium.org> Reviewed-by: Paul Hobbs <phobbs@google.com> [modify] https://crrev.com/72c1b4d0673cd0fb967b4541b3efe349c266eccf/lib/workqueue/tasks_unittest.py [modify] https://crrev.com/72c1b4d0673cd0fb967b4541b3efe349c266eccf/lib/workqueue/service_unittest.py [modify] https://crrev.com/72c1b4d0673cd0fb967b4541b3efe349c266eccf/lib/workqueue/service.py [modify] https://crrev.com/72c1b4d0673cd0fb967b4541b3efe349c266eccf/lib/workqueue/tasks.py
,
Sep 27 2017
|
||
►
Sign in to add a comment |
||
Comment 1 by jrbarnette@chromium.org
, Sep 20 2017