Swarming: calculate total cost of a task's tree and expose this for parent task |
|||
Issue descriptionA task can have multiple children (>100) tasks. Each task has a cost associated to it. Calculating the total cost of the tree currently means fetching the results of every single task, which takes several seconds. This AI is to precalculate in the data store the sum of all the children tasks into the parent's task. The task could have two TaskRunResult. At the TaskResultSummary level, we want the whole cost. Storing the data inside the TaskRunResult is insufficient, it needs to be stored in the TaskResultSummary as a list. This is tricky as this happens when the task is completed, not at task creation, which means that this will double the number of transactions in the parent tasks. An option is to store each children task data as a child entity to the TaskResultSummary, so summation can be done more easily, without any Tx needed when writing nor reading. This is purely an append-only operation. As a matter of fact, we could move the _TaskResultCommon.children_task_ids too, so this would further remove another Tx in the child task creation process. This opens the door to higher reliability in a workflow (reentrancy) that is becoming more and more common.
,
Jun 9 2018
,
Jun 11 2018
Extending to include swarming total duration and total overhead. So it really needs to be an entity, that can then be merged into the result entity as a sum.
,
Oct 15
As for overhead, it would enable calculating the efficiency as per https://en.wikipedia.org/wiki/Amdahl%27s_law. |
|||
►
Sign in to add a comment |
|||
Comment 1 by mar...@chromium.org
, Jun 9 2018