New issue
Advanced search Search tips

Issue 851271 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocking:
issue 851272
issue 851273



Sign in to add a comment

Swarming: calculate total cost of a task's tree and expose this for parent task

Project Member Reported by mar...@chromium.org, Jun 9 2018

Issue description

A task can have multiple children (>100) tasks. Each task has a cost associated to it. Calculating the total cost of the tree currently means fetching the results of every single task, which takes several seconds.

This AI is to precalculate in the data store the sum of all the children tasks into the parent's task.

The task could have two TaskRunResult. At the TaskResultSummary level, we want the whole cost.

Storing the data inside the TaskRunResult is insufficient, it needs to be stored in the TaskResultSummary as a list.
 
This is tricky as this happens when the task is completed, not at task creation, which means that this will double the number of transactions in the parent tasks.

An option is to store each children task data as a child entity to the TaskResultSummary, so summation can be done more easily, without any Tx needed when writing nor reading. This is purely an append-only operation.

As a matter of fact, we could move the _TaskResultCommon.children_task_ids too, so this would further remove another Tx in the child task creation process. This opens the door to higher reliability in a workflow (reentrancy) that is becoming more and more common.
 
Blocking: 851272
Blocking: 851273

Comment 3 by mar...@chromium.org, Jun 11 2018

Extending to include swarming total duration and total overhead. So it really needs to be an entity, that can then be merged into the result entity as a sum.
Summary: Swarming: calculate total cost of a task's tree and expose this for parent task (was: Swarming: precalculate total cost of a task's tree)
As for overhead, it would enable calculating the efficiency as per https://en.wikipedia.org/wiki/Amdahl%27s_law.

Sign in to add a comment