New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 729565 link

Starred by 3 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature


Sign in to add a comment

Create a duration estimator by continuously doing a regression on tags->duration Swarming task data

Project Member Reported by mar...@chromium.org, Jun 5 2017

Issue description

Assumptions:
- Swarming tasks have tags that helps "describe" the content of the task or at least, bucket them. For example, buildername, stepname, etc.
- Each dimension key value pair is automatically added as a tag, so these can be leveraged implicitly.
- There's a correlation overtime with a subset of the task tags and the task's duration.

Goals:
- Create a query engine that for a given list of tags, provide an estimated duration value within <500ms, preferably with confidence level.
- This can be used for multiple items: Determine cold/hot cache estimated durations, CQ SLO, throughput estimated, early load shedding, etc.


AIs:
https://xkcd.com/1838/
 
Cc: smut@chromium.org
Blocking: 706586
Another goal is to couple this with TaskDimensions to better understand the pending queues load and how the throughput (especially w.r.t. pending times) will look in the immediate future to understand if early load shedding should be done right away.

For example  issue 706586  would become much smarter; instead of provided a boolean value (there's at least one bot vs no bot to run this task), it could now provide an estimated pending time. This estimation would fluctuate over time as higher priority tasks are enqueued, which would preempt the task under consideration.

Comment 3 by estaab@chromium.org, Jun 20 2017

Status: Available (was: Untriaged)
Blocking: 342266
keyword: ETA

Comment 6 by mar...@chromium.org, Jun 22 2018

Cc: -phajdan.jr@chromium.org -aludwin@google.com -ryanmartens@google.com -smut@chromium.org s...@google.com
Cc: akes...@chromium.org
Cc: smut@chromium.org
Cc: -s...@google.com
Blocking: 916548
Blocking: 916562

Sign in to add a comment