Create a duration estimator by continuously doing a regression on tags->duration Swarming task data |
||||||||||
Issue descriptionAssumptions: - Swarming tasks have tags that helps "describe" the content of the task or at least, bucket them. For example, buildername, stepname, etc. - Each dimension key value pair is automatically added as a tag, so these can be leveraged implicitly. - There's a correlation overtime with a subset of the task tags and the task's duration. Goals: - Create a query engine that for a given list of tags, provide an estimated duration value within <500ms, preferably with confidence level. - This can be used for multiple items: Determine cold/hot cache estimated durations, CQ SLO, throughput estimated, early load shedding, etc. AIs: https://xkcd.com/1838/
,
Jun 5 2017
Another goal is to couple this with TaskDimensions to better understand the pending queues load and how the throughput (especially w.r.t. pending times) will look in the immediate future to understand if early load shedding should be done right away. For example issue 706586 would become much smarter; instead of provided a boolean value (there's at least one bot vs no bot to run this task), it could now provide an estimated pending time. This estimation would fluctuate over time as higher priority tasks are enqueued, which would preempt the task under consideration.
,
Jun 20 2017
,
Aug 4 2017
,
Feb 9 2018
keyword: ETA
,
Jun 22 2018
,
Jun 22 2018
,
Jul 26
,
Jul 26
,
Dec 19
,
Dec 19
|
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by mar...@chromium.org
, Jun 5 2017