New issue
Advanced search Search tips

Issue 656862 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: Feb 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature



Sign in to add a comment

Swarming: implement task creation rate limiting

Project Member Reported by vadimsh@chromium.org, Oct 18 2016

Issue description

A buggy recipe shouldn't be able to trigger unlimited number of jobs and overload the whole infrastructure.

I'm not clear where exactly this should be implemented (Swarming? Buildbucket?). Thus vague "jobs".
 
Cc: mar...@chromium.org
Status: Available (was: Untriaged)
If swarming is a public API, in the sense that people and systems outside of chrome-infra's direct control can talk to machines that are also being used by chrome-infra's directly controlled machines, then, yes, we need some sort of quota or ddos mechanism.

buildbucket - which is also clearly a public API - also needs such a thing. 

Perhaps we need two bugs, then.

Comment 2 by mar...@chromium.org, Jan 19 2017

Issue 617846 is slightly related, as a recovery mechanism after the fact.

We care more about the workers, less about the server. Pending tasks can be canceled via scripts at worst, e.g. cancelling all state=PENDING, pool:Chrome-perf tasks:

./swarming.py query -S chromium-swarm.appspot.com --limit 0 'tasks/list?state=PENDING&tags=pool:Chrome-perf' > out.json
for task_id in $(python -c "import json; print '\n'.join(i['task_id'] for i in json.load(open('out.json'))['items'])"); do
  echo $task_id
  ./swarming.py cancel -S chromium-swarm.appspot.com  $task_id
done

Project Member

Comment 3 by sheriffbot@chromium.org, Feb 21 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 4 by mar...@chromium.org, Feb 21 2018

Cc: -mar...@chromium.org
Labels: -Type-Bug -Hotlist-Recharge-Cold Type-Feature
Status: Available (was: Untriaged)
Summary: Swarming: implement task creation rate limiting (was: We need some kind of quota mechanism for jobs)

Comment 5 by no...@chromium.org, Feb 24 2018

Mergedinto: 812021
Status: Duplicate (was: Available)

Sign in to add a comment