New issue
Advanced search Search tips

Issue 784995 link

Starred by 1 user
Status: Fixed
Owner:
mar...@chromium.org
Closed: Mar 2018
Cc:
mar...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature



Sign in to add a comment

Implement fast-path for high throughput task queues

Project Member Reported by mar...@chromium.org, Nov 14 2017 
See http://go/swarming-high-throughput for the design doc.

The goal is to create a low friction fast path and reduce the stress on the Cloud DB global TaskToRun index.
Project Member

Comment 1 by bugdroid1@chromium.org, Nov 15 2017 

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-py.git/+/f6ff19440fc924bddde276b1abf2f4e5bfe702f7

commit f6ff19440fc924bddde276b1abf2f4e5bfe702f7
Author: Marc-Antoine Ruel <maruel@chromium.org>
Date: Wed Nov 15 21:46:02 2017

swarming: Implement TaskToRun lookup pipeline

This will cost more in memcache RPCs but should accelerate the polling
significantly by pipelining look ups for high throughput task queues.

Add one last memcache lookup before the actual Tx reap, to further
reduce DB Tx contention.

Bug:  784995 
Change-Id: I051d63864aae1e642c49bf9df68a61910a897900
Reviewed-on: https://chromium-review.googlesource.com/771772
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>

[modify] https://crrev.com/f6ff19440fc924bddde276b1abf2f4e5bfe702f7/appengine/swarming/server/task_scheduler.py
[modify] https://crrev.com/f6ff19440fc924bddde276b1abf2f4e5bfe702f7/appengine/swarming/server/task_to_run.py
[modify] https://crrev.com/f6ff19440fc924bddde276b1abf2f4e5bfe702f7/appengine/swarming/server/task_to_run_test.py

Comment 2 by mar...@chromium.org, Mar 1 2018

Owner: mar...@chromium.org
Status: Fixed (was: Available)
►

Sign in to add a comment