New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 638644 link

Starred by 1 user

Issue metadata

Status: Archived
Owner: ----
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: ----
OKR



Sign in to add a comment

Make gs_offloader faster

Project Member Reported by xixuan@chromium.org, Aug 17 2016

Issue description

gs_offloader is a little bit slow now. At least it is probably slower than the speed of shard copying testing logs from DUT, which is a potential factor that makes shard's disk usage increase.

So if gs_offloader gets much faster, shard's disk won't get fulled so quickly.
 
For reference, see  bug 638257 .  One of the problems leading
to that failure was, in essence, that results from the DUTs
arrived faster than the ability of gs_offloader to offload and
clear them.

gs_offloader has parameters for allowing more/less parallelism,
so the first thing to look at is whether things can go faster
simply by increasing the number of simultaneous gsutil subprocesses.

Comment 2 by aut...@google.com, Aug 23 2016

I think we decided to attack https://bugs.chromium.org/p/chromium/issues/detail?id=638641 first and maybe do this later. Adding Allen to this one as well. 
It's relatively easy to increase parallelism.  If that gives
us more throughput, we really ought to do it.  Fixing  bug 638641 
only helps us with one specific kind of failure.  There are
other kinds of problems for which improving gs_offloader performance
is our best bet for staving off failure.

Comment 4 by aut...@google.com, Aug 30 2016

Labels: OKR

Comment 5 by aut...@google.com, Aug 30 2016

Moving to OKR as it doesn't sound urgent. Needs more investigation 
Are we still running into problems here?

I seem to recall some where also related to large volume of crash dumps (eating up upload bandwidth?)
Status: Archived (was: Untriaged)

Sign in to add a comment