New issue
Advanced search Search tips

Issue 801870 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: May 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Feature

Blocking:
issue 844151



Sign in to add a comment

isolateserver.py: trim old items

Project Member Reported by mar...@chromium.org, Jan 14 2018

Issue description

Right now items in the bot local isolated caches are removed when:
- Maximum number of files in the cache (self.policies.max_items)
- Maximum total bytes of the cache (self.policies.max_cache_size)
- Minimum free disk space (self.policies.min_free_space)


Proposal:
Also trim items not touched since N days. The hypothesis is that keeping objects more than 30 days old is dead weight not worth keeping. The number of days to be fine tuned, 9 days is likely a good number. The change is fairly simple and harmless.


Benefit:
Will significantly reduce the size of the cache for bots that are not continuously recycled.


AIs:
- Add flag to run_isolated.py/isolateserver.py
- Implement trimming of old items in isolateserver.DiskCache._trim()
- Expose flag in bot_config.get_settings()
 

Comment 1 by mar...@chromium.org, Feb 18 2018

Another AI is that we need to report duration and results to the server when it's done.
Project Member

Comment 2 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-py.git/+/a13cead8267a44095f7851bed2b50f0edf7babdc

commit a13cead8267a44095f7851bed2b50f0edf7babdc
Author: Marc-Antoine Ruel <maruel@chromium.org>
Date: Wed Apr 18 22:07:21 2018

[client] Trim items from the local cache older than 3 weeks

This will help performance by removing dead luggage. I suspect this is a
significant issue on non-recycled bots. This shouldn't have performance cost on
recycled bots.

Tweak LRUDict loading to be faster by loading all at once instead of doing an
iteration.

Update run_isolated_test that was using really old time, so adding time based
eviction broke the test.

Bug:  801870 
Change-Id: I930bbb83d619fc49c1b51f25d9d884b40c8f3b1d
Reviewed-on: https://chromium-review.googlesource.com/1017603
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>

[modify] https://crrev.com/a13cead8267a44095f7851bed2b50f0edf7babdc/client/isolateserver.py
[modify] https://crrev.com/a13cead8267a44095f7851bed2b50f0edf7babdc/client/tests/isolateserver_test.py
[modify] https://crrev.com/a13cead8267a44095f7851bed2b50f0edf7babdc/client/tests/run_isolated_test.py
[modify] https://crrev.com/a13cead8267a44095f7851bed2b50f0edf7babdc/client/utils/lru.py

Comment 3 by mar...@chromium.org, May 15 2018

Owner: mar...@chromium.org
Status: Assigned (was: Available)
Project Member

Comment 4 by bugdroid1@chromium.org, May 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-py.git/+/c12f6ca8939f882d5d3373b29297468037b269b3

commit c12f6ca8939f882d5d3373b29297468037b269b3
Author: Marc-Antoine Ruel <maruel@chromium.org>
Date: Thu May 17 19:07:51 2018

[client] trim old named caches and CIPD.

- CIPD cache is now 512Gb, 300 items and 3 weeks old.
- CIPD client cache is now 1Gb, 10 items and 3 weeks old
- Named cache is now 3 weeks old, 50 items.
  - Maximum size is not implemented yet but planned to be 1Tb.
  - We can make this configurable later if needed.

Refactor towards a more coherent object eviction policy. The end goal is to move
DiskCache and named caches into local_caching.py.

Bug:  801870 
Change-Id: I6a2d19ea8ddc1e6e812e85a75599e8bf688e90ef
Reviewed-on: https://chromium-review.googlesource.com/1060296
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/appengine/swarming/server/bot_archive.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/appengine/swarming/swarming_bot/bot_code/task_runner_test.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/cipd.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/isolateserver.py
[add] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/local_caching.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/named_cache.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/run_isolated.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/tests/isolateserver_test.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/tests/named_cache_test.py
[modify] https://crrev.com/c12f6ca8939f882d5d3373b29297468037b269b3/client/tests/run_isolated_test.py

Comment 5 by mar...@chromium.org, May 17 2018

Blocking: 844151

Comment 6 by mar...@chromium.org, May 22 2018

Status: Fixed (was: Assigned)
There's still some refactoring to do but the crux is done. Rest is tracked as  issue 844151 .

Sign in to add a comment