Swarming should be resilient to isolated_cache being purged |
|||
Issue descriptionWhat steps will reproduce the problem? 1. Have a bot with low disk space (e.g. https://buganizer.corp.google.com/issues/34049848) 2. Delete items from isolated_cache because it is large (see comment 2) What is the expected result? 3. Isolate re-downloads items if it needs them again What happens instead of that? 3. It breaks (see comment 3) It's nice that isolate tries to clean up this directory itself, and apparently we're already working on tweaking the threshold (see comment 4), but isolated_cache is a cache. Items being deleted from it should not lead to fatal failures.
,
Jan 4 2017
Looks like a regression. It used to be resilent to cache corruption: https://github.com/luci/luci-py/blob/master/client/isolateserver.py#L771 This function should catch ENOENT and return False (instead of raising OSError), it will be equivalent to a cache miss: https://github.com/luci/luci-py/blob/master/client/isolateserver.py#L348 I'll make this change.
,
Jan 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-py.git/+/ae56b3a60ca775afa9e049fa32470a15ef2e1c13 commit ae56b3a60ca775afa9e049fa32470a15ef2e1c13 Author: vadimsh <vadimsh@chromium.org> Date: Thu Jan 05 00:42:46 2017 Make isolate cache resilient to unexpectedly deleted items. It seems to be a regression. Add a test. R=agable@chromium.org BUG= 678396 Review-Url: https://codereview.chromium.org/2610273002 [modify] https://crrev.com/ae56b3a60ca775afa9e049fa32470a15ef2e1c13/client/isolateserver.py [modify] https://crrev.com/ae56b3a60ca775afa9e049fa32470a15ef2e1c13/client/tests/isolateserver_test.py
,
Jan 5 2017
This has been deployed. |
|||
►
Sign in to add a comment |
|||
Comment 1 by vadimsh@chromium.org
, Jan 4 2017