cipd: caching |
||||
Issue descriptionImplement caching of package instances in cipd client. Add -cache-dir flag. The cache dir contains <hash>.cipd package instance files and state.txt. state.txt is current state of the cache dir in text proto format: for each instance it will store last access time. Each time a package instance is needed (by hash) and -cache-dir is specified, we check if the instance is present in the cache. Then we unconditionally read state.txt. If the instance is not in state.txt, we add to it; otherwise update last access time. Then if the total # of instances is beyond a limit, we delete ones that were not accessed for a long time. Then save the file back. We don't use locking, generally because cipd never did locking. If concurrent access to site/cache dir is required, the caller would be responsible for synchronization.
,
Apr 8 2016
,
Apr 8 2016
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/81ae6f3a3836e9f32c9d99398974d69bf382ff80 commit 81ae6f3a3836e9f32c9d99398974d69bf382ff80 Author: nodir <nodir@chromium.org> Date: Fri Apr 08 20:31:02 2016 cipd: shared tag cache Add -cache-dir flag to most of the commands: a path to a shared cache dir. ATM only tag cache uses it. In the next CL it will be used for a shared instance cache. No synchronization between different cipd processes that access same cache dir, because cipd never used locking R=vadimsh@chromium.org BUG= 601017 Review URL: https://codereview.chromium.org/1862303004 [modify] https://crrev.com/81ae6f3a3836e9f32c9d99398974d69bf382ff80/client/cipd/client.go [modify] https://crrev.com/81ae6f3a3836e9f32c9d99398974d69bf382ff80/client/cmd/cipd/friendly.go [modify] https://crrev.com/81ae6f3a3836e9f32c9d99398974d69bf382ff80/client/cmd/cipd/main.go
,
Apr 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b commit d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b Author: nodir <nodir@chromium.org> Date: Wed Apr 13 23:31:37 2016 cipd: accept a function in EnsureFile Expose lower level API in FileSystem.EnsureFile R=vadimsh@chromium.org BUG= 601017 Review URL: https://codereview.chromium.org/1872363004 [modify] https://crrev.com/d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b/client/cipd/client.go [modify] https://crrev.com/d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b/client/cipd/local/deployer.go [modify] https://crrev.com/d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b/client/cipd/local/fs.go [modify] https://crrev.com/d91fccd83c7eff7aece07d5886a4ea2b2e9cc34b/client/cipd/local/fs_test.go
,
Apr 13 2016
The following revision refers to this bug: https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/6c20df86be89df4837ccf85f19827e69b1cf584c commit 6c20df86be89df4837ccf85f19827e69b1cf584c Author: nodir <nodir@chromium.org> Date: Wed Apr 13 23:40:53 2016 cipd: instance cache Cache package instances on file system in a shared cache, as specified by -cache-dir flag. Use LRU for GC. The implementation is similar to the one of tag cache: a binary-protobuf file with cache state is loaded/saved every time it is accessed. The state contains last access time of instances. If the state file is not found, corrupted or was not synced with fs for a long time, the state is synced. Newly discovered instances are considered last accessed at EPOCH. GC happens on sync or when an instance is added: cutoff date is determined and first N instances that were accessed before cutoff date are removed, where N is the number of instance beyond cache limit. R=vadimsh@chromium.org BUG= 601017 Review URL: https://codereview.chromium.org/1870263002 [modify] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cipd/client.go [add] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cipd/internal/instancecache.go [add] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cipd/internal/instancecache_test.go [modify] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cipd/internal/messages/messages.pb.go [modify] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cipd/internal/messages/messages.proto [modify] https://crrev.com/6c20df86be89df4837ccf85f19827e69b1cf584c/client/cmd/cipd/main.go
,
Apr 14 2016
,
Apr 27 2016
|
||||
►
Sign in to add a comment |
||||
Comment 1 by no...@chromium.org
, Apr 6 2016