New issue
Advanced search Search tips

Issue 674378 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

Write a server configuration package, with support for different authorities and datastore caching.

Project Member Reported by d...@chromium.org, Dec 15 2016

Issue description

Write a server configuration package for LUCI that is optimized for our services. It should use common/config.Interface as its true backend, but provide higher-level abstraction and functionality targeted towards service usage, caching, and deployment.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Jan 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/642108c30d02d7384b5bd960e470a8b4e0ef045b

commit 642108c30d02d7384b5bd960e470a8b4e0ef045b
Author: dnj <dnj@chromium.org>
Date: Tue Jan 10 19:36:05 2017

Move "common/config" common types into cfgtypes.

Move ProjectName and ProjectConfigSet into its own generic package:
"luci_config/common/cfgtypes".

BUG=chromium:674378
TEST=None

Review-Url: https://codereview.chromium.org/2626433004

[delete] https://crrev.com/94914026ea13c7896d0a6b5fdb51ae05c622cfb1/common/config/util.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/config/config.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/config/projects.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/context.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/context_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/coordinatorTest/context.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/coordinatorTest/logStream.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/coordinatorTest/service.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/logs/get.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/logs/get_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/logs/list_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/logs/query_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/logs/service.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/registration/registerPrefix_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/registration/service.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/services/registerStream_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/endpoints/services/service.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/hierarchy/hierarchy.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/hierarchy/hierarchy_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/hierarchy/project.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/project.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/appengine/coordinator/service.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/butler/bootstrap/env.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/butler/bundler/bundler.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/butler/butler.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/butler/output/logdog/output.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/butlerlib/bootstrap/bootstrap.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cli/main.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cli/path.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cli/subcommandCat.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cli/subcommandList.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cmd/logdog_annotee/link.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cmd/logdog_annotee/main.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/cmd/logdog_butler/main.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/coordinator/client.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/coordinator/list.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/coordinator/query.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/coordinator/query_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/client/coordinator/stream.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/archive/storage.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/bigtable/cache.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/bigtable/logdog_bigtable_test/main.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/bigtable/storage.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/bigtable/storage_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/memory/memory.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/memory/memory_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/storage/storage.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/viewer/url.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/common/viewer/url_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/archivist/archivist.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/archivist/archivist_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/archivist/storageSource.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/cmd/logdog_archivist/main.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/collector.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/collector_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/coordinator/cache.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/coordinator/cache_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/coordinator/coordinator.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/collector/utils_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/service/config/config.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/logdog/server/service/service.go
[add] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/luci_config/common/cfgtypes/config_set.go
[add] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/luci_config/common/cfgtypes/config_set_test.go
[rename] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/luci_config/common/cfgtypes/project_name.go
[rename] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/luci_config/common/cfgtypes/project_name_test.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/milo/appengine/logdog/build.go
[modify] https://crrev.com/642108c30d02d7384b5bd960e470a8b4e0ef045b/milo/appengine/logdog/http.go

Project Member

Comment 2 by bugdroid1@chromium.org, Jan 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/0e70e04e28394c34fad2ec3ba85ff402074f429e

commit 0e70e04e28394c34fad2ec3ba85ff402074f429e
Author: dnj <dnj@chromium.org>
Date: Tue Jan 10 23:07:36 2017

Implement a server-side config service interface.

This implements "server/config", a service-specific config service
interface. This interface is noticably different from the
"common/config.Interface" in the following ways:
- It accepts an authority: on whose behalf is the config request being
  issued?
- Defines a greatly-reduced interface, allowing for much simpler cache
  implementations.
- Allows for format translation, enabling implicit conversion and cache
  storage of configuration data in a different format from its native
  config service representation. Specifically, this targets storing text
  protobufs as binary.

Ultimately, we will delete the equivalent packages out of
"common/config", leaving it as a client endpoint to the config service.
Updates to services, notably "appengine/gaeconfig" will follow.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2580713002

[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/authority.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/authority_string.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/backend.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/client/client.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/client/client_test.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/doc.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/format/format.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/format/format_test.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/gen.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/get_all.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/backend/meta.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/config.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/config_test.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/doc.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/format.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/naming.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/naming_test.go
[add] https://crrev.com/0e70e04e28394c34fad2ec3ba85ff402074f429e/luci_config/server/cfgclient/resolver.go

Project Member

Comment 3 by bugdroid1@chromium.org, Jan 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/aae008db5bab0425635902546ca339bda0ec7f5c

commit aae008db5bab0425635902546ca339bda0ec7f5c
Author: dnj <dnj@chromium.org>
Date: Tue Jan 10 23:15:04 2017

server/config: Add text protobuf support.

Add a text protobuf configuration Resolver. This will load text
protobufs into their proto.Message targets. It will also expose a binary
version of the protobufs for caching, enabling the protobufs to be
loaded from cache as binaries.

Loading from binaries is particularly useful because binaries ignore
unknown tags, whereas text protobufs error on them. This reduces risk of
application rollbacks when loading cached configurations.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2578893002

[add] https://crrev.com/aae008db5bab0425635902546ca339bda0ec7f5c/luci_config/server/cfgclient/textproto/resolver.go
[add] https://crrev.com/aae008db5bab0425635902546ca339bda0ec7f5c/luci_config/server/cfgclient/textproto/resolver_test.go

Project Member

Comment 4 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/99375447064d421783741c2e11ef4bf736839c9d

commit 99375447064d421783741c2e11ef4bf736839c9d
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 03:32:40 2017

server/config: Service-side project access checks.

Add the ability for services to mirror the project access checks that
the "luci-config" service performs. This targets caching and testing
support, enabling non-service authorities to use cached service entries
by pruning the results in the service.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2576993002

[add] https://crrev.com/99375447064d421783741c2e11ef4bf736839c9d/luci_config/server/cfgclient/access/access.go
[add] https://crrev.com/99375447064d421783741c2e11ef4bf736839c9d/luci_config/server/cfgclient/access/access_test.go

Project Member

Comment 5 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/efb529dd4da484266debb645d9ac30e8d83f1329

commit efb529dd4da484266debb645d9ac30e8d83f1329
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 14:17:06 2017

server/config: Add an erroring config wrapper.

Add an erroring config Backend, mirroring that in
common/config/filter/erroring. This will replace that package after it
is integrated into appengine/gaeconfig.

BUG=chromium:674378
TEST=None

Review-Url: https://codereview.chromium.org/2576343002

[add] https://crrev.com/efb529dd4da484266debb645d9ac30e8d83f1329/luci_config/server/cfgclient/backend/erroring/erroring.go

Project Member

Comment 6 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/bb51c387acaf3309d1ae17d64dbb1e467f158497

commit bb51c387acaf3309d1ae17d64dbb1e467f158497
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 20:46:50 2017

server/config: Add test configuration package.

Add a test configuration package that can bolt on any (read: filesystem
or memory) common/config.Interface implementation for testing.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2582433002

[add] https://crrev.com/bb51c387acaf3309d1ae17d64dbb1e467f158497/luci_config/server/cfgclient/backend/testconfig/local_service.go
[add] https://crrev.com/bb51c387acaf3309d1ae17d64dbb1e467f158497/luci_config/server/cfgclient/backend/testconfig/local_service_test.go

Project Member

Comment 7 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/bb51c387acaf3309d1ae17d64dbb1e467f158497

commit bb51c387acaf3309d1ae17d64dbb1e467f158497
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 20:46:50 2017

server/config: Add test configuration package.

Add a test configuration package that can bolt on any (read: filesystem
or memory) common/config.Interface implementation for testing.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2582433002

[add] https://crrev.com/bb51c387acaf3309d1ae17d64dbb1e467f158497/luci_config/server/cfgclient/backend/testconfig/local_service.go
[add] https://crrev.com/bb51c387acaf3309d1ae17d64dbb1e467f158497/luci_config/server/cfgclient/backend/testconfig/local_service_test.go

Project Member

Comment 8 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4

commit ae167b8559d9a2e0de3210b4dea1c665ae25d6d4
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 20:53:27 2017

server/config: Generic caching backend.

Add a generic caching backend. The backend works by reducing config
service operations to unique cache keys and supporting various utility
serialization/deserialization methods.

The cache package is currently implemented for "proccache", which uses
the process cache to store configurations. Other cache implementations
will also use this package.

BUG=chromium:674378
TEST=unit

Review-Url: https://codereview.chromium.org/2573403002

[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/codec.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/codec_test.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/config.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/config_test.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/doc.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/proccache.go
[add] https://crrev.com/ae167b8559d9a2e0de3210b4dea1c665ae25d6d4/luci_config/server/cfgclient/backend/caching/proccache_test.go

Project Member

Comment 9 by bugdroid1@chromium.org, Jan 11 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/external/github.com/luci/luci-go.git/+/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c

commit 45e9a86d32dabca5671d0c1773e124dbc2ec8b5c
Author: dnj <dnj@chromium.org>
Date: Wed Jan 11 23:21:38 2017

Add server/cache support to gaeconfig.

Rewrite "gaeconfig" and apps that use it to use "server/cache" instead.

Because this fully replaces usage of common/config/caching and
common/config/impl/erroring, delete those packages. Also, common/config
is no longer used in Contexts, so delete that as well.

BUG=chromium:674378
TEST=local,unit

Review-Url: https://codereview.chromium.org/2575383002

[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/appengine/gaeconfig/cache.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/appengine/gaeconfig/cache_test.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/appengine/gaeconfig/default.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/appengine/gaemiddleware/context.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/common/config/context.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/common/config/filters/caching/config.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/common/config/filters/caching/config_test.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/common/config/filters/caching/doc.go
[delete] https://crrev.com/0e1e01b5fd687f16f2df2566e3cefc96097ce3ae/common/config/impl/erroring/erroring.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/common/data/text/templateproto/loader.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/common/data/text/templateproto/loader_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/dm/api/template/loader.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/dm/appengine/deps/auth.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/dm/appengine/distributor/fake/fake.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/dm/appengine/distributor/registry.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/auth.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/config/config.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/config/projects.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/context.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/context_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/coordinatorTest/context.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/coordinatorTest/logStream.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/endpoints/logs/service.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/endpoints/services/getConfig.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/hierarchy/hierarchy.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/hierarchy/hierarchy_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/hierarchy/project.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/appengine/coordinator/project.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/butler/bundler/bundler.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/cli/subcommandCat.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/cli/subcommandList.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/cmd/logdog_annotee/main.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/coordinator/query.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/coordinator/query_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/client/coordinator/stream.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/common/storage/bigtable/bigtable_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/common/storage/bigtable/storage.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/common/storage/bigtable/storage_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/archivist/archivist_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/archivist/storageSource.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/collector/collector.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/collector/coordinator/cache.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/collector/coordinator/cache_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/logdog/server/collector/coordinator/coordinator.go
[add] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/backend/memcache/cache.go
[add] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/backend/memcache/cache_test.go
[add] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/format/context.go
[add] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/format/default.go
[add] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/gaeconfig/default.go
[rename] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/gaeconfig/doc.go
[rename] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/luci_config/appengine/gaeconfig/settings.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/milo/appengine/buildbot/pubsub_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/milo/appengine/logdog/http.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/milo/appengine/settings/acl_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/milo/appengine/settings/config.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/milo/appengine/settings/config_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/scheduler/appengine/catalog/catalog.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/scheduler/appengine/catalog/catalog_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/tokenserver/appengine/certconfig/rpc_import_ca_configs.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/tokenserver/appengine/certconfig/rpc_import_ca_configs_test.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/tokenserver/appengine/delegation/rpc_import_delegation_configs.go
[modify] https://crrev.com/45e9a86d32dabca5671d0c1773e124dbc2ec8b5c/tokenserver/appengine/delegation/rpc_import_delegation_configs_test.go

Comment 11 by d...@chromium.org, Jan 12 2017

Cc: no...@chromium.org vadimsh@chromium.org iannucci@chromium.org estaab@chromium.org hinoka@chromium.org
This is now landed and running happily on LogDog prod. +people with apps.

To enable:
1) Add a datastore update cron to your app, something along the lines of the cache manager here: https://github.com/luci/luci-go/blob/c987eb8dfe2e137603d56565ada0bd8cbe5a0ad1/appengine/gaemiddleware/resources.cfg
2) Go to the /admin/settings/gaeconfig panel of your app and set the datastore cache to either Enabled or Strict.

Happy to work with anyone who needs help.
Project Member

Comment 12 by bugdroid1@chromium.org, Jan 12 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/5ad7c2d321ee23d64311b2a9b962dd97c8bdf2ab

commit 5ad7c2d321ee23d64311b2a9b962dd97c8bdf2ab
Author: Dan Jacques <dnj@chromium.org>
Date: Thu Jan 12 18:03:38 2017

Update deps, roll luci-go.

Added golang.org/x/text to DEPS to satisfy new golang.org/x/net
requirements.

infra/go/src/github.com/luci/luci-go:
c987eb8d tumble: If there is work always loop.
b650a40d Tumble, datastore cleanup.
a6a07cee Implement config service cache on top of datastore
45e9a86d Add server/cache support to gaeconfig.
0e1e01b5 Tumble operates on per-namespace shards.
ae167b85 server/config: Generic caching backend.
bb51c387 server/config: Add test configuration package.
efb529dd server/config: Add an erroring config wrapper.
99375447 server/config: Service-side project access checks.

TBR=iannucci@chromium.org
BUG=chromium:678896, chromium:674378
TEST=None

Change-Id: I08ce122ec808aefe20f23bff606967b8222ee4a9
Reviewed-on: https://chromium-review.googlesource.com/427284
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/5ad7c2d321ee23d64311b2a9b962dd97c8bdf2ab/DEPS
[modify] https://crrev.com/5ad7c2d321ee23d64311b2a9b962dd97c8bdf2ab/go/deps.lock
[modify] https://crrev.com/5ad7c2d321ee23d64311b2a9b962dd97c8bdf2ab/go/deps.yaml

Cc: -iannucci@chromium.org iannu...@google.com

Sign in to add a comment