New issue
Advanced search Search tips

Issue 680867 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Per-project PubSub topics in LogDog?

Project Member Reported by vadimsh@chromium.org, Jan 13 2017

Issue description

(Opening the ticket to discuss how hard it will be to do or whether it makes sense at all or not).

I've just gone through the process of allowing some service account to upload logs to Fuchsia logdog project. It consists of two actions:
1. Adding the service account to appropriate pre-setup chrome-infra-auth group (to allow it to talk to the coordinator, I presume). This step is fine and natural enough.
2. Granting this service account PubSub Viewer and Publisher roles on projects/luci-logdog/topics/logs topic. This is not very good.

Doing (2) requires admin-level access to LogDog Cloud project. We also somewhat reduce isolation between projects (for example, Cloud Monitoring metrics for the topic are all mixed up). We also bill all PubSub storage to us.

An alternative way to do it is:
1. Ask customers to setup their own PubSub topic and a subscription for that topic in Cloud Project of their choosing. Ask customers to grant well-known logdog account Subscriber role in the subscription.
2. Ask customers to specify this PubSub subscription and topic full names in their luci-logdog.cfg config.
3. Make logdog butler to push to this topic and logdog service to pull from the subscription.

That way customers can easily add more producers without bothering logdog admins. They will also pay for transient PubSub storage (Pull calls will still bill to LogDog though). Each project will have its own dedicated PubSub monitoring and customers can see for themselves how much stuff they produce.

The downsides:
1. Code complexity. Collectors would need to learn the list of subscriptions to pull, and pull them in parallel.
2. We still have shared service at collector level and below. So projects are still not really isolated (one spammy project may affect all other projects).
 

Comment 1 by d...@chromium.org, Jan 13 2017

It's not a bad idea. The downsides are pretty much why I didn't implement it this way. Additionally, when the system was firs set up, Coordinators didn't have per-project configs (so no real way of knowing what is up) and Collectors didn't pull configs from Coordinator. That's all changed now.

WRT downside #1, this is sort of big. Collector is a pretty highly optimized process, and having it poll N subscriptions instead of 1 increases caching and scaling complexity. It's probably fine.

WRT downside #2, also true with archival and Coordinator quotas. We do use per-project GS archival buckets though, so that at least is shared. Not a bad idea to have LogDog only host intermediary things.

One thing to mention is that, while there *can* be project cross-talk, each log stream is pre-authenticated via a key generated based on per-project write access by the Coordinator. If a log stream enters the pipe that doesn't have a project/stream-matching key, the Collector will discard it. So, in theory, the worst a project can do in that case is waste the Collector's time by making it /dev/null a bunch of invalidly-keyed logs.

WRT advantages, one nice advantage is that Pub/Sub uses a project-wide quota, so allowing that quota to be distributed amongst projects would give us some more freedom.
Status: Available (was: Untriaged)
Project Member

Comment 3 by sheriffbot@chromium.org, Mar 8 2018

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available. If you change it back, also remove the "Hotlist-Recharge-Cold" label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Hotlist-Recharge-Cold LUCI-Backlog
Status: Available (was: Untriaged)
Seems still relevant to me.

Sign in to add a comment