run alerts-dispatcher as a SoM cron task |
||
Issue descriptionMove all of the alerts-dispatcher logic to an appengine cron task that runs inside SoM. I think we're able to do that now. The pubsub logic has been slow going (still plenty of work to do!). This would get us into a much simpler single-deployable-app state much more quickly. SoM on appengine can get all master data from Milo API today (correct?). test-results, FindIt, build.chromium.org etc I think are all visible from GAE. We can get already gatekeeper config data from gitiles at this point. And we can probably do without the snapshot/replay feature for the time being, and later re-implement it with a more general purpose {g|p}RPC log replay system. Re: record/replay: martiniss@ says "Talked to sean friday, we agreed it'd be good. Want to keep replay ability though, so we should figure out where to store the data. Vadim said that we can stream data to cloud storage, and pretend it's a file, so that might work. https://godoc.org/cloud.google.com/go/storage#ObjectHandle.NewWriter looks like the appropriate function."
,
Dec 21 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/c121dd502b0398ab6d6165c543cfb9c9c3b4a115 commit c121dd502b0398ab6d6165c543cfb9c9c3b4a115 Author: Sean McCullough <seanmccullough@chromium.org> Date: Wed Dec 21 00:22:18 2016 [som] /_cron/analyze/:tree handler This basically runs alerts-dispatcher as an appenengine cron job. It just spits out the alerts json for now. Subsequent CLs will wire it up to the datastore. BUG= 675688 Change-Id: Ibf8faddf2652699fa819cc2d0247171454e6efbc Reviewed-on: https://chromium-review.googlesource.com/422494 Commit-Queue: Sean McCullough <seanmccullough@chromium.org> Reviewed-by: Stephen Martinis <martiniss@chromium.org> [add] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/appengine/sheriff-o-matic/som/analyze.go [add] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/appengine/sheriff-o-matic/som/analyze_test.go [modify] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/appengine/sheriff-o-matic/som/main.go [modify] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/monitoring/client/client.go [modify] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/monitoring/client/milo.go [modify] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/monitoring/client/test/test.go [modify] https://crrev.com/c121dd502b0398ab6d6165c543cfb9c9c3b4a115/go/src/infra/monitoring/dispatcher/dispatcher.go
,
Dec 21 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/521dc46ba8c12f5eb82facbd37553e8464cd0b63 commit 521dc46ba8c12f5eb82facbd37553e8464cd0b63 Author: Sean McCullough <seanmccullough@chromium.org> Date: Wed Dec 21 20:06:27 2016 [som] Add cron tasks, actually store alerts from analyzer running in SoM This prefixes the tree name with "milo." in the datastore key so as not to overwrite the current alerts from alerts-dispatcher. This will let us diff the two for some time to make sure the cron job is not causing regressions. It also runs a separate cron task for each tree. This allows us some level of isolation between clients (they can run on different schedules, not interfere with eachother if one of them crashes etc). BUG= 675688 Change-Id: I7c4b258c9ada5e740f0e0327ce850a542441889f Reviewed-on: https://chromium-review.googlesource.com/422993 Commit-Queue: Sean McCullough <seanmccullough@chromium.org> Reviewed-by: Stephen Martinis <martiniss@chromium.org> [modify] https://crrev.com/521dc46ba8c12f5eb82facbd37553e8464cd0b63/go/src/infra/appengine/sheriff-o-matic/cron.yaml [modify] https://crrev.com/521dc46ba8c12f5eb82facbd37553e8464cd0b63/go/src/infra/appengine/sheriff-o-matic/som/analyze.go [modify] https://crrev.com/521dc46ba8c12f5eb82facbd37553e8464cd0b63/go/src/infra/appengine/sheriff-o-matic/som/analyze_test.go
,
Jan 23 2017
|
||
►
Sign in to add a comment |
||
Comment 1 by bugdroid1@chromium.org
, Dec 20 2016