This requires the crosskylabadmin app to be able to commit changes to chromeos/infra_internal/skylab_inventory project. Manual EnsurePoolHealthy tool was added in issue 900006
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/4e797411d57db21060f9881a5febef494b21ea22 commit 4e797411d57db21060f9881a5febef494b21ea22 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Tue Dec 11 23:17:55 2018 skylab: Extract a printError function BUG=chromium:913674 Change-Id: I4293fa18c5e986d8bdc2c3a9f874d0f70fecda5f Reviewed-on: https://chromium-review.googlesource.com/c/1372214 Reviewed-by: Allen Li <ayatane@chromium.org> Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#19496} [modify] https://crrev.com/4e797411d57db21060f9881a5febef494b21ea22/go/src/infra/cmd/skylab/internal/cmd/ensure-pool-healthy.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/7717cb87160f62088c8b072e7660538bfc21a726 commit 7717cb87160f62088c8b072e7660538bfc21a726 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Wed Dec 12 00:17:10 2018 skylab: push parseArgs down the stack BUG=chromium:913674 Change-Id: Ic1325e19b958b49683a18699615229aa1be2494c Reviewed-on: https://chromium-review.googlesource.com/c/1372215 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Cr-Commit-Position: refs/heads/master@{#19499} [modify] https://crrev.com/7717cb87160f62088c8b072e7660538bfc21a726/go/src/infra/cmd/skylab/internal/cmd/ensure-pool-healthy.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/e7696ba2a36e736adb88b5c3d45011c1c26dc9de commit e7696ba2a36e736adb88b5c3d45011c1c26dc9de Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Wed Dec 12 00:21:43 2018 skylab/ensure-pool-healthy: Fix panic() while printing BUG=chromium:913674 Change-Id: I4082472f73f23cdf426d88cff47e5c80f288bfc1 Reviewed-on: https://chromium-review.googlesource.com/c/1372217 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Cr-Commit-Position: refs/heads/master@{#19500} [modify] https://crrev.com/e7696ba2a36e736adb88b5c3d45011c1c26dc9de/go/src/infra/cmd/skylab/internal/cmd/ensure-pool-healthy.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/687d3cc3c7882aef224f3b5d43a8b805ee442937 commit 687d3cc3c7882aef224f3b5d43a8b805ee442937 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Wed Dec 12 00:37:16 2018 skylab: Split parseArgs BUG=chromium:913674 Change-Id: I117e8d3f33004860838f7a573548cff1b2a4523d Reviewed-on: https://chromium-review.googlesource.com/c/1372216 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Cr-Commit-Position: refs/heads/master@{#19501} [modify] https://crrev.com/687d3cc3c7882aef224f3b5d43a8b805ee442937/go/src/infra/cmd/skylab/internal/cmd/ensure-pool-healthy.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/ffcb03c4fc1fa0f9c8b39413589faff17112a578 commit ffcb03c4fc1fa0f9c8b39413589faff17112a578 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Wed Dec 12 00:55:01 2018 skylab: Add -all-models to ensure-pool-healthy subcommand BUG=chromium:913674 Change-Id: Iee7b7e5fdca0a1bda11993bad8127b72efd6cc55 Reviewed-on: https://chromium-review.googlesource.com/c/1372218 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Cr-Commit-Position: refs/heads/master@{#19502} [modify] https://crrev.com/ffcb03c4fc1fa0f9c8b39413589faff17112a578/go/src/infra/cmd/skylab/internal/cmd/ensure-pool-healthy.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/d1f94abb4da00b19f465b411820fcbe2c94eeb93 commit d1f94abb4da00b19f465b411820fcbe2c94eeb93 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Tue Jan 08 20:10:44 2019 crosskylabadmin: Extract ensurePoolHealthyFor free function BUG=chromium:913674 Change-Id: I98f9993c5cce9da409463bed45cd4e782c438fc5 Reviewed-on: https://chromium-review.googlesource.com/c/1399337 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Cr-Commit-Position: refs/heads/master@{#19832} [modify] https://crrev.com/d1f94abb4da00b19f465b411820fcbe2c94eeb93/go/src/infra/appengine/crosskylabadmin/app/frontend/inventory/inventory.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/704ed33ea4d2dc7ef53ef3f1388c9fb2a02a969b commit 704ed33ea4d2dc7ef53ef3f1388c9fb2a02a969b Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Tue Jan 08 21:35:00 2019 crosskylabadmin: Split off pool operations to own file BUG=chromium:913674 Change-Id: I3a3c9cdb2da58ddabd97e72c6765145448482278 Reviewed-on: https://chromium-review.googlesource.com/c/1399338 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Cr-Commit-Position: refs/heads/master@{#19834} [modify] https://crrev.com/704ed33ea4d2dc7ef53ef3f1388c9fb2a02a969b/go/src/infra/appengine/crosskylabadmin/app/frontend/inventory/inventory.go [add] https://crrev.com/704ed33ea4d2dc7ef53ef3f1388c9fb2a02a969b/go/src/infra/appengine/crosskylabadmin/app/frontend/inventory/pool_operations.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/8d63c3afc4f921859cd2b47605fd7e216b1f4d3e commit 8d63c3afc4f921859cd2b47605fd7e216b1f4d3e Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Thu Jan 10 23:09:04 2019 crosskylabadmin: Add pool balancer configuration to luci-config proto BUG=chromium:913674 Change-Id: I5de7cf3661960c7a68303e8e1c42096c72a2fa02 Reviewed-on: https://chromium-review.googlesource.com/c/1399340 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Cr-Commit-Position: refs/heads/master@{#19911} [modify] https://crrev.com/8d63c3afc4f921859cd2b47605fd7e216b1f4d3e/go/src/infra/appengine/crosskylabadmin/app/appengine/devcfg/services/dev/config-template.cfg [modify] https://crrev.com/8d63c3afc4f921859cd2b47605fd7e216b1f4d3e/go/src/infra/appengine/crosskylabadmin/app/config/config.proto [modify] https://crrev.com/8d63c3afc4f921859cd2b47605fd7e216b1f4d3e/go/src/infra/appengine/crosskylabadmin/app/config/config.pb.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f80b4a3720701f3381dc0d864fcf03630c4a9a28 commit f80b4a3720701f3381dc0d864fcf03630c4a9a28 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Fri Jan 18 01:05:37 2019 crosskylabadmin: Add endpoint to ensure pool health for all models BUG=chromium:913674 Change-Id: I2fa1048d91bca44c1ab9d01dd7b364bfded276ae Reviewed-on: https://chromium-review.googlesource.com/c/1399339 Reviewed-by: Aviv Keshet <akeshet@chromium.org> Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#20065} [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/inventory.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/app/frontend/inventory/pool_operations.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/tracker.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/tasker.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/tracker.mock.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/validation.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/inventory.proto [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/inventoryserver_dec.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/inventory.mock.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/app/clients/mock/swarming.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/common.pb.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/pb.discovery.go [modify] https://crrev.com/f80b4a3720701f3381dc0d864fcf03630c4a9a28/go/src/infra/appengine/crosskylabadmin/api/fleet/v1/tasker.mock.pb.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c237470ce177ce6c600cd72f164b94793f607f58 commit c237470ce177ce6c600cd72f164b94793f607f58 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Fri Jan 18 17:48:04 2019 crosskylabadmin: Add cron job to balance critical pools for all models BUG=chromium:913674 Change-Id: Ib93b856662c8dc111bba1e7fc8dd75bbe571993c Reviewed-on: https://chromium-review.googlesource.com/c/1399721 Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Cr-Commit-Position: refs/heads/master@{#20076} [modify] https://crrev.com/c237470ce177ce6c600cd72f164b94793f607f58/go/src/infra/appengine/crosskylabadmin/app/cron/cron.go [modify] https://crrev.com/c237470ce177ce6c600cd72f164b94793f607f58/go/src/infra/appengine/crosskylabadmin/app/clients/gitiles.go [modify] https://crrev.com/c237470ce177ce6c600cd72f164b94793f607f58/go/src/infra/appengine/crosskylabadmin/app/clients/gerrit.go
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/6863e9388877a7404f392e3589114491b3232fb8 commit 6863e9388877a7404f392e3589114491b3232fb8 Author: Prathmesh Prabhu <pprabhu@chromium.org> Date: Fri Jan 18 18:15:41 2019
Comment 1 by bugdroid1@chromium.org
, Dec 11