This isn't about the bot pool which the builder runs on right?

I think milo could benefit from being able to display the bot pool status (20 free, 1000 available, 790 in use, 50 dead, etc).  That's probably a different but.

yeah, good idea. Outside of the scope of this bug though

This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

+swarming component, since it needs some info from swarming.

The API is already there to do counts:
https://chromium-swarm.appspot.com/_ah/api/explorer/#p/swarming/v1/swarming.bots.count

And the link can be something like:
https://chromium-swarm.appspot.com/bot?id=build108-b4

The API isn't super usable for Milo since it takes like 5-10s though.

I wonder if it can be sped up, or if we can do some clever caching on the Milo side.

Either async JS or pre-cache via cron job every minute.

I didn't see a way to speed up count queries in datastore.  Most solutions call for implementing a counter, which requires a decent amount of work.

I like the cron job every minute.  async JS on a 10s latency endpoint introduces a lot of jank.

Proposal (all milo side):
1. Create a new field in BuilderSummary called swarming_dimensions.  It is a JSON list of swarming dimensions for the builder.  It is updated whenever BuilderSummary is updated.
2. Create a new entity, BotCount, child of BuilderSummary.  It contains the bot count information.
3. Create a 1 min cron job that looks for all unique swarming_dimensions in BuilderSummary (w/ dedupe), queries swarming, and updates the BotCount entity.

This'll increase traffic to swarming, but probably not by too much compared to the current traffic (since this endpoint is hit on every task load)
Current QPS: http://shortn/_5vr6glSg8a (Approx 1-2QPS)

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/23541dc56ff843150029d03c1dee3004a198ab6c

commit 23541dc56ff843150029d03c1dee3004a198ab6c
Author: Ryan Tseng <hinoka@google.com>
Date: Fri Apr 06 23:41:39 2018

[cr-buildbucket] Add swarming dimensions to get_builders API

Bug:  687335 
Change-Id: I9c839c592fe5d195fd7e5cd10551bb5fd6875e19
Reviewed-on: https://chromium-review.googlesource.com/1000539
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>

[modify] https://crrev.com/23541dc56ff843150029d03c1dee3004a198ab6c/appengine/cr-buildbucket/swarming/test/swarmbucket_api_test.py
[modify] https://crrev.com/23541dc56ff843150029d03c1dee3004a198ab6c/appengine/cr-buildbucket/swarming/swarmbucket_api.py

I'm starting to see people run into friction during migration wrt bot pools and capacity visibility, so I'm bumping the priority of this.

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/6494f084385777d58521f81773308b0b1a062489

commit 6494f084385777d58521f81773308b0b1a062489
Author: Ryan Tseng <hinoka@google.com>
Date: Wed Apr 11 23:03:23 2018

[cr-buildbucket] Add swarming_hostname as part of the get_builders API

Bug:  687335 
Change-Id: I2661b4e6b062d708c699f4f37e97586b258285ca
Reviewed-on: https://chromium-review.googlesource.com/1008827
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>

[modify] https://crrev.com/6494f084385777d58521f81773308b0b1a062489/appengine/cr-buildbucket/swarming/test/swarmbucket_api_test.py
[modify] https://crrev.com/6494f084385777d58521f81773308b0b1a062489/appengine/cr-buildbucket/swarming/swarmbucket_api.py

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/7a968d18303c8dddf0fb38c89b201a21c2b891c5

commit 7a968d18303c8dddf0fb38c89b201a21c2b891c5
Author: Ryan Tseng <hinoka@google.com>
Date: Thu Apr 12 00:57:36 2018

Update buildbucket api definitions

This picks up changes in https://chromium-review.googlesource.com/c/infra/infra/+/1008827

Bug:  687335 
Change-Id: I460fd6c87d8e2795d71d4d7bf6fd0419d1ac8757
Reviewed-on: https://chromium-review.googlesource.com/1009069
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>

[modify] https://crrev.com/7a968d18303c8dddf0fb38c89b201a21c2b891c5/common/api/buildbucket/buildbucket/v1/buildbucket-api.json
[modify] https://crrev.com/7a968d18303c8dddf0fb38c89b201a21c2b891c5/common/api/buildbucket/swarmbucket/v1/swarmbucket-api.json
[modify] https://crrev.com/7a968d18303c8dddf0fb38c89b201a21c2b891c5/common/api/buildbucket/swarmbucket/v1/swarmbucket-gen.go

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b

commit e49fb6ea30ef5e992d84711d9b837cf40c06ae6b
Author: Ryan Tseng <hinoka@google.com>
Date: Mon Apr 30 23:11:35 2018

[milo] Add cron job to update machine pool for luci builders from swarming

This adds a cron job to update a couple new entities:
* BotPool: Keyed on Swarming Host + Dimensions.  Contains list of bots and statuses.
* BuilderPool: Child of BuilderSummary.  References BotPool via PoolDescriptor.

These combined are enough to display bot information like we do in buildbot.

Bug:  687335 
Change-Id: Ib725c26dad77ebf85c98fe1bb9f17a2123af64aa
Reviewed-on: https://chromium-review.googlesource.com/1009166
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[add] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/buildsource/buildbucket/pools.go
[add] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/buildsource/buildbucket/pools_test.go
[add] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/common/model/bots.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/common/model/builder_summary.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/common/model/status.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/common/model/status_string.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/frontend/appengine/cron.yaml
[delete] https://crrev.com/6d14560e12c752e215e4bdce309096b8b4224e5f/milo/frontend/cron_stats.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/frontend/handler_fixdatastore.go
[modify] https://crrev.com/e49fb6ea30ef5e992d84711d9b837cf40c06ae6b/milo/frontend/routes.go

This is live on prod now, but there are still 2 bugs:

* The "builder" dimension is special, and isn't taken into consideration.
* Pools with over 200 bots are limited at 200

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/ec23b13659f70a0656498b52bdebd69183a74610

commit ec23b13659f70a0656498b52bdebd69183a74610
Author: Ryan Tseng <hinoka@google.com>
Date: Thu May 10 18:26:50 2018

[cr-buildbucket] Update dimension calculation in get_builders

So that behavior matches task creation.  I.E. the "builder" dimension is factored in.

Also removes the ability to specify dimensions via task templates,
because all of our dimensions are specified via cr-buildbucket.cfg.

Bug:  687335 
Change-Id: Ib0fa86a152090db1ff4f9074497c22c5350d83b7
Reviewed-on: https://chromium-review.googlesource.com/1050973
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/ec23b13659f70a0656498b52bdebd69183a74610/appengine/cr-buildbucket/swarming/swarming.py
[modify] https://crrev.com/ec23b13659f70a0656498b52bdebd69183a74610/appengine/cr-buildbucket/swarming/test/swarmbucket_api_test.py
[modify] https://crrev.com/ec23b13659f70a0656498b52bdebd69183a74610/appengine/cr-buildbucket/swarming/test/swarming_test.py
[modify] https://crrev.com/ec23b13659f70a0656498b52bdebd69183a74610/appengine/cr-buildbucket/swarming/swarmbucket_api.py
[modify] https://crrev.com/ec23b13659f70a0656498b52bdebd69183a74610/appengine/cr-buildbucket/swarming/swarmingcfg.py

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/36a8f991102d8b97c0b42c907fffacc457bc253d

commit 36a8f991102d8b97c0b42c907fffacc457bc253d
Author: Ryan Tseng <hinoka@google.com>
Date: Thu May 10 19:18:58 2018

[milo] Page through Swarming botlist call for bot pool info

The botlist call has an implicit limit of 200.
Follow the cursor until all items have been exhausted.

Bug:  687335 
Change-Id: I2970a5f5fd884e92d0659825c3dd1730ba5a2dc1
Reviewed-on: https://chromium-review.googlesource.com/1052497
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/36a8f991102d8b97c0b42c907fffacc457bc253d/milo/buildsource/buildbucket/pools.go

Should be fixed now.