New issue
Advanced search Search tips

Issue 671870 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Dec 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 0
Type: Bug



Sign in to add a comment

Alerts are firing for milo due to client.art

Project Member Reported by hinoka@chromium.org, Dec 7 2016

Issue description

https://app.google.stackdriver.com/account/login/luci-milo/incidents/0.k56bf8mrngxq

https://screenshot.googleplex.com/dtas9nGFrsH

16:29:16.027
Message ID "65350684880432" from subscription projects/luci-milo/subscriptions/buildbot-public is 5554647 bytes long
16:29:18.523
There are 10 builds
16:29:18.523
The master name is client.art
16:29:18.561
Fetching info for master client.art
16:29:22.071
Length of json data: 15723082
16:29:22.071
Length of gzipped data: 1364318
16:29:22.188
Could not save master in datastore Call error 5: API request too large: 1364444. Limit for datastore_v3 is: 1048576 :: {"error":"Call error 5: API request too large: 1364444. Limit for datastore_v3 is: 1048576"}
 
Deploying a hack to ignore client.art, the master looks exceptionally abandoned: https://uberchromegw.corp.google.com/i/client.art/builders (OVER 9000.... pending builds on most builders)
Status: WontFix (was: Untriaged)
Closing since the outage is over.  The root cause was a git merge that added 9000 builds to the master in a short period of time.  Milo has been reverted to its original state.

Sign in to add a comment