New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 712869 link

Starred by 1 user

Issue metadata

Status: Duplicate
Owner: ----
Closed: Apr 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

ts_mon: /chrome/infra/chromeos/autotest/drone/active_process_percentage changed types error

Project Member Reported by nxia@chromium.org, Apr 18 2017

Issue description

See the RESOURCE_EXHAUSTED in master scheduler log every other minutes, is this affecting the scheduler tick rate. 


04/18 14:53:56.742 WARNI|          monitors:0161| HttpsMonitor.send received status 429: {
  "error": {
    "code": 429,
    "message": "Resource has been exhausted (e.g. check quota).",
    "status": "RESOURCE_EXHAUSTED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.DebugInfo",
        "detail": "[ORIGINAL ERROR] generic::resource_exhausted: APPLICATION_ERROR;streamz/StreamzCollection.Write;Point value for metric '/chrome/infra/chromeos/autotest/drone/active_process_percentage' not of expected type 'INT64': column { name: \"drone_hostname\" string_value: \"chromeos-server11.hot.corp.google.com\" } double_value: 0.13833333333333334 start_timestamp { seconds: 1492537073 } end_timestamp { seconds: 1492552435 };AppErrorCode=9;StartTimeMs=1492552436631;tcp;Deadline(sec)=15.0;ResFormat=UNCOMPRESSED;ServerTimeSec=0.03117084503173828;LogBytes=256;FailFast;EffSecLevel=none;ReqFormat=UNCOMPRESSED;ReqID=f11fddb44942002;GlobalID=a91dd1189f7c4949;Server=10.87.117.193:4801"
      }
    ]
  }
}
04/18 14:54:14.137 INFO |        thread_lib:0071| (Worker.cros-autotest-shard1.cbf.corp.google.com) finished.


 
Cc: pprabhu@chromium.org
Is this another of those cases where "RESOURCE_EXHAUSTED" actually means "you used an invalid field" ?
Summary: ts_mon: /chrome/infra/chromeos/autotest/drone/active_process_percentage changed types error (was: a lot RESOURCE_EXHAUSTED errors in master scheduelr)
Yep.
If safe, blow away data for /chrome/infra/chromeos/autotest/drone/active_process_percentage and this will go away.
Mergedinto: 709178
Status: Duplicate (was: Untriaged)

Sign in to add a comment