New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 616663 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

chromium-swarm.appspot.com is over quota, CQ blocked

Project Member Reported by nedngu...@google.com, Jun 2 2016

Issue description

telemetry_gpu_unittests is failing & blocking this CL:
https://codereview.chromium.org/2024733003/

I suspect this is an infra issue:
17399 2016-06-02 02:56:30.108 E: Unable to open given url, https://chromium-swarm.appspot.com/_ah/api/swarming/v1/tasks/new, after 30 attempts.
503 Server Error: Service Unavailable for url: https://chromium-swarm.appspot.com/_ah/api/swarming/v1/tasks/new
----------
Alternate-protocol: 443:quic
X-xss-protection: 1; mode=block
X-content-type-options: nosniff
Content-encoding: gzip
Transfer-encoding: chunked
Expires: Thu, 02 Jun 2016 02:56:30 GMT
Server: GSE
Cache-control: private, max-age=0
Date: Thu, 02 Jun 2016 02:56:30 GMT
X-frame-options: SAMEORIGIN
Alt-svc: quic=":443"; ma=2592000; v="34,33,32,31,30,29,28,27,26,25"
Content-type: application/json; charset=UTF-8

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "backendError",
    "message": "\n\u003chtml\u003e\u003chead\u003e\n\u003cmeta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\"\u003e\n\u003ctitle\u003e503 Over Quota\u003c/title\u003e\n\u003cstyle\u003e\u003c!--\nbody {font-family: arial,sans-serif}\ndiv.nav {margin-top: 1ex}\ndiv.nav A {font-size: 10pt; font-family: arial,sans-serif}\nspan.nav {font-size: 10pt; font-family: arial,sans-serif; font-weight: bold}\ndiv.nav A,span.big {font-size: 12pt; color: #0000cc}\ndiv.nav A {font-size: 10pt; color: black}\nA.l:link {color: #6f6f6f}\nA.u:link {color: green}\n//--\u003e\u003c/style\u003e\n\n\u003c/head\u003e\n\u003cbody text=#000000 bgcolor=#ffffff\u003e\n\u003ctable border=0 cellpadding=2 cellspacing=0 width=100%\u003e\n\u003ctr\u003e\u003ctd bgcolor=#3366cc\u003e\u003cfont face=arial,sans-serif color=#ffffff\u003e\u003cb\u003eError\u003c/b\u003e\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\u003cblockquote\u003e\n\u003cH1\u003eOver Quota\u003c/H1\u003e\nThis application is temporarily over its serving quota.  Please try again later.\n\n\u003cp\u003e\n\u003c/blockquote\u003e\n\u003ctable width=100% cellpadding=0 cellspacing=0\u003e\u003ctr\u003e\u003ctd bgcolor=#3366cc\u003e\u003cimg alt=\"\" width=1 height=4\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\u003c/body\u003e\u003c/html\u003e\n"
   }
  ],
  "code": 503,
  "message": "\n\u003chtml\u003e\u003chead\u003e\n\u003cmeta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\"\u003e\n\u003ctitle\u003e503 Over Quota\u003c/title\u003e\n\u003cstyle\u003e\u003c!--\nbody {font-family: arial,sans-serif}\ndiv.nav {margin-top: 1ex}\ndiv.nav A {font-size: 10pt; font-family: arial,sans-serif}\nspan.nav {font-size: 10pt; font-family: arial,sans-serif; font-weight: bold}\ndiv.nav A,span.big {font-size: 12pt; color: #0000cc}\ndiv.nav A {font-size: 10pt; color: black}\nA.l:link {color: #6f6f6f}\nA.u:link {color: green}\n//--\u003e\u003c/style\u003e\n\n\u003c/head\u003e\n\u003cbody text=#000000 bgcolor=#ffffff\u003e\n\u003ctable border=0 cellpadding=2 cellspacing=0 width=100%\u003e\n\u003ctr\u003e\u003ctd bgcolor=#3366cc\u003e\u003cfont face=arial,sans-serif color=#ffffff\u003e\u003cb\u003eError\u003c/b\u003e\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003ctd\u003e \u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\u003cblockquote\u003e\n\u003cH1\u003eOver Quota\u003c/H1\u003e\nThis application is temporarily over its serving quota.  Please try again later.\n\n\u003cp\u003e\n\u003c/blockquote\u003e\n\u003ctable width=100% cellpadding=0 cellspacing=0\u003e\u003ctr\u003e\u003ctd bgcolor=#3366cc\u003e\u003cimg alt=\"\" width=1 height=4\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\u003c/body\u003e\u003c/html\u003e\n"
 }
}

----------
Sending the report ...17399 2016-06-02 02:56:30.135 E: Unable to open given url, https://chromium-swarm.appspot.com/ereporter2/api/v1/on_error, after 1 attempts.
503 Server Error: Service Unavailable for url: https://chromium-swarm.appspot.com/ereporter2/api/v1/on_error
----------
Alternate-protocol: 443:quic
Content-length: 970
Server: Google Frontend
Date: Thu, 02 Jun 2016 02:56:30 GMT
Alt-svc: quic=":443"; ma=2592000; v="34,33,32,31,30,29,28,27,26,25"
Content-type: text/html


<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>503 Over Quota</title>
<style><!--
body {font-family: arial,sans-serif}
div.nav {margin-top: 1ex}
div.nav A {font-size: 10pt; font-family: arial,sans-serif}
span.nav {font-size: 10pt; font-family: arial,sans-serif; font-weight: bold}
div.nav A,span.big {font-size: 12pt; color: #0000cc}
div.nav A {font-size: 10pt; color: black}
A.l:link {color: #6f6f6f}
A.u:link {color: green}
//--></style>

</head>
<body text=#000000 bgcolor=#ffffff>
<table border=0 cellpadding=2 cellspacing=0 width=100%>
<tr><td bgcolor=#3366cc><font face=arial,sans-serif color=#ffffff><b>Error</b></td></tr>
<tr><td>&nbsp;</td></tr></table>
<blockquote>
<H1>Over Quota</H1>
This application is temporarily over its serving quota.  Please try again later.

<p>
</blockquote>
<table width=100% cellpadding=0 cellspacing=0><tr><td bgcolor=#3366cc><img alt="" width=1 height=4></td></tr></table>
</body></html>

----------
 failed!
Failed to trigger task telemetry_gpu_unittests (with patch)/Ubuntu-12.04/7c03fd04cd/linux_chromium_rel_ng/239480
step returned non-zero exit code: 1
@@@STEP_LOG_LINE@json.output (invalid)@null@@@
@@@STEP_LINK@logdog-->json.output (invalid)@https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.linux%2Flinux_chromium_rel_ng%2F239480%2F%2B%2Frecipes%2Fsteps%2Fs__trigger__telemetry_gpu_unittests__with_patch__on_Ubuntu-12.04%2F0%2Flogs%2Fjson.output__invalid_%2F0@@@
@@@STEP_LOG_END@json.output (invalid)@@@
@@@STEP_EXCEPTION@@@
@@@SET_BUILD_PROPERTY@failure_hash@"87f3e7eb97ba8955872a383ee2d6af4cf683bea6"@@@

 
Summary: chromium-swarm.appspot.com is over quota, CQ blocked (was: telemetry_gpu_unittests is failing, blocking unrelated CL)
Cc: kouhei@chromium.org
Cc: kcaratt...@chromium.org
Components: -Infra Infra>Platform>Swarming
Labels: -Pri-0 Pri-1
Owner: mar...@chromium.org
Status: Started (was: Untriaged)
I believe maruel@ has already resolved the outage. Not P0 any more, and I suspect a dup.
Project Member

Comment 6 by sheriffbot@chromium.org, Jun 2 2016

Labels: Hotlist-Google
Status: Fixed (was: Started)
This was fixed last night. A postmortem is in progress.

Sign in to add a comment