Builder statuses pages are incredibly slow to load |
||||||
Issue description<b>Version: <Kenneth, what is the frequency?></b> <b>OS: <please tell me it's not XP></b> What steps will reproduce the problem? (1) Go to https://uberchromegw.corp.google.com/i/chromium.perf/builders/Win%2010%20High-DPI%20Perf%20(1) EXPECTED: Page loads quickly. ACTUAL: Page loads slowly (>30s) This is definitely making it hard to stay on top of bot health.
,
Sep 1 2016
Hey Charlie, I'll take a look but can you get some info you need from https://luci-milo.appspot.com/ for now, e.g. https://luci-milo.appspot.com/buildbot/chromium.perf/Win%2010%20High-DPI%20Perf%20%281%29/ That's a fresh mirror of everything happening in buildbot, including build logs, that will never get deleted, either.
,
Sep 1 2016
,
Sep 1 2016
Woooah: what is that site and how is it 10,000 times faster than the old perf dashboard?!
,
Sep 1 2016
:) +hinoka who wrote it. It's part of LUCI, our new build and test infrastructure that we're (slowly) rolling out.
,
Sep 1 2016
I didn't see anything crazy like multiple chromium.perf masters running. Would it be ok if I restart this master to see if that helps?
,
Sep 1 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/master-manager.git/+/bb3bb562c0d4faea7bf4283b4c7560f67bf4dc69 commit bb3bb562c0d4faea7bf4283b4c7560f67bf4dc69 Author: estaab <estaab@google.com> Date: Thu Sep 01 22:59:26 2016
,
Sep 2 2016
Hmm, that didn't help. I logged into the master and the logs are full of: 2016-09-01 17:53:48-0700 [HTTPChannel,3547,127.0.0.1] Finished processing request with id: 14326096816 2016-09-01 17:53:48-0700 [HTTPChannel,3475,127.0.0.1] Received request for /json/builders/Mac%20Retina%20Perf%20(2)/builds/4131 from Go-http-client/1.1, id: 14273236072 2016-09-01 17:53:48-0700 [HTTPChannel,3475,127.0.0.1] Finished processing request with id: 14273236072 2016-09-01 17:53:48-0700 [HTTPChannel,3481,127.0.0.1] Received request for /json/builders/Android%20One%20Perf%20(2)/builds/4062 from Go-http-client/1.1, id: 47829709368424 so maybe some client written in Go is hammering the master, or maybe this is normal. I'm not sure what Go clients we have that would be doing this, maybe builder alerts? cc'ing martiniss and seanmccullough in case they know.
,
Sep 2 2016
we're pretty sure it was our client, alerts-dispatcher. Fix is in https://chromium-review.googlesource.com/c/380411/ - but I think it was rolled back because it quit generating alerts at all. I'm building and deploying https://chromium-review.googlesource.com/c/380378/ now
,
Sep 2 2016
Nice! Well, it's loading fast now. Feel free to close this when everything is deployed.
,
Sep 2 2016
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by nedngu...@google.com
, Sep 1 2016