actually  bug 848346  is not the cause here. 

https://pantheon.corp.google.com/traces/traces?project=luci-milo&q=%2Fp%2Fchromium%2Fbuilders%2Fluci.chromium.try%2F&start=1527881921155&end=1527925121155&tid=a3888cfe904e39d5e51712e9ab9c8988

here milo took a lot of time to load emails of CL authors. Looks we've added slow gerrit to a hot path

we may have to revert back to including CL email authors in the build, in a tag.
it may be generally useful: you can find all your tryjobs

On the other hand, if Milo needs it it should be in the proto. Also this is another example of Gerrit/Gitiles metadata that buildbucket users need.

I’ll add author email, gitiles ref and Gerrit project to bb protos and populate them from existing v1 stuff

While having this data in buildbucket will resolve the problem, there is an easy fix: query Gerrit in parallel for each build: https://chromium-review.googlesource.com/#/c/infra/luci/luci-go/+/1084313

With CL from #8, 60s deadline is still triggered when querying 200 builds. In other words, Gerrit is really slow even when fetching just authors :( However, subsequent page loads take only ~5s, most of which is spent on fetching 200 builds from buildbucket.

Nice, I didn’t notice that they are sequential 

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/a75f9b2f009a1d36f37ceb24d73a3b5a43f54388

commit a75f9b2f009a1d36f37ceb24d73a3b5a43f54388
Author: Andrii Shyshkalov <tandrii@chromium.org>
Date: Wed Jun 06 01:39:10 2018

[milo] fetch Gerrit CL author for many builds in parallel.

R=hinoka@chromium.org, nodir@chromium.org

Bug:  848363 
Change-Id: Iec66ee0941de55ef3c978fe4a641e56f1844ef2d
Reviewed-on: https://chromium-review.googlesource.com/1084313
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/a75f9b2f009a1d36f37ceb24d73a3b5a43f54388/milo/buildsource/buildbucket/build.go
[modify] https://crrev.com/a75f9b2f009a1d36f37ceb24d73a3b5a43f54388/milo/buildsource/buildbucket/builder.go

Deployed as version 3164-a268639.

nodir@ do you want to do anything else here?

it got better now, but it is still slow because of buildbucket.
At this point I think it is fare to merge this bug to 848346 because buildbucket Search RPC takes most of the time and THAT can be fixed by using new search api