It might be worth doing some sample CL testing. I suspect there is something causing PreCQ builds to be picked up very slowly, but have no idea what yet.

I have a metrics CL out for review that will let us see how busy the PreCQ builders pool is, but that's only one potential cause for this kind of delay.

Um... sample CL testing means select a small number and roughly timeline out what happens when.

You should be able to construct a pretty complete picture from the clActionTable entries for a given CL.

-> dgarrett@ is looking at pre-cq timeline for CLs at the moment.

Picking a random CL (mine):

https://chromium-review.googlesource.com/#/c/536353/

Gerrit flags set:           17:06
Launcher launches builders: 17:11:50
All builder launch:         ~17:12
cbuildbot:                   17:15 to 17:20
CL comment w build links:    17:24

The gap between setting flags in gerrit and the launcher scheduling PreCQ builds is 6 minutes. One sample builder took about 1 minute to launch, and 2.5 minutes to get through the build startup.

That's ~9.5 minutes from flags set to cbuildbot reached on the PreCQ builder. But on the CL, it took 24 minutes before the build links were added to the CL.

The slowest starting slave took 6 minutes to perform it's initial repo sync.

https://luci-milo.appspot.com/buildbot/chromiumos.tryserver/no_vmtest_pre_cq/58875

Removing from triage queue.