New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 758747 link

Starred by 2 users

Issue metadata

Status: Duplicate
Merged: issue 764042
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Gerrit is very slow

Project Member Reported by eugene...@chromium.org, Aug 24 2017

Issue description

Loading codereview page and going to the next file takes non-trivial amount of time for me
 
Status: Fixed (was: Untriaged)
This issue has been resolved.
Status: Untriaged (was: Fixed)
I just opened CL from email, clicked on the first file and it tool like 8 seconds to load this page for me:
https://chromium-review.googlesource.com/c/chromium/src/+/627598/4/ios/chrome/test/earl_grey/chrome_earl_grey.h


Components: -Infra Infra>Codereview>Gerrit

Comment 4 by pkl@chromium.org, Aug 28 2017

Cc: pkl@chromium.org
Labels: OS-Android OS-Chrome OS-Fuchsia OS-Linux OS-Mac OS-Windows
Owner: aga...@chromium.org
Status: Assigned (was: Untriaged)
Is it possible to report on the overall performance dashboard/trend of gerrit so we know if the slowness reported here is a one-time thing or a general problem for all gerrit users?

Comment 5 by aga...@chromium.org, Aug 28 2017

Status: Fixed (was: Assigned)
Yes, we track outages in chopsdash.appspot.com, and you can see any graph you care to name at http://vi/gerritcodereview/gerrit. 

Comment 6 Deleted

Comment 7 by aga...@chromium.org, Aug 28 2017

Labels: -OS-Linux -OS-Android -OS-Windows -OS-iOS -OS-Chrome -OS-Mac -OS-Fuchsia
Please avoid putting .corp URLs in external bugs.

Comment 8 by pkl@chromium.org, Aug 28 2017

Sorry. Looking at the dashboard, it appears that there was indeed an increase in latency around middle of the day of August 24 (which coincided with the report date of this crbug).
Once again it took ~5 seconds to fully load the page. After reloading I've got blank page and then had to wait until it's content is loaded. Do you have any monitoring tools to measure load latency? Do you think it's only a problem on my end? 

I don't want to reopen the bug again, but Geritt quite often is slow for me. I never had similar issues with Rietveld. 
Cc: sop@google.com
The GoB team is working very hard on three classes of performance problems: slow loading of Gerrit pages, slow fetches (especially of refs/changes/* refs), and slow pushes (especially to refs/changes/* refs). All of these problems are interconnected, and are due to the combination of the large size of our repository and the large rate of creation of changes refs. (Incidentally, Android is way worse off than we are on all of these metrics, so that's fun.) All of these are being tracked in b/ bugs by the GoB team.
Thank you for the detailed answer.
Cc: jparent@chromium.org
Status: Assigned (was: Fixed)
It has been 2 month and performance has not improved. 5 seconds is enough for me to get distracted and start working on something else which significantly affects my productivity.

I do not agree that closing this bug is a reasonable resolution. External Dependency or Duplicate can be a valid status, but not Fixed.

Julie, is it possible to prioritize Gerrit performance work?  
Cc: logan@google.com
My understanding is that this *is* the top priority for Gerrit team.

Aaron, Shawn, Logan - can you confirm?
Shawn is in the best position to provide details about priority, progress, and timelines. But I can say that I have been reassured by Shawn multiple times that performance is the Gerrit backend team's top priority.
Thank you for the update. Do you have references to b/ bugs mentioned in comment #10?

Comment 16 by sop@google.com, Nov 14 2017

b/35998605 is the internal master bug that we believe root causes the performance problems. Its being worked on full time by engineers, and has been for months.

Unfortunately its a difficult backend data migration that must take place to correct the problem. We've been uncovering some "unknown unknowns" related to how to do this safely without breaking the entire system and causing a work stoppage for our customers. Resolving these and iterating has been slowing progress.

We are also subject to some rollout speed concerns; we only get 1 release per week. If we uncover something about a new release on Monday, we have to wait a full 7 days before we can resolve whatever that was and try again. That has been severely slowing our calendar. If it were possible to iterate faster, we would uncover these "unknown unknowns" more rapidly.

As I told Julie, this is a top priority to my team. We are working on it.
Mergedinto: 764042
Status: Duplicate (was: Assigned)

Sign in to add a comment