New issue
Advanced search Search tips

Issue 674095 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: 2016-12-20
OS: ----
Pri: 3
Type: Bug-Regression



Sign in to add a comment

git cl upload is slow because of a git fetch refs/meta/config

Project Member Reported by primiano@chromium.org, Dec 14 2016

Issue description

I've been waiting for some minutes for a git cl upload.
At some point while waiting I start seeing "Remote: counting objects ...."

A ps revealed that this was happening:

git fetch https://chromium.googlesource.com/chromium/src.git +refs/meta/config:refs/git_cl/meta/config +refs/gnumbd-config/main:refs/git_cl/gnumbd-config/main

Why git cl upload does a fetch? :/ 
 
Owner: tandrii@chromium.org
Status: Assigned (was: Untriaged)
I made it load that config instead of coderereview.settings to get up to date info, so that turning off gnumbd is not noticeable.

In my chrome checkout it is adds less than a second on all but the first fetch, which I reasoned was less than presubmit takes or even just checking if tree is dirty.

Does it run as slowly on subsequent invocations? The ref itself is super small, so I suspect you are also hit by that weird bug when git negotiation takes too long.

Components: Infra>SDK
NextAction: 2016-12-20
Yes your comment is spot on. The problem is that fetching from gob is unpredictable and every now and then it ends up counting and sending *all* the objects even if you want to fetch a branch which has few objects like infra. 
Out of curiosity have you considered getting the same data by poking the http gitiles fronted and using ?format= DATA which returns the base 64 encoded raw content   
Yep, I did consider it, but originally dismissed it as more complex and likely slower, as I need data from two refs, and the git fetch was also easier solution to be copied to CQ.

But this seemingly inefficient object negotiation didn't enter my mind till now. So, I think I can change git cl to do two concurrent http requests. Sadly, I can't use gerrit_util ready made methos because it doesn't respect proxy setting. I will try to hack something this evening.
Cc: tandrii@chromium.org
Labels: -Type-Bug Milestone-Launch Proj-Gerrit-Migration Type-Bug-Regression
Owner: ----
Status: Available (was: Assigned)
Primiano, I think given the rarity of changes to refs/meta/config and refs/gnumbd-config/main, this bad behavior shouldn't affect you more than 2 times again if not upgraded from git to gitiles. 

If this doesn't affect bigger number of devs, my time is better spent on bring Gerrit to chrome, as that will make this bug WontFix. But if this happens frequently, please assign me back to this bug and I'll code a workaround.
Labels: -Milestone-Launch Milestone-Afterglow

Comment 8 by aga...@chromium.org, Jan 13 2017

Since this codepath is only executed for Rietveld changes, it is not a launch blocker. It may even be wontfix, but we'll re-triage after the main migration is over.
Summary: git cl upload is slow because of a git fetch refs/meta/config (was: git cl upload is slow because of a git fetch gnumbd-config)
Update: since gnumbd is gone, only refs/meta/config is fetched.
Labels: -Milestone-Afterglow
Removing Milestone-Afterglow, as it has ceased to have meaning. More refined milestones may be added back in the near future.
Status: Fixed (was: Available)
With Rietveld gone, this issue is gone.

Sign in to add a comment