New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 626824 link

Starred by 3 users

Issue metadata

Status: Archived
Owner:
Last visit > 30 days ago
Closed: Jan 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

manifest-versions/.git is now 889M in size.

Project Member Reported by dgarr...@chromium.org, Jul 8 2016

Issue description

Manifest versions has gotten really big in size again. Perhaps we can kill history again to make it more manageable.
 

Comment 1 by autumn@chromium.org, Jul 12 2016

Is this the proposed shallow checkout fix? Who should own this? 
We can short term fix by squashing history for this repo (which is what we've done in the past), but the number of PreCQ runs has the history growing much, much faster than it used too. This requires elevated Git permissions, but is quick/easy if you have them and know how (David or Sosa).

We can shallow checkout in cbuildbot to generally reduce the impact the size has. Probably a small chance.
s/chance/change/

Comment 4 by aut...@google.com, Jul 19 2016

Labels: -current-issue Hotlist-Fixit
Owner: dgarr...@chromium.org
don, assigning to you for now, pls assign to Ningning if that's best
Owner: nxia@chromium.org
Owner: dgarr...@chromium.org
trybot .git is now 1.6G.
Labels: -Pri-2 Pri-1
Let's do the squash during fixit week once and for all (and probably for the last time, since we are rapidly moving to buildbucket for everything)
Status: Available (was: Untriaged)
Owner: ----
Owner: chingcodes@chromium.org
The internal one is the one that triggered this bug and which sees the most use, but we should check the external one as well.
Should I also do the trybot repos?

chromeos/tryjobs
chromiumos/tryjobs
See how big they are. In particular, compare the .git contents to the non-.git contents. But if it'll help... excellent!

Try not to break an in-progress builds/tryjobs. I have no idea how to manage that, except that missing files will break things, but missing git history doesn't matter (as I understand it).
5.7G    chromeos/manifest-versions
2.1G    chromeos/tryjobs
2.0G    chromiumos/manifest-versions
30M     chromiumos/tryjobs

Dave said that it shouldn't effect in-flight jobs.

Writing a script that will try and do everything as fast as possible to keep everything in sync.
1.1G    chromeos/manifest-versions/.git
1.6G    chromeos/tryjobs/.git
214M    chromiumos/manifest-versions/.git
9.8M    chromiumos/tryjobs/.git
Hum... worth measuring the compressed .git size as well to make sure it really helps.
Added script to be used for auto-squashing repos

example:
git clone https://chromium.googlesource.com/chromiumos/tryjobs
./squash-repo.sh tryjobs
squash-repo.sh
1.1 KB View Download
Working with deputy (Aviv) to close tree before doing manifest-versions repos.

ETA: tonight
Labels: cros-infra-fixedit-q117
Status: Fixed (was: Available)
Maintenance complete.
Attaching final script used.

Commands run:
$ git clone https://chromium.googlesource.com/chromiumos/tryjobs tryjobs.chromiumos.git
$ git clone https://chromium.googlesource.com/chromiumos/manifest-versions manifest-versions.chromiumos.git
$ git clone https://chrome-internal.googlesource.com/chromeos/manifest-versions manifest-versions.chromeos.git
$ git clone https://chrome-internal.googlesource.com/chromeos/tryjobs tryjobs.chromeos.git
$ ./squash-repo.sh *.git
squash-repo.sh
1.1 KB View Download
Can we check that script in somewhere?

chromite/scripts *MIGHT* be the right place.
hmm, I would want to clean it up some more if we did that. Kinda scary what it does (force-push and all).

Comment 24 by dchan@google.com, Apr 17 2017

Labels: VerifyIn-59

Comment 25 by dchan@google.com, May 30 2017

Labels: VerifyIn-60
Labels: VerifyIn-61

Comment 27 by dchan@chromium.org, Oct 14 2017

Status: Archived (was: Fixed)

Sign in to add a comment