New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 24 users

Issue metadata

Status: Assigned
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment
link

Issue 126394: My Google Chrome.app bundle is 1.4GB in size

Reported by asvitk...@chromium.org, May 5 2012 Project Member

Issue description

Mark, is this normal? Shouldn't AU be deleting the old directories in Versions?
 
versions.png
22.4 KB View Download

Comment 1 by asvitk...@chromium.org, May 5 2012

Summary: My Google Chrome.app bundle is 1.4GB in size

Comment 2 by mark@chromium.org, May 5 2012

Yeah, that’s wrong.

What’s your uptime?

Comment 3 by asvitk...@chromium.org, May 5 2012

$ uptime
18:44  up 4 days, 17:34, 3 users, load averages: 0.59 0.33 0.35

Or did you mean Chrome's uptime?

Comment 4 by mark@chromium.org, May 5 2012

No, I meant your uptime.

OK, let’s crank up the updater verbosity. Create a file at "/Library/Google/Google Chrome Updater Debug". The next time an update runs, lots of extra information will get dumped into the Keystone log. That will help us trace through what’s happening and figure out why the old directories aren’t getting cleaned up.

See chrome/installer/mac/keystone_install.sh.

You haven’t replaced /bin/bash, have you?

Comment 5 by asvitk...@chromium.org, May 6 2012

Nope, /bin/bash is still stock.

Created the file, let's see what happens.

Comment 6 by asvitk...@chromium.org, May 16 2012

Here's the log from 18 -> 19 update, which again failed to delete the old version directories:

.keystone_install: 21336 2012-05-15 19:46:40 -0400: checking update
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: is_patch = y
.keystone_install: 21336 2012-05-15 19:46:40 -0400: dirpatcher = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/dirpatcher.sh
.keystone_install: 21336 2012-05-15 19:46:40 -0400: reading update values
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_app_old = 18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_app = 19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_ks = 19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:40 -0400: product_id = com.google.Chrome
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_app_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/application.dirpatch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_versioned_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/version_18.0.1025.168_19.0.1084.46.dirpatch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: checking Keystone
.keystone_install: 21336 2012-05-15 19:46:40 -0400: ksadmin_path = /Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle/Contents/Frameworks/UpdateEngine.framework/../../MacOS/ksadmin
.keystone_install: 21336 2012-05-15 19:46:40 -0400: ksadmin_version_string = 1.0.9.2865
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app = /Applications/Google Chrome.app
.keystone_install: 21336 2012-05-15 19:46:40 -0400: system_ticket = 
.keystone_install: 21336 2012-05-15 19:46:40 -0400: reading install values
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app_plist = /Applications/Google Chrome.app/Contents/Info
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app_plist_path = /Applications/Google Chrome.app/Contents/Info.plist
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_version_app = 18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_versions_dir = /Applications/Google Chrome.app/Contents/Versions
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_versioned_dir = /Applications/Google Chrome.app/Contents/Versions/18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_ks_plist = /Applications/Google Chrome.app/Contents/Info
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_brand = 
.keystone_install: 21336 2012-05-15 19:46:42 -0400: creating installed_versions_dir
.keystone_install: 21336 2012-05-15 19:46:42 -0400: new_versioned_dir = /Applications/Google Chrome.app/Contents/Versions/19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:42 -0400: versioned_dir_target = /Applications/Google Chrome.app/Contents/Versions/19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:42 -0400: dirpatching versioned directory
.keystone_install: 21336 2012-05-15 19:46:57 -0400: g_temp_dir = /tmp/keystone_install.B5ILIGrR
.keystone_install: 21336 2012-05-15 19:46:57 -0400: update_app = /tmp/keystone_install.B5ILIGrR/Google Chrome.app
.keystone_install: 21336 2012-05-15 19:46:57 -0400: dirpatching app directory
.keystone_install: 21336 2012-05-15 19:46:58 -0400: needs_touch = 
.keystone_install: 21336 2012-05-15 19:46:58 -0400: rsyncing app directory
rsync: delete_file: unlink "/Applications/Google Chrome.app/Contents/Extensions/external_extensions.json" failed: Permission denied (13)
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-40/rsync/main.c(992) [sender=2.6.9]
.keystone_install: 21336 2012-05-15 19:46:58 -0400: rsync of app directory failed, status 23

Comment 7 by asvitk...@chromium.org, May 16 2012

The file in question seems to have the following permissions:

-rw-r--r--   1 root          wheel  192 20 Feb 23:37 external_extensions.json

Perhaps the updater should detect when some files have bad permissions and prompt the user for a password when updating in that case?

And orthogonally, the script should still try to perform the "delete old versions" step even if some other step failed...

Comment 8 by asvitk...@chromium.org, May 16 2012

Looking at that file, it appears it was installed by the DivX plugin, which explains why it has wrong permissions (I guess):

$ cat "/Applications/Google Chrome.app/Contents/Extensions/external_extensions.json" 

{
	"nneajnkjbffgblleaoojgaacokifdkhm" : 
	{
		"external_crx" : "/Library/Internet Plug-Ins/DivXBrowserPlugin.plugin/Contents/Resources/DivXHTML5.crx",
		"external_version" : "2.1.2.145"
	}
}

Comment 9 by mark@chromium.org, May 16 2012

First, can you give me a better idea of all of the owners and permissions involved? Please run:

find "/Applications/Google Chrome.app/Contents/Extensions" -ls

Comment 10 by mark@chromium.org, May 16 2012

I can think of a few ways I might be able to detect and work around this in the updater script. I can’t just blow past a failed rsync and start nuking things, though. A failed rsync leaves the app in an inconsistent and unknown state, but it’s at least guaranteed to launch, so it’s partially working. Since it’s inconsistent, I can’t pick which versioned directories to trash and which to keep, because I don’t actually know which one the app will want to use. I’ve got to leave all of them around.

I think that what I’ll wind up doing if I detect the presence of Contents/Extensions is to just move it to /tmp if possible, and if that fails because /tmp is on a different filesystem, to try to move it to ../../Garbage-[randomcharacters]. The external_extensions.json file in that Contents/Extensions location is no longer supported (and was never supported, if you ask me) and is no longer respected.

Comment 11 by asvitk...@chromium.org, May 17 2012

52421317        0 drwxr-xr-x    3 root     wheel         102 20 Feb 23:36 /Applications/Google Chrome.app/Contents/Extensions
52421326        8 -rw-r--r--    1 root     wheel         192 20 Feb 23:37 /Applications/Google Chrome.app/Contents/Extensions/external_extensions.json

Given that they're both root, I'm pretty certain they were created by the DiVX installed and I guess this issue probably affects other users who have installed that plugin.

Comment 12 by mark@chromium.org, Jun 12 2012

I can’t do what I planned to do in comment 10 because the Mac doesn’t let me move a directory aside when I can’t write to the directory itself AND it contains files. (I can move the directory aside when it doesn’t have anything in it, but then I could have removed it, too.)

Comment 13 by mark@chromium.org, Jun 22 2012

Cc: aharper@chromium.org mark@chromium.org
 Issue 134139  has been merged into this issue.

Comment 14 by Deleted ...@, Aug 8 2012

I'm having the same problem, my Chrome app ended up being > 2 GB.

The Contents/Extensions dir might be my problem as well
drwxr-xr-x   3 root   wheel   102B Feb 15 16:05 Extensions/
with
-rw-r--r--  1 root   wheel   192B Aug  8 12:50 external_extensions.json
(I ran a divX update today.)

May i suggest to run the rsync as you do now, but if that fails to run it again just without the --delete ?
If that works you just weren't able to delete some files, which shouldn't be too bad, so you could proceed and delete all the old versions?

Finally just some terms for google (as i couldn't find this before being pointed to it):
Chrome app Mac OS X huge size, big filesize, autoupdater problem

Comment 15 by mark@chromium.org, Aug 14 2012

If you’re seeing this problem, the best thing you can do to recover is “set up automatic updates for all users” in Chrome. That allows the updater to run as root, so that it will have permission to clean up after poorly-behaved installers like the DivX one that litters our application with junk. Click the button in “About Google Chrome” and authenticate when prompted.

Beyond that, there isn’t really anything we can do here mitigation-wise. Affected users should take complaints to DivX, since they’re causing the problem by modifying our application bundle.

Comment 16 by asvitk...@chromium.org, Aug 14 2012

Mark, couldn't we make the updater detect the incorrect permissions and bring up an admin prompt that will relaunch itself in an elevated mode so that it could fix them?

e.g. via Authorization Services: https://developer.apple.com/library/mac/#documentation/security/Reference/authorization_ref/Reference/reference.html

Comment 17 by asvitk...@chromium.org, Aug 14 2012

Or at least perhaps just detect this and surface the suggestion for users to go "set up automatic updates for all users".

Comment 18 by mark@chromium.org, Aug 14 2012

The updater doesn’t run in a context where it’s possible to do any UI.

Comment 19 by asvitk...@chromium.org, Aug 14 2012

Could it communicate with the current running Chrome instance (if any), so that Chrome surfaces something?

Comment 20 by mark@chromium.org, Aug 14 2012

Owner: ----
Status: Available
I don’t want to complicate the updater for something like this.

About all I could see doing here is detecting this in the app itself and setting the state to “needs promotion,” or a variation on “needs promotion” that, in addition to offering the user the promotion UI, links to a help center article explaining that some third-party software did something evil.

Comment 21 by abcarne...@gmail.com, Feb 10 2013

I'm seeing this problem as well and my bundle currently includes 4.39 GB of versions. Any solution to this besides manually deleting the old versions?

Comment 22 by bugdroid1@chromium.org, Mar 10 2013

Project Member
Labels: -Area-Internals Cr-Internals

Comment 23 by mark@chromium.org, Apr 12 2013

Cc: sail@chromium.org thakis@chromium.org lafo...@chromium.org
 Issue 230343  has been merged into this issue.

Comment 24 by asvitk...@chromium.org, May 11 2013

 Issue 240009  has been merged into this issue.

Comment 25 Deleted

Comment 26 by chad.bur...@gmail.com, Aug 15 2013

Just a followup note--I had a similar problem, though it wasn't caused by a mis-behaving extension.  Rather, mine was caused by my Chrome being installed by a different user.  I.e, you could see this problem show up any time you have multiple users installing apps on the same computer without the "set up automatic updates for all users" option set.  If any documentation is added to address this issue, you should include that, as well as a pointer to that "set up automatic updates for all users" option.

Comment 27 by dxie@chromium.org, Jul 17 2014

Cc: futurapr...@gmail.com
 Issue 390341  has been merged into this issue.

Comment 28 by Deleted ...@, Mar 15 2015

Is there an accessible solution for this problem?

Comment 29 by mark@chromium.org, Mar 16 2015

Cc: rsesek@chromium.org
 Issue 464541  has been merged into this issue.

Comment 30 by mark@chromium.org, Mar 16 2015

The easiest solution is to let the auto-updater run as root by using the “Set Up Automatic Updates for All Users” button in chrome://help. You will need to authenticate with an administrator’s credentials. When Chrome’s auto-update runs as root, it should be able to clean up these old versions.

If you’re in this state because badly-behaved third-party software has made changes to Chrome’s files on disk, please contact the developers of that software, and let them know how their mucking around in Chrome’s internals has caused a colossal waste of disk space.

Comment 31 by jordan.b...@gmail.com, Apr 6 2015

My chrome is > 7 GB. I will try the "Set Up Automatic Updates for All Users" workaround.

Comment 32 by rsesek@chromium.org, Apr 9 2015

 Issue 475656  has been merged into this issue.

Comment 33 by rsesek@chromium.org, Apr 9 2015

Is there any way we can detect this situation from within Chrome and prompt the user to promote to a machine install? That seems like a better experience than chewing up disk space with old code.

Comment 34 by mark@chromium.org, Apr 9 2015

We could count the number of old versions and prompt (infobar?) if it's over some threshold (5?)

Comment 35 by asvitk...@chromium.org, Apr 24 2015

Cc: pinkerton@chromium.org
+pink to get it on his radar.

It seems like there's support for actually doing something about this - so maybe worth considering for Mac team okrs?

Comment 36 by pinkerton@chromium.org, Apr 27 2015

We should at least get ConOps involved so they can be on the lookout for issues and point people to the correct solution. Is Melody the right person?

Mark: comment #26 mentions just installing chrome as a different user causes this problem too. So we shouldn't be quick to point the finger at other poorly written installers, right? Or am I misreading what's going on?

Comment 37 by mark@chromium.org, Apr 27 2015

That’s correct, although when we detect that Chrome is installed by a different (non-root) user, we do show the in-app prompt infobar to promote it to a system ticket and system Keystone. We don’t do anything to detect poorly-written installers.

Comment 38 by rsesek@chromium.org, Jun 15 2015

Cc: erikc...@chromium.org jazon.bu...@gmail.com
 Issue 500727  has been merged into this issue.

Comment 39 by rsesek@chromium.org, Aug 26 2015

 Issue 525178  has been merged into this issue.

Comment 40 by Deleted ...@, Nov 21 2015

Mine is 14.38GB! WTF

Comment 41 by walterni...@gmail.com, Dec 2 2016

15.1 GB is a DOS attack on my laptop. I have Chrome 24 from 2013! If you are under attack from DivX, I expect you to defend yourself. How about you ask for privilege when the updater indicates a warning state? Or ignore that particular path?

Comment 42 by pinkerton@chromium.org, Dec 2 2016

Cc: shrike@chromium.org
+shrike

Comment 43 by lafo...@chromium.org, Dec 2 2016

Cc: -lafo...@chromium.org anan...@chromium.org kerz@chromium.org
+kerz, anantha
-laforge

Comment 44 by shrike@chromium.org, Dec 2 2016

Cc: borisv@chromium.org
borisv@ - will the change to auto-promote Chrome to system install solve this problem and clean up the old versions?

Comment 45 by borisv@chromium.org, Dec 6 2016

Auto-promoting happens only if the update fails first. Also, the user must have system Keystone installed already (~30% of the users). I am not sure that the update fails if it cannot delete old folders. 
Mark (mark@chromium.org), what do you think? 
Here is the code that deletes the old folders: https://cs.chromium.org/chromium/src/chrome/installer/mac/keystone_install.sh?rcl=0&l=1345

Comment 46 by mark@chromium.org, Dec 6 2016

Let’s detect the failure-to-delete in .keystone_install and leave a marker somewhere that we can pick up in Chrome and use as an additional wants-promotion signal.

Comment 47 by shrike@chromium.org, Dec 6 2016

Owner: borisv@chromium.org
Status: Assigned (was: Available)
borisv@ - can I assign this to you for the suggested change in #46?

Comment 48 by borisv@chromium.org, Dec 6 2016

Sure, but I need some guidance from Mark.

Mark, can the script return non-zero after Chrome is updated in this case? Say error 15? Alternatively, we can decide to do that only if system Keystone is present, by checking existence of /Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle. 
Or Keystone can pass an environment variable to the script: KS_SYSTEM_KEYSTONE set to "YES"/"NO". I can implement this today and fix it in our December release.

Comment 49 by mark@chromium.org, Dec 6 2016

The script can return nonzero for this if you need to communicate something to Keystone. I don’t think that we need to gate that on whether system Keystone is present.

We’d still need the script to write the marker in addition to having a discrete exit code, because the if Chrome’s about page isn’t driving the update request, Chrome would never learn about the exit code.

Comment 50 by mark@chromium.org, Dec 6 2016

(The discrete code that we you still should be interpreted as “update success” to Keystone for all purposes other than “wants system promotion.”)

Comment 51 by borisv@chromium.org, Dec 6 2016

"We’d still need the script to write the marker in addition to having a discrete exit code, because the if Chrome’s about page isn’t driving the update request, Chrome would never learn about the exit code."
Why does Chrome care?

'The discrete code that we you still should be interpreted as “update success” to Keystone for all purposes other than “wants system promotion.”'
What is the downside of reporting error 15 all the way? We will see it in the dashboards as error, but we will know about it. If everything goes right, we should see it only once, as promotion will succeed otherwise. At minimum we will know how common the problem is and how much we need to act on it from Chrome itself.

Comment 52 by mark@chromium.org, Dec 6 2016

> Why does Chrome care?

If there’s no system Keystone, we want to show the promotion infobar so that we can get system Keystone installed, and get Chrome on a system ticket.

> What is the downside of reporting error 15 all the way?

There was no error from Chrome’s perspective. “Failure to clean up” is not an error because Chrome was update. If the update was driven from Chrome’s about page, we want to show it as success and offer the restart button.

You can carry it as an error on the reporting end, but I think that the KeystoneRegistration client (or at least THIS KeystoneRegistration client) should treat it as success.

Comment 53 by borisv@chromium.org, Dec 22 2016

Based on the discussion above, we can do this when we fail to clean up older versions:

1. The script reports success when invoked by the user: ${KS_USER_INITIATED} will be set to 'YES', even if the cleanup was unsuccessful.
2. If not user-initiated (aka not launched by Chrome about page), the script returns a predefined error when the cleanup did not succeed. Error 15 is ok and Keystone does not need to be aware of it. This will result in:
      a) Error 15 will be reported all the way to the auto update server
      b) If system Keystone is present, Keystone will automatically attempt to promote the ticket (this logic is already implemented).
3. The script always writes down a marker about the ability to clean up these folders. E.g. ${HOME}/Library/Application\ Support/${KS_TICKET_PRODUCT_ID}.update_state.plist
4. Chrome can read that marker to determine the state of the cleanup, suggesting promotion.

This course of action requires no Chrome-specific change from Keystone (we do not have other applications that get into this situation).

Comment 54 by meh...@chromium.org, Dec 28 2017

 Issue 797959  has been merged into this issue.

Comment 55 by meh...@chromium.org, Dec 28 2017

FYI: The Canary file size from the reporter of the merged  issue 797959  is about 30GB :-(

Comment 56 by shrike@chromium.org, Jan 2 2018

My Canary contains two framework version folders (65.0.3295.0 and 65.0.3309.0), and the older one is not going away when I launch the app. Canary is 367MB on my machine.

Comment 57 by mark@chromium.org, Jan 2 2018

No deletion happens at app launch (with a system ticket, this wouldn’t even be possible unless you ran Chrome as root—not recommended). All deletion happens at update time, right after the update has been applied.

The logic in the updater is:

 - Preserve the new version being updated “to” (of course)
 - Preserve the version being updated “from”
 - Preserve any version that appears to be in use (based on “lsof” and “ps”)
 - Delete everything else.

Once the updater has applied at least one update, there will always be at least two directories in Versions.

Comment 58 by borisv@google.com, Feb 5

Owner: norberg@google.com

Sign in to add a comment