New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 126394 link

Starred by 24 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

My Google Chrome.app bundle is 1.4GB in size

Project Member Reported by asvitk...@chromium.org, May 5 2012

Issue description

Mark, is this normal? Shouldn't AU be deleting the old directories in Versions?
 
versions.png
22.4 KB View Download
Summary: My Google Chrome.app bundle is 1.4GB in size

Comment 2 by mark@chromium.org, May 5 2012

Yeah, that’s wrong.

What’s your uptime?
$ uptime
18:44  up 4 days, 17:34, 3 users, load averages: 0.59 0.33 0.35

Or did you mean Chrome's uptime?

Comment 4 by mark@chromium.org, May 5 2012

No, I meant your uptime.

OK, let’s crank up the updater verbosity. Create a file at "/Library/Google/Google Chrome Updater Debug". The next time an update runs, lots of extra information will get dumped into the Keystone log. That will help us trace through what’s happening and figure out why the old directories aren’t getting cleaned up.

See chrome/installer/mac/keystone_install.sh.

You haven’t replaced /bin/bash, have you?
Nope, /bin/bash is still stock.

Created the file, let's see what happens.
Here's the log from 18 -> 19 update, which again failed to delete the old version directories:

.keystone_install: 21336 2012-05-15 19:46:40 -0400: checking update
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: is_patch = y
.keystone_install: 21336 2012-05-15 19:46:40 -0400: dirpatcher = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/dirpatcher.sh
.keystone_install: 21336 2012-05-15 19:46:40 -0400: reading update values
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_app_old = 18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_app = 19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:40 -0400: update_version_ks = 19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:40 -0400: product_id = com.google.Chrome
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_app_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/application.dirpatch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: patch_versioned_dir = /tmp/UpdateEngine-mount.jAvTahbCRL/.patch/version_18.0.1025.168_19.0.1084.46.dirpatch
.keystone_install: 21336 2012-05-15 19:46:40 -0400: checking Keystone
.keystone_install: 21336 2012-05-15 19:46:40 -0400: ksadmin_path = /Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle/Contents/Frameworks/UpdateEngine.framework/../../MacOS/ksadmin
.keystone_install: 21336 2012-05-15 19:46:40 -0400: ksadmin_version_string = 1.0.9.2865
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app = /Applications/Google Chrome.app
.keystone_install: 21336 2012-05-15 19:46:40 -0400: system_ticket = 
.keystone_install: 21336 2012-05-15 19:46:40 -0400: reading install values
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app_plist = /Applications/Google Chrome.app/Contents/Info
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_app_plist_path = /Applications/Google Chrome.app/Contents/Info.plist
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_version_app = 18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: installed_versions_dir = /Applications/Google Chrome.app/Contents/Versions
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_versioned_dir = /Applications/Google Chrome.app/Contents/Versions/18.0.1025.168
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_ks_plist = /Applications/Google Chrome.app/Contents/Info
.keystone_install: 21336 2012-05-15 19:46:40 -0400: old_brand = 
.keystone_install: 21336 2012-05-15 19:46:42 -0400: creating installed_versions_dir
.keystone_install: 21336 2012-05-15 19:46:42 -0400: new_versioned_dir = /Applications/Google Chrome.app/Contents/Versions/19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:42 -0400: versioned_dir_target = /Applications/Google Chrome.app/Contents/Versions/19.0.1084.46
.keystone_install: 21336 2012-05-15 19:46:42 -0400: dirpatching versioned directory
.keystone_install: 21336 2012-05-15 19:46:57 -0400: g_temp_dir = /tmp/keystone_install.B5ILIGrR
.keystone_install: 21336 2012-05-15 19:46:57 -0400: update_app = /tmp/keystone_install.B5ILIGrR/Google Chrome.app
.keystone_install: 21336 2012-05-15 19:46:57 -0400: dirpatching app directory
.keystone_install: 21336 2012-05-15 19:46:58 -0400: needs_touch = 
.keystone_install: 21336 2012-05-15 19:46:58 -0400: rsyncing app directory
rsync: delete_file: unlink "/Applications/Google Chrome.app/Contents/Extensions/external_extensions.json" failed: Permission denied (13)
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-40/rsync/main.c(992) [sender=2.6.9]
.keystone_install: 21336 2012-05-15 19:46:58 -0400: rsync of app directory failed, status 23

The file in question seems to have the following permissions:

-rw-r--r--   1 root          wheel  192 20 Feb 23:37 external_extensions.json

Perhaps the updater should detect when some files have bad permissions and prompt the user for a password when updating in that case?

And orthogonally, the script should still try to perform the "delete old versions" step even if some other step failed...
Looking at that file, it appears it was installed by the DivX plugin, which explains why it has wrong permissions (I guess):

$ cat "/Applications/Google Chrome.app/Contents/Extensions/external_extensions.json" 

{
	"nneajnkjbffgblleaoojgaacokifdkhm" : 
	{
		"external_crx" : "/Library/Internet Plug-Ins/DivXBrowserPlugin.plugin/Contents/Resources/DivXHTML5.crx",
		"external_version" : "2.1.2.145"
	}
}



Comment 9 by mark@chromium.org, May 16 2012

First, can you give me a better idea of all of the owners and permissions involved? Please run:

find "/Applications/Google Chrome.app/Contents/Extensions" -ls

Comment 10 by mark@chromium.org, May 16 2012

I can think of a few ways I might be able to detect and work around this in the updater script. I can’t just blow past a failed rsync and start nuking things, though. A failed rsync leaves the app in an inconsistent and unknown state, but it’s at least guaranteed to launch, so it’s partially working. Since it’s inconsistent, I can’t pick which versioned directories to trash and which to keep, because I don’t actually know which one the app will want to use. I’ve got to leave all of them around.

I think that what I’ll wind up doing if I detect the presence of Contents/Extensions is to just move it to /tmp if possible, and if that fails because /tmp is on a different filesystem, to try to move it to ../../Garbage-[randomcharacters]. The external_extensions.json file in that Contents/Extensions location is no longer supported (and was never supported, if you ask me) and is no longer respected.
52421317        0 drwxr-xr-x    3 root     wheel         102 20 Feb 23:36 /Applications/Google Chrome.app/Contents/Extensions
52421326        8 -rw-r--r--    1 root     wheel         192 20 Feb 23:37 /Applications/Google Chrome.app/Contents/Extensions/external_extensions.json

Given that they're both root, I'm pretty certain they were created by the DiVX installed and I guess this issue probably affects other users who have installed that plugin.

Comment 12 by mark@chromium.org, Jun 12 2012

I can’t do what I planned to do in comment 10 because the Mac doesn’t let me move a directory aside when I can’t write to the directory itself AND it contains files. (I can move the directory aside when it doesn’t have anything in it, but then I could have removed it, too.)

Comment 13 by mark@chromium.org, Jun 22 2012

Cc: aharper@chromium.org mark@chromium.org
 Issue 134139  has been merged into this issue.

Comment 14 by Deleted ...@, Aug 8 2012

I'm having the same problem, my Chrome app ended up being > 2 GB.

The Contents/Extensions dir might be my problem as well
drwxr-xr-x   3 root   wheel   102B Feb 15 16:05 Extensions/
with
-rw-r--r--  1 root   wheel   192B Aug  8 12:50 external_extensions.json
(I ran a divX update today.)

May i suggest to run the rsync as you do now, but if that fails to run it again just without the --delete ?
If that works you just weren't able to delete some files, which shouldn't be too bad, so you could proceed and delete all the old versions?

Finally just some terms for google (as i couldn't find this before being pointed to it):
Chrome app Mac OS X huge size, big filesize, autoupdater problem

Comment 15 by mark@chromium.org, Aug 14 2012

If you’re seeing this problem, the best thing you can do to recover is “set up automatic updates for all users” in Chrome. That allows the updater to run as root, so that it will have permission to clean up after poorly-behaved installers like the DivX one that litters our application with junk. Click the button in “About Google Chrome” and authenticate when prompted.

Beyond that, there isn’t really anything we can do here mitigation-wise. Affected users should take complaints to DivX, since they’re causing the problem by modifying our application bundle.
Mark, couldn't we make the updater detect the incorrect permissions and bring up an admin prompt that will relaunch itself in an elevated mode so that it could fix them?

e.g. via Authorization Services: https://developer.apple.com/library/mac/#documentation/security/Reference/authorization_ref/Reference/reference.html
Or at least perhaps just detect this and surface the suggestion for users to go "set up automatic updates for all users".

Comment 18 by mark@chromium.org, Aug 14 2012

The updater doesn’t run in a context where it’s possible to do any UI.
Could it communicate with the current running Chrome instance (if any), so that Chrome surfaces something?

Comment 20 by mark@chromium.org, Aug 14 2012

Owner: ----
Status: Available
I don’t want to complicate the updater for something like this.

About all I could see doing here is detecting this in the app itself and setting the state to “needs promotion,” or a variation on “needs promotion” that, in addition to offering the user the promotion UI, links to a help center article explaining that some third-party software did something evil.
I'm seeing this problem as well and my bundle currently includes 4.39 GB of versions. Any solution to this besides manually deleting the old versions?
Project Member

Comment 22 by bugdroid1@chromium.org, Mar 10 2013

Labels: -Area-Internals Cr-Internals

Comment 23 by mark@chromium.org, Apr 12 2013

Cc: sail@chromium.org thakis@chromium.org lafo...@chromium.org
 Issue 230343  has been merged into this issue.
 Issue 240009  has been merged into this issue.

Comment 25 Deleted

Just a followup note--I had a similar problem, though it wasn't caused by a mis-behaving extension.  Rather, mine was caused by my Chrome being installed by a different user.  I.e, you could see this problem show up any time you have multiple users installing apps on the same computer without the "set up automatic updates for all users" option set.  If any documentation is added to address this issue, you should include that, as well as a pointer to that "set up automatic updates for all users" option.

Comment 27 by dxie@chromium.org, Jul 17 2014

Cc: futurapr...@gmail.com
 Issue 390341  has been merged into this issue.

Comment 28 by Deleted ...@, Mar 15 2015

Is there an accessible solution for this problem?

Comment 29 by mark@chromium.org, Mar 16 2015

Cc: rsesek@chromium.org
 Issue 464541  has been merged into this issue.

Comment 30 by mark@chromium.org, Mar 16 2015

The easiest solution is to let the auto-updater run as root by using the “Set Up Automatic Updates for All Users” button in chrome://help. You will need to authenticate with an administrator’s credentials. When Chrome’s auto-update runs as root, it should be able to clean up these old versions.

If you’re in this state because badly-behaved third-party software has made changes to Chrome’s files on disk, please contact the developers of that software, and let them know how their mucking around in Chrome’s internals has caused a colossal waste of disk space.
My chrome is > 7 GB. I will try the "Set Up Automatic Updates for All Users" workaround.
 Issue 475656  has been merged into this issue.
Is there any way we can detect this situation from within Chrome and prompt the user to promote to a machine install? That seems like a better experience than chewing up disk space with old code.

Comment 34 by mark@chromium.org, Apr 9 2015

We could count the number of old versions and prompt (infobar?) if it's over some threshold (5?)
Cc: pinkerton@chromium.org
+pink to get it on his radar.

It seems like there's support for actually doing something about this - so maybe worth considering for Mac team okrs?
We should at least get ConOps involved so they can be on the lookout for issues and point people to the correct solution. Is Melody the right person?

Mark: comment #26 mentions just installing chrome as a different user causes this problem too. So we shouldn't be quick to point the finger at other poorly written installers, right? Or am I misreading what's going on?

Comment 37 by mark@chromium.org, Apr 27 2015

That’s correct, although when we detect that Chrome is installed by a different (non-root) user, we do show the in-app prompt infobar to promote it to a system ticket and system Keystone. We don’t do anything to detect poorly-written installers.
Cc: erikc...@chromium.org jazon.bu...@gmail.com
 Issue 500727  has been merged into this issue.
 Issue 525178  has been merged into this issue.

Comment 40 by Deleted ...@, Nov 21 2015

Mine is 14.38GB! WTF
15.1 GB is a DOS attack on my laptop. I have Chrome 24 from 2013! If you are under attack from DivX, I expect you to defend yourself. How about you ask for privilege when the updater indicates a warning state? Or ignore that particular path?
Cc: shrike@chromium.org
+shrike
Cc: -lafo...@chromium.org anan...@chromium.org kerz@chromium.org
+kerz, anantha
-laforge

Cc: borisv@chromium.org
borisv@ - will the change to auto-promote Chrome to system install solve this problem and clean up the old versions?

Auto-promoting happens only if the update fails first. Also, the user must have system Keystone installed already (~30% of the users). I am not sure that the update fails if it cannot delete old folders. 
Mark (mark@chromium.org), what do you think? 
Here is the code that deletes the old folders: https://cs.chromium.org/chromium/src/chrome/installer/mac/keystone_install.sh?rcl=0&l=1345

Comment 46 by mark@chromium.org, Dec 6 2016

Let’s detect the failure-to-delete in .keystone_install and leave a marker somewhere that we can pick up in Chrome and use as an additional wants-promotion signal.
Owner: borisv@chromium.org
Status: Assigned (was: Available)
borisv@ - can I assign this to you for the suggested change in #46?

Sure, but I need some guidance from Mark.

Mark, can the script return non-zero after Chrome is updated in this case? Say error 15? Alternatively, we can decide to do that only if system Keystone is present, by checking existence of /Library/Google/GoogleSoftwareUpdate/GoogleSoftwareUpdate.bundle. 
Or Keystone can pass an environment variable to the script: KS_SYSTEM_KEYSTONE set to "YES"/"NO". I can implement this today and fix it in our December release.

Comment 49 by mark@chromium.org, Dec 6 2016

The script can return nonzero for this if you need to communicate something to Keystone. I don’t think that we need to gate that on whether system Keystone is present.

We’d still need the script to write the marker in addition to having a discrete exit code, because the if Chrome’s about page isn’t driving the update request, Chrome would never learn about the exit code.

Comment 50 by mark@chromium.org, Dec 6 2016

(The discrete code that we you still should be interpreted as “update success” to Keystone for all purposes other than “wants system promotion.”)
"We’d still need the script to write the marker in addition to having a discrete exit code, because the if Chrome’s about page isn’t driving the update request, Chrome would never learn about the exit code."
Why does Chrome care?

'The discrete code that we you still should be interpreted as “update success” to Keystone for all purposes other than “wants system promotion.”'
What is the downside of reporting error 15 all the way? We will see it in the dashboards as error, but we will know about it. If everything goes right, we should see it only once, as promotion will succeed otherwise. At minimum we will know how common the problem is and how much we need to act on it from Chrome itself.

Comment 52 by mark@chromium.org, Dec 6 2016

> Why does Chrome care?

If there’s no system Keystone, we want to show the promotion infobar so that we can get system Keystone installed, and get Chrome on a system ticket.

> What is the downside of reporting error 15 all the way?

There was no error from Chrome’s perspective. “Failure to clean up” is not an error because Chrome was update. If the update was driven from Chrome’s about page, we want to show it as success and offer the restart button.

You can carry it as an error on the reporting end, but I think that the KeystoneRegistration client (or at least THIS KeystoneRegistration client) should treat it as success.
Based on the discussion above, we can do this when we fail to clean up older versions:

1. The script reports success when invoked by the user: ${KS_USER_INITIATED} will be set to 'YES', even if the cleanup was unsuccessful.
2. If not user-initiated (aka not launched by Chrome about page), the script returns a predefined error when the cleanup did not succeed. Error 15 is ok and Keystone does not need to be aware of it. This will result in:
      a) Error 15 will be reported all the way to the auto update server
      b) If system Keystone is present, Keystone will automatically attempt to promote the ticket (this logic is already implemented).
3. The script always writes down a marker about the ability to clean up these folders. E.g. ${HOME}/Library/Application\ Support/${KS_TICKET_PRODUCT_ID}.update_state.plist
4. Chrome can read that marker to determine the state of the cleanup, suggesting promotion.

This course of action requires no Chrome-specific change from Keystone (we do not have other applications that get into this situation).

 Issue 797959  has been merged into this issue.
FYI: The Canary file size from the reporter of the merged  issue 797959  is about 30GB :-(
My Canary contains two framework version folders (65.0.3295.0 and 65.0.3309.0), and the older one is not going away when I launch the app. Canary is 367MB on my machine.

Comment 57 by mark@chromium.org, Jan 2 2018

No deletion happens at app launch (with a system ticket, this wouldn’t even be possible unless you ran Chrome as root—not recommended). All deletion happens at update time, right after the update has been applied.

The logic in the updater is:

 - Preserve the new version being updated “to” (of course)
 - Preserve the version being updated “from”
 - Preserve any version that appears to be in use (based on “lsof” and “ps”)
 - Delete everything else.

Once the updater has applied at least one update, there will always be at least two directories in Versions.

Sign in to add a comment