New issue
Advanced search Search tips
Starred by 20 users

Issue metadata

Status: Verified
Owner:
Closed: Apr 2015
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 2
Type: Bug



Sign in to add a comment

org.chromium.chromoting.log.0 file balloons to take up entire hard disk

Reported by alyssanf...@gmail.com, Jan 26 2015 Back to list

Issue description

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 Safari/537.36

Steps to reproduce the problem:
1. Install Chrome Remote Desktop
2. Use CRD
3. Notice drastically reduced free space

What is the expected behavior?
log file should not grow to more than 1MB.

What went wrong?
org.chromium.chromoting.log.0 began to grow until it was taking up all remaining free space on the HD.  350GB the first time.  deleting the file and uninstalling/re

WebStore page: 

Did this work before? N/A 

Chrome version: 40.0.2214.91  Channel: stable
OS Version: OS X 10.10.1
Flash Version: Shockwave Flash 16.0 r0

deleting the file and uninstalling/reinstalling the host app worked temporarily but it has happened three times now.

The org.chromium.chromoting.conf file DOES exist in /etc/newsyslog.d
 
Labels: -Cr-Platform-Apps Cr-Services-Chromoting
Can you confirm the contents of the log file (I imagine there will be a lot of repeated entries--it's sufficient to attach a portion of the log file just before those repeated entries, and let us know what the repeated entries are). Also, if you restart the host process, does that solve the problem? You can restart with:

  killall -HUP remoting_me2me_host

My guess is that, although the log file is being rotated when it exceeds a certain size, the process still has an open handle to it and so it continues to write to the old file.

Comment 3 by cohen...@gmail.com, Jan 28 2015

As luck would have it this has not occurred again since I posted. As soon as it happens again I will attach part of the file.

Comment 4 by a...@chromium.org, Jan 28 2015

Labels: Needs-Feedback
One other thing that would be useful if this happens again: does the .log.0 file continue to grow, or is its size fixed (and perhaps the corresponding .log file is growing)?
I'm still waiting for it to happen again but I can tell you that it's not a fixed size.  The first time it happened I didn't realize it until it had grown to use up the entirety of the ~350GB of free space on my main HD and I started getting HD space warnings.

As far as I can tell, it's only the log.0 file that balloons uncontrollably.  the .log file has never been abnormally large.

Comment 7 by cohen...@gmail.com, Jan 30 2015

Sorry, I just realized I've posted here from three different email addresses.  I'll stick with this one to avoid confusion from here out.

Comment 8 by cohen...@gmail.com, Feb 1 2015

It just happened again.  I have a 31GB org.chromium.chromoting.log.0 file.  Do you have any suggestions on how to open or truncate it?  I can't open it in text wrangler (says I have insufficient memory).

Also, although it's not supposed to be running, the remoting_me2me_host process is running at almost 200% CPU (I have 8 cores).

The org.chromium.chromoting.log file simply says this: "Feb  1 12:30:00 Aarons-Mac-Pro newsyslog[2348]: logfile turned over due to size>1000K"

I tried moving the org.chromium.chromoting.log.0 file to my desktop to see if it would be recreated in its normal directory but that has not happened.

I also noticed that a bzip2 process is running, using about 100% CPU.  Not sure what that is doing but I sampled it and attached it here just in case it's related.  I force quit the remoting_me2me_host before I thought to sample it.
Sample of bzip2.txt
31.4 KB View Download

Comment 9 by cohen...@gmail.com, Feb 1 2015

Ok, I just managed to trigger it again.  I started a session from my iPhone and then closed it.  The second time I did this the removing_me2me_host process continued to run at about 100% CPU (the first time it closed properly.  The process was still running but at very low CPU usage).

This time I was able to catch the file very early.  I have a 90MB org.chromium.chromoting.log file to attach.  The log.0 file has not been created yet.  I copied the org.chromium.chromoting.log file to my desktop a couple minutes ago and since that time it has already grown to about 1.5GB. I don't know at what point the log.0 file will be created but it has not been yet.

Ok, the max attachment size is 10MB so I truncated the file.  The last line just repeats forever: "[warn] kevent: Bad file descriptor"
org.chromium.chromoting.log
5.5 KB View Download
[0201/143006:ERROR:udp_socket_libevent.cc(113)] close: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor

This seems like a bug in udp_socket_libevent.cc


Labels: M-42
Owner: sergeyu@chromium.org
Status: Assigned (was: NULL)
We would be better off crashing in udp_socket_libevent.cc in this situation so we could get a useful back-trace and not fill up peoples' hard-drives.
Just happened again.  Found it at 430GB and the drive had 16.5 MB remaining.  This could really cause issues if not checked.
Project Member

Comment 13 by bugdroid1@chromium.org, Feb 17 2015

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fa5f41608e6deac84fe35519d402e3f3e6e82d36

commit fa5f41608e6deac84fe35519d402e3f3e6e82d36
Author: sergeyu <sergeyu@chromium.org>
Date: Tue Feb 17 22:15:08 2015

PCHECK when closing UDP sockets

With this change failure to close UDP socket will crash the process.
This will help to debug  bug 452121 , but is also necessary to avoid
leaking file descriptors.

BUG= 452121 

Review URL: https://codereview.chromium.org/932913002

Cr-Commit-Position: refs/heads/master@{#316668}

[modify] http://crrev.com/fa5f41608e6deac84fe35519d402e3f3e6e82d36/net/udp/udp_socket_libevent.cc

Status: Fixed (was: NULL)
The original problem (log ballooning) should be fixed now. Will reopen if we start getting crash reports from the new CHECK().

Comment 15 by Deleted ...@, Feb 21 2015

Happened to me today. Found a 109 GB log file. I love the extension, so I'd hate it if I weren't able to keep using it.

Comment 16 by cohen...@gmail.com, Feb 22 2015

This just occured for me again as well.  Same exact behavior as before.

Comment 17 by tow...@gmail.com, Feb 27 2015

Same problem, only mine is 950gb, I don't need something taking up all my hard disk space and taking a chance of crashing my hard drive.  This needs to be fixed ASAP. 

Comment 18 by cohen...@gmail.com, Feb 27 2015

Incidentally, I've noticed that about half of the time I disconnect a session, the remoting_me2me_host process continues to run at about 100% CPU.  This does not always translate to a ballooning log file but that might just be because I now ALWAYS check for that running process and kill it if necessary.  

Comment 19 by Deleted ...@, Feb 27 2015

Hey, Sergey--  How can I apply your patch?  I have the same problem; my log grew to 650GB+; like others mostly 
 
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
...

Comment 20 by tow...@gmail.com, Feb 27 2015

Noticed, that the fix was released on the 17th, I haven't updated in the past two weeks, I'll do that when I get home and see if it fixes the problem.  Is the only way to update is to uninstall and reinstall? 

Comment 21 by sergeyu@google.com, Feb 27 2015

We are going to release a build with a fix soon, probably in 2-3 weeks. The host updates automatically so you won't need to do anything to get the fix once it's released.
I got home from a 3 week vacation where I left my Mac desktop open (so I could access from laptop, if needed).  This bug created a log file as described by others, that was 2.5TB big.   Yes, TB. Problem has definitely not been fixed.
Cc: sko...@chromium.org
 Issue 431405  has been merged into this issue.
It happened again today; any estimate on when the fix will be released?
Again today, crippling my machine and causing data loss when files could not be saved.

When is the fix going to be released?
We are planning to roll out the new version this week. Before it's autoupdated you can get the new version here: http://dl.google.com/dl/chrome-remote-desktop/chromeremotedesktophost-42.0.2311.36.dmg
It's back, same symptoms as before.

My client version is 42.0.2311.37. I don't know how to check the version of the host.

Comment 28 by ren...@gmail.com, Apr 11 2015

Just had this issue crop up on my iMac. It's currently 640.94 GB. I didn't notice until my 1.5TB HDD started complaining that it was full.
It happened again, same symptoms.

Just a reminder that these symptoms include the host machine becoming unstable and losing data when all disk writes start failing because the disk is full. CRD is ticking time bomb while this issue remains unfixed.
Other reports of the same problem are being redirected here, but this issue is marked as "Fixed".

Should I be opening a new issue, or continuing to update this one?
If you're still seeing the problem, can you confirm the host version number? The easiest way to get it is to run the Chrome app on the host computer (no need to connect to anything) and look in the Javascript console (visit chrome://extensions and click the main.html link for Chrome Remote Desktop).
It happened again, same symptoms.

I don't see a "main.html link for Chrome Remote Desktop" at chrome://extensions

What I do see is attached as a screenshot.
Screen Shot 2015-04-20 at 9.16.57 PM.png
150 KB View Download
42.0.2311.37

Filled hard drive up in a matter of 8 hours. (400GB log file).
Mac OS X 10.9
durandal42: Sorry, I neglected to mention that you need to check the "Developer mode" option at the top of the page.
I've attached a screenshot of what I see when "Developer mode" is checked (apologies for the poor quality; it's a screenshot of CRD's view of the host machine :P). There's still no "main.html" link.

Interestingly, on the *client* machine, I do see a "main.html" link, right next to "background.html".
Screen Shot 2015-04-21 at 1.17.11 PM.png
203 KB View Download
main.html should be there if the app is running. Can you make sure that the app is running and check again? You may need to refresh chrome://extensions.
I was connected to the host machine via CRD in order to take that screenshot; how could it not be running?

On the off chance that you need to be running the client app on the host machine in order to check the host version...?

Host version: 42.0.2311.36

That is an ... unintuitive way to check a version number. :-/
Whoops, garbled the screenshot.
Screen Shot 2015-04-21 at 2.11.21 PM.png
61.0 KB View Download
Status: Untriaged (was: NULL)
Judging by comments, and after confirming with durandal42 that he's running 42.0.2311.36, it doesn't look like the fix causes a crash after all (or something more subtle remains broken after the process restarts).

The next release changes how we do logging, which should make the symptom of this problem go away. However, the underlying problem will still be there, and will likely have other, unforeseen consequences.

Reopening for discussion at triage.
Status: Assigned (was: NULL)
If anyone is seeing this on version 42.0.2311.36, can you check the log for the sequence described in #10:

[0201/143006:ERROR:udp_socket_libevent.cc(113)] close: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor
[warn] kevent: Bad file descriptor

From the crash reports we get in chrome after crrev.com/323624 was landed it appears that the problem is that something in System.framework may close UDP file descriptors it doesn't own. So it's possible to get into situation when kevent start logging the error, before close() call crashes, but presumably that should still end when session disconnected (and the host crashes after the failed close()). I suspect that the problem in System.framework may be related to bug 472291, which was fixed recently. I'm going to wait and see if we get new crashes with that fix.
Labels: -Needs-Feedback
I have the same problem on 3 macs with 10.10.3 installed.  It occurs every three weeks or so and fills the disk with Log files of 80GB to 130GB is size.  Usually only occurs after 5 or 6 remote sessions using the CRD plugin remotely.   It would be very helpful to have this fixed as it crashes my OS X Server every so often due to lack of free space.   Thanks, Brian

Status: Fixed (was: NULL)
This problem should be fixed in 43 host release - the root cause seems to be the same as in bug 472291 and the fix has been merged to M43 branch.
bug 472291?

Comment 46 by Deleted ...@, Apr 30 2015

I have run into this issue today as well.  400+ GB log file.  Am I understanding this is from using the Chrome Remote Desktop?  I won't be able to get to that computer for a few hours so cannot tell you what host version I am using or anything right now.
Same symptoms, tried all recommendations on the post. Is the fix out yet?

Comment 48 by Deleted ...@, May 13 2015

Just had it happen to me today - 671GB log file. Chrome 43.0.2357.52 beta (64-bit)

Comment 49 by Deleted ...@, May 22 2015

Unfortunately it is still happening. I just deleted an 800GB one the other day and then had another 350GB one came back this morning.
This issue just happened for me also.
For those seeing this problem, you can try installing http://dl.google.com/dl/chrome-remote-desktop/chromeremotedesktophost-43.0.2357.39.dmg.
My org.chromium.chromoting.log.0 just ate 459.09GB (all available space) after deleting and rebooting. Host computer is OSX.
Your install link seems to have fixed it. Thanks.

Comment 54 by dhw@chromium.org, Jun 4 2015

Labels: -M-42 M-43
From Comment 44, fix merged to M43; updating milestone.

Comment 55 Deleted

This just happened to me today. 49.9GB file with the same symptoms in this issue. I'm going to try the suggested link to see if it helps. (I made sure this is not 431405).

Comment 57 by Deleted ...@, Jun 12 2015

I just experienced this today. The org.chromium file filled up 74 GB on a 250GB drive, and then the drive was 100% full!  I just deleted the file and got my space back.  I am also downloading the fix from the link.  I went through hours of pain to figure it out.
CRD has updated to a newer version that should not be causing this problem. Is anyone still seeing this?
Status: Verified (was: NULL)

Sign in to add a comment