9202.70.0 Consuming more RAM space |
||||||
Issue description
Device: Guado-Moblab
Version: R57-9202.70.0 test image
Device rebooting frequently as Gaudo device has just 4GB RAM and it is not sufficient for this build. older builds were working fine with 4GB RAM
Increased to 8GB RAM, but apache2 consuming 5GBs of RAM
Apache2 is filling RAM, I upgraded 4GB to 8GB RAM, but still it is filling almost RAM.
Might be issue with Apache2. Killing apache2 processes freeing 5GB of RAM
Just after reboot:
Mem: 8099748 1001100 7098648 30396 97628 350780
Services are coming up:
Mem: 8099748 4233648 3866100 30856 132620 411120
AFE up:
Mem: 8099748 4384768 3714980 30900 132668 438048
Test launch:
Mem: 8099748 5660140 2439608 35984 134192 1152360
Mem: 8099748 5990456 2109292 35724 134400 1173552
Mem: 8099748 6113848 1985900 36008 134616 1182300
Mem: 8099748 6341116 1758632 36048 134892 1184068
Mem: 8099748 6581088 1518660 36008 135184 1175620
Memory consuming processes (There are 50 Apache2 instances):
localhost ~ # ps aux --sort=-%mem | awk 'NR<=10{print $0}'
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
apache 4154 1.3 2.3 450332 191316 ? Sl 16:41 0:07 apache2
apache 4153 1.2 2.3 450332 191304 ? Sl 16:41 0:06 apache2
apache 4172 0.9 1.8 355100 147492 ? Sl 16:41 0:04 apache2
apache 4147 1.1 1.8 354832 147260 ? Sl 16:41 0:05 apache2
apache 4165 1.1 1.7 352028 138856 ? Sl 16:41 0:05 apache2
apache 4152 1.1 1.7 352028 138800 ? Sl 16:41 0:06 apache2
apache 4178 1.0 1.7 352540 138460 ? Sl 16:41 0:05 apache2
apache 4148 0.9 1.7 351760 137724 ? Sl 16:41 0:05 apache2
apache 4180 0.9 1.7 351772 137720 ? Sl 16:41 0:05 apache2
Kill Apache2:
killall apache2:
Mem: 8099748 2373240 5726508 36012 135416 1176052
,
May 2 2017
,
May 2 2017
,
May 2 2017
Minor update (roopesh cannot update the bug due to access issues): The device is stable again since the RAM upgrade
,
May 2 2017
I am fighting issues on 59 ( like the config wizard not working ) but I have also noticed issues with memory. We upgrading all the guado_moblab devices in the lab to 16GB since that is what we require partners to have now. My guess is this CL might have caused the memory usage increase, but I will need to do more testing to confirm. https://chromium-review.googlesource.com/#/c/477171/
,
May 2 2017
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/ee9b8f2e54a12e718bfc514ec241ecd7d55af1f0 commit ee9b8f2e54a12e718bfc514ec241ecd7d55af1f0 Author: Keith Haddow <haddowk@chromium.org> Date: Wed May 10 03:36:43 2017 [moblab] Set an Apache define to reflect the apache is on moblab. In other configuraitons we need to be able to change what the main lab setting is. For cases where apache configuraition does not allow settings to be overridden ( line WSGI ) then use this define to add configuration logic so the correct settings are used for the correct platform. TEST=Manual testing on moblab BUG= chromium:716181 Change-Id: I542fd5df1119df413db3613404239de3d445284f Reviewed-on: https://chromium-review.googlesource.com/499087 Commit-Ready: Keith Haddow <haddowk@chromium.org> Tested-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Michael Tang <ntang@chromium.org> [modify] https://crrev.com/ee9b8f2e54a12e718bfc514ec241ecd7d55af1f0/project-moblab/chromeos-base/chromeos-bsp-moblab/files/moblab-apache-settings.conf [rename] https://crrev.com/ee9b8f2e54a12e718bfc514ec241ecd7d55af1f0/project-moblab/chromeos-base/chromeos-bsp-moblab/chromeos-bsp-moblab-0.0.5-r55.ebuild
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/0f4dfded8d342339d111608694dd33283915f8aa commit 0f4dfded8d342339d111608694dd33283915f8aa Author: Keith Haddow <haddowk@chromium.org> Date: Wed May 10 04:12:53 2017 [autotest] Exclude the WSGI server from being started on moblab. Moblab needs different process settings than the lab as the machines they run on have very different memory footprints. Apache does not allow a WSGI process to be "overridden" so exclude running the demon if on moblab, depends on another change in the moblab configuration. TEST=Manual testing on moblab. BUG= chromium:716181 Depends-on: I542fd5df1119df413db3613404239de3d445284f Change-Id: I8cc0ba08bb6c2323e1c4df5fef673362095b0dfd Reviewed-on: https://chromium-review.googlesource.com/498689 Reviewed-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Michael Tang <ntang@chromium.org> Tested-by: Keith Haddow <haddowk@chromium.org> [modify] https://crrev.com/0f4dfded8d342339d111608694dd33283915f8aa/apache/conf/django-directives
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/af7997ba5b79c3cfe32182c29233a15132fbb345 commit af7997ba5b79c3cfe32182c29233a15132fbb345 Author: Keith Haddow <haddowk@chromium.org> Date: Wed May 10 21:51:20 2017 [autotest] Exclude the WSGI server from being started on moblab. Moblab needs different process settings than the lab as the machines they run on have very different memory footprints. Apache does not allow a WSGI process to be "overridden" so exclude running the demon if on moblab, depends on another change in the moblab configuration. TEST=Manual testing on moblab. BUG= chromium:716181 Depends-on: I542fd5df1119df413db3613404239de3d445284f Change-Id: I8cc0ba08bb6c2323e1c4df5fef673362095b0dfd Reviewed-on: https://chromium-review.googlesource.com/498689 Reviewed-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Michael Tang <ntang@chromium.org> Tested-by: Keith Haddow <haddowk@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/501989 Commit-Queue: Keith Haddow <haddowk@chromium.org> [modify] https://crrev.com/af7997ba5b79c3cfe32182c29233a15132fbb345/apache/conf/django-directives
,
May 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/c9f89a5d6d8d511ccd919988eca5d1d86a27c517 commit c9f89a5d6d8d511ccd919988eca5d1d86a27c517 Author: Keith Haddow <haddowk@chromium.org> Date: Wed May 10 21:52:58 2017 [moblab] Set an Apache define to reflect the apache is on moblab. In other configuraitons we need to be able to change what the main lab setting is. For cases where apache configuraition does not allow settings to be overridden ( line WSGI ) then use this define to add configuration logic so the correct settings are used for the correct platform. TEST=Manual testing on moblab BUG= chromium:716181 Change-Id: I542fd5df1119df413db3613404239de3d445284f Reviewed-on: https://chromium-review.googlesource.com/499087 Commit-Ready: Keith Haddow <haddowk@chromium.org> Tested-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Keith Haddow <haddowk@chromium.org> Reviewed-by: Michael Tang <ntang@chromium.org> (cherry picked from commit ee9b8f2e54a12e718bfc514ec241ecd7d55af1f0) Reviewed-on: https://chromium-review.googlesource.com/501250 Commit-Queue: Keith Haddow <haddowk@chromium.org> [modify] https://crrev.com/c9f89a5d6d8d511ccd919988eca5d1d86a27c517/project-moblab/chromeos-base/chromeos-bsp-moblab/files/moblab-apache-settings.conf [rename] https://crrev.com/c9f89a5d6d8d511ccd919988eca5d1d86a27c517/project-moblab/chromeos-base/chromeos-bsp-moblab/chromeos-bsp-moblab-0.0.5-r55.ebuild
,
May 10 2017
,
May 10 2017
Thanks Keith! |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by danepollock@google.com
, May 2 2017