New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 596545 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Jun 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

"Android Clang Builder (dbg)" builder keeps failing builds because of out of memory

Project Member Reported by jam@chromium.org, Mar 21 2016

Issue description

Android Clang Builder (dbg)

Looking at https://build.chromium.org/p/chromium.linux/builders/Android%20Clang%20Builder%20%28dbg%29?numbuilds=200, the last 200 builds have at least 6 compile failures because of memory exhaustion. Output contains strings like


CalledProcessError: Command failed: ( cd /mnt/data/b/build/slave/Android_Clang_Builder__dbg_/build/src/media; ant -verbose -DHAS_CODE=true -DANDROID_SDK_ROOT=/mnt/data/b/build/slave/Android_Clang_Builder__dbg_/build/src/third_party/android_tools/sdk/ -DANDROID_SDK_TOOLS=/mnt/data/b/build/slave/Android_Clang_Builder__dbg_/build/src/third_party/android_tools/sdk//build-tools/23.0.1 -DRESOURCE_PACKAGED_APK_NAME=media_unittests-resources.ap_ -DNATIVE_LIBS_DIR=../out/Debug/media_unittests_apk/libs -DAPK_NAME=media_unittests -DCONFIGURATION_NAME=Debug -DOUT_DIR=../out/Debug/media_unittests_apk -DUNSIGNED_APK_PATH=../out/Debug/media_unittests_apk/media_unittests-unsigned.apk -DEMMA_INSTRUMENT=0 -DEMMA_DEVICE_JAR=/mnt/data/b/build/slave/Android_Clang_Builder__dbg_/build/src/third_party/android_tools/sdk//tools/lib/emma_device.jar -Dbasedir=. -buildfile ../build/android/ant/apk-package.xml -DDEX_FILE_PATH=../out/Debug/media_unittests_apk/classes.dex )
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000705200000, 175112192, 0) failed; error='Cannot allocate memory' (errno=12)
#


or

/mnt/data/b/build/slave/Android_Clang_Builder__dbg_/build/src/third_party/android_tools/ndk//toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-strip:../out/Debug/lib/libwebcore_shared.cr.so: Memory exhausted


This bot has 16GB memory. That seems like a lot already.
Can the compiler flags be changed so that it uses less? Or can/should it get additional RAM?
 

Comment 1 by thakis@chromium.org, Mar 21 2016

Looks like it doesn't use symbol_level=1 (the gn spelling of symbol_level=1). The normal Android dbg builder sets fastbuild=1 (https://build.chromium.org/p/chromium.linux/builders/Android%20Builder%20%28dbg%29/builds/102093/steps/generate_build_files/logs/stdio).

I bet that helps.

Comment 2 by thakis@chromium.org, Mar 21 2016

It looks a bit like this bot was converted from gyp to gn not too long ago; probably the setting got lost there.
Owner: jbudorick@chromium.org
Status: Assigned (was: Unconfirmed)
well, to mb, but yeah, that's probably what happened.
 Issue 596610  has been merged into this issue.
Project Member

Comment 5 by bugdroid1@chromium.org, Mar 21 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/a5a755d1ec910cc8adf5ea4a05211e4f0af7e788

commit a5a755d1ec910cc8adf5ea4a05211e4f0af7e788
Author: jbudorick <jbudorick@chromium.org>
Date: Mon Mar 21 20:38:33 2016

[Android] Add minimal_symbols to remaining bots on chromium.{linux,android}.

This includes:
  chromium.linux
    Android Clang Builder (dbg)
    Android GN (dbg)

  chromium.android
    Android GN Builder (dbg)

BUG= 596545 

Review URL: https://codereview.chromium.org/1817193002

Cr-Commit-Position: refs/heads/master@{#382375}

[modify] https://crrev.com/a5a755d1ec910cc8adf5ea4a05211e4f0af7e788/tools/mb/mb_config.pyl

Comment 6 by benhenry@google.com, Apr 27 2016

Components: Infra>Labs
Labels: -Infra-Labs

Comment 7 by benhenry@google.com, Apr 27 2016

Components: Infra>Client>Android
Labels: -Infra-Android

Comment 8 by stip@chromium.org, Jun 2 2016

I only see one purple failure in the last 200 builds, and it doesn't *explicitly* look like a memory failure (although the bot died in an unclean fashion). I think we can close this?
Status: Fixed (was: Assigned)
that purple build was me restarting chromium.linux this morning.

Sign in to add a comment