New issue
Advanced search Search tips

Issue 828484 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: ----



Sign in to add a comment

eve-arcnext-chrome-pfq:56 failed

Project Member Reported by sheriff-...@appspot.gserviceaccount.com, Apr 3 2018

Issue description

Filed by sheriff-o-matic@appspot.gserviceaccount.com on behalf of weidongg@google.com

eve-arcnext-chrome-pfq:56 failed

Builders failed on: 
- eve-arcnext-chrome-pfq: 
  http://cros-goldeneye/chromeos/healthmonitoring/buildDetails?buildbucketId=8950279389616111440

There was a CL to mark the build as important:
https://chromium-review.googlesource.com/c/chromiumos/chromite/+/990808


 
Cc: weidongg@chromium.org
Here is the stack trace:

chromeos-chrome-67.0.3387.0_rc-r1: ninja: build stopped: subcommand failed.
chromeos-chrome-67.0.3387.0_rc-r1:  * ERROR: chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos failed (compile phase):
chromeos-chrome-67.0.3387.0_rc-r1:  *   (no error message)
chromeos-chrome-67.0.3387.0_rc-r1:  * 
chromeos-chrome-67.0.3387.0_rc-r1:  * Call stack:
chromeos-chrome-67.0.3387.0_rc-r1:  *     ebuild.sh, line  133:  Called src_compile
chromeos-chrome-67.0.3387.0_rc-r1:  *   environment, line 4295:  Called chrome_make 'chrome_sandbox' 'libosmesa.so' 'chrome' 'jpeg_decode_accelerator_unittest' 'media_unittests' 'sandbox_linux_unittests' 'video_decode_accelerator_unittest' 'video_encode_accelerator_unittest' 'wayland_client_perftests' 'ppapi/examples/video_decode' 'bitmaptools' 'clear_system_cache' 'minidump_stackwalk' 'chromedriver' 'nacl_helper_bootstrap' 'nacl_helper'
chromeos-chrome-67.0.3387.0_rc-r1:  *   environment, line 1262:  Called die
chromeos-chrome-67.0.3387.0_rc-r1:  * The specific snippet of code:
chromeos-chrome-67.0.3387.0_rc-r1:  *       [[ "${ret}" -eq 0 ]] || die
chromeos-chrome-67.0.3387.0_rc-r1:  * 
chromeos-chrome-67.0.3387.0_rc-r1:  * If you need support, post the output of `emerge --info '=chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos'`,
chromeos-chrome-67.0.3387.0_rc-r1:  * the complete build log and the output of `emerge -pqv '=chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos'`.
chromeos-chrome-67.0.3387.0_rc-r1:  * The complete build log is located at '/build/eve-arcnext/tmp/portage/logs/chromeos-base:chromeos-chrome-67.0.3387.0_rc-r1:20180403-095216.log'.
chromeos-chrome-67.0.3387.0_rc-r1:  * For convenience, a symlink to the build log is located at '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/temp/build.log'.
chromeos-chrome-67.0.3387.0_rc-r1:  * The ebuild environment file is located at '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/temp/environment'.
chromeos-chrome-67.0.3387.0_rc-r1:  * Working directory: '/home/chrome-bot/chrome_root/src'
chromeos-chrome-67.0.3387.0_rc-r1:  * S: '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/work/chromeos-chrome-67.0.3387.0_rc'
chromeos-chrome-67.0.3387.0_rc-r1: >>> Failed to emerge chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1 for /build/eve-arcnext/, Log file:
chromeos-chrome-67.0.3387.0_rc-r1: >>>  '/build/eve-arcnext/tmp/portage/logs/chromeos-base:chromeos-chrome-67.0.3387.0_rc-r1:20180403-095216.log'
chromeos-chrome-67.0.3387.0_rc-r1: 
chromeos-chrome-67.0.3387.0_rc-r1:  * Messages for package chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1 merged to /build/eve-arcnext/:
chromeos-chrome-67.0.3387.0_rc-r1: 
chromeos-chrome-67.0.3387.0_rc-r1:  * CHROME_ORIGIN VALUE is LOCAL_SOURCE
chromeos-chrome-67.0.3387.0_rc-r1:  * /home/chrome-bot/chrome_root should be set here properly
chromeos-chrome-67.0.3387.0_rc-r1:  * ERROR: chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos failed (compile phase):
chromeos-chrome-67.0.3387.0_rc-r1:  *   (no error message)
chromeos-chrome-67.0.3387.0_rc-r1:  * 
chromeos-chrome-67.0.3387.0_rc-r1:  * Call stack:
chromeos-chrome-67.0.3387.0_rc-r1:  *     ebuild.sh, line  133:  Called src_compile
chromeos-chrome-67.0.3387.0_rc-r1:  *   environment, line 4295:  Called chrome_make 'chrome_sandbox' 'libosmesa.so' 'chrome' 'jpeg_decode_accelerator_unittest' 'media_unittests' 'sandbox_linux_unittests' 'video_decode_accelerator_unittest' 'video_encode_accelerator_unittest' 'wayland_client_perftests' 'ppapi/examples/video_decode' 'bitmaptools' 'clear_system_cache' 'minidump_stackwalk' 'chromedriver' 'nacl_helper_bootstrap' 'nacl_helper'
chromeos-chrome-67.0.3387.0_rc-r1:  *   environment, line 1262:  Called die
chromeos-chrome-67.0.3387.0_rc-r1:  * The specific snippet of code:
chromeos-chrome-67.0.3387.0_rc-r1:  *       [[ "${ret}" -eq 0 ]] || die
chromeos-chrome-67.0.3387.0_rc-r1:  * 
chromeos-chrome-67.0.3387.0_rc-r1:  * If you need support, post the output of `emerge --info '=chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos'`,
chromeos-chrome-67.0.3387.0_rc-r1:  * the complete build log and the output of `emerge -pqv '=chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1::chromiumos'`.
chromeos-chrome-67.0.3387.0_rc-r1:  * The complete build log is located at '/build/eve-arcnext/tmp/portage/logs/chromeos-base:chromeos-chrome-67.0.3387.0_rc-r1:20180403-095216.log'.
chromeos-chrome-67.0.3387.0_rc-r1:  * For convenience, a symlink to the build log is located at '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/temp/build.log'.
chromeos-chrome-67.0.3387.0_rc-r1:  * The ebuild environment file is located at '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/temp/environment'.
chromeos-chrome-67.0.3387.0_rc-r1:  * Working directory: '/home/chrome-bot/chrome_root/src'
chromeos-chrome-67.0.3387.0_rc-r1:  * S: '/build/eve-arcnext/tmp/portage/chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1/work/chromeos-chrome-67.0.3387.0_rc'
=== Complete: job chromeos-chrome-67.0.3387.0_rc-r1 (1m1.5s) ===
Failed chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1 (in 1m1.5s). Your build has failed.
Pending 21/799, [Time 02:53:18 | Elapsed 31m2.1s | Load 3.32 19.36 32.94]
Packages failed:
	chromeos-base/chromeos-chrome-67.0.3387.0_rc-r1
using /tmp/goma_tmp_dir.owbvh_ as tmpdir
Killing compiler proxy.
compiler proxy status: http://127.0.0.1:8088 quit!
ERROR   : Tue Apr  3 02:53:19 PDT 2018
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 10: ./build_packages '--board=eve-arcnext' '--accept_licenses=@CHROMEOS' '--withdebugsymbols' '--skip_chroot_upgrade' '--run_goma' '--withevents' '--eventfile=/mnt/host/source/buildbot_archive/eve-arcnext-chrome-pfq/R67-10541.0.0-rc3/build-events.json' 'virtual/target-os' 'virtual/target-os-dev' 'virtual/target-os-test' 'virtual/target-os-factory' 'virtual/target-os-factory-shim' 'chromeos-base/autotest-all'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  build_packages:329:main(), called: die_err_trap  
ERROR   : 
ERROR   : Command failed:
ERROR   :   Command '( if [[ "${FLAGS_run_goma}" -eq "${FLAGS_TRUE}" ]]; then
ERROR   :     info "Starting goma compiler_proxy."; goma_ctl="${GOMA_DIR:-${HOME}/goma}/goma_ctl.py"; "${goma_ctl}" restart; trap "'${goma_ctl}' stop" EXIT;
ERROR   : fi; set -o pipefail; sudo -E "${EMERGE_CMD[@]}" "${EMERGE_FLAGS[@]}" "${PACKAGES[@]}" | tee "${tmpfile}" )' exited with nonzero code: 1
02:53:20: ERROR: 
return code: 1; command: /b/c/cbuild/repository/chromite/bin/cros_sdk --goma_dir /b/c/goma_client --goma_client_json /creds/service_accounts/service-account-goma-client.json '--chrome_root=/b/c/cbuild/repository/.cache/distfiles/target-master/chrome-src-internal' 'GOMA_SERVICE_ACCOUNT_JSON_FILE=/creds/service_accounts/service-account-goma-client.json' 'USE=chrome_internal' 'USE_GOMA=true' 'FEATURES=separatedebug' 'CHROME_ORIGIN=LOCAL_SOURCE' 'GOMA_BACKEND_SOFT_STICKINESS=false' 'GOMA_CACHE_DIR=/home/chrome-bot/goma/goma_cache/BuildPackages' 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmp2k6y8c' 'GOMA_MAX_COMPILER_DISABLED_TASKS=30' 'GOMA_DIR=/home/chrome-bot/goma' 'GLOG_log_dir=/tmp/goma_tmp_dir.owbvh_/log_dir' 'GOMA_TMP_DIR=/tmp/goma_tmp_dir.owbvh_' 'GOMA_DEPS_CACHE_FILE=goma.deps' -- ./build_packages '--board=eve-arcnext' '--accept_licenses=@CHROMEOS' --withdebugsymbols --skip_chroot_upgrade --run_goma --withevents '--eventfile=/mnt/host/source/buildbot_archive/eve-arcnext-chrome-pfq/R67-10541.0.0-rc3/build-events.json' virtual/target-os virtual/target-os-dev virtual/target-os-test virtual/target-os-factory virtual/target-os-factory-shim chromeos-base/autotest-all
cmd=['/b/c/cbuild/repository/chromite/bin/cros_sdk', '--goma_dir', '/b/c/goma_client', '--goma_client_json', '/creds/service_accounts/service-account-goma-client.json', '--chrome_root=/b/c/cbuild/repository/.cache/distfiles/target-master/chrome-src-internal', 'GOMA_SERVICE_ACCOUNT_JSON_FILE=/creds/service_accounts/service-account-goma-client.json', u'USE=chrome_internal', 'USE_GOMA=true', 'FEATURES=separatedebug', 'CHROME_ORIGIN=LOCAL_SOURCE', 'GOMA_BACKEND_SOFT_STICKINESS=false', 'GOMA_CACHE_DIR=/home/chrome-bot/goma/goma_cache/BuildPackages', 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmp2k6y8c', 'GOMA_MAX_COMPILER_DISABLED_TASKS=30', 'GOMA_DIR=/home/chrome-bot/goma', 'GLOG_log_dir=/tmp/goma_tmp_dir.owbvh_/log_dir', 'GOMA_TMP_DIR=/tmp/goma_tmp_dir.owbvh_', 'GOMA_DEPS_CACHE_FILE=goma.deps', '--', './build_packages', u'--board=eve-arcnext', '--accept_licenses=@CHROMEOS', '--withdebugsymbols', '--skip_chroot_upgrade', '--run_goma', '--withevents', u'--eventfile=/mnt/host/source/buildbot_archive/eve-arcnext-chrome-pfq/R67-10541.0.0-rc3/build-events.json', 'virtual/target-os', 'virtual/target-os-dev', 'virtual/target-os-test', 'virtual/target-os-factory', 'virtual/target-os-factory-shim', 'chromeos-base/autotest-all'], cwd=/b/c/cbuild/repository, extra env={'GOMA_SERVICE_ACCOUNT_JSON_FILE': '/creds/service_accounts/service-account-goma-client.json', 'USE': u'chrome_internal', 'USE_GOMA': 'true', 'FEATURES': 'separatedebug', 'CHROME_ORIGIN': 'LOCAL_SOURCE', 'GOMA_BACKEND_SOFT_STICKINESS': 'false', 'GOMA_CACHE_DIR': '/home/chrome-bot/goma/goma_cache/BuildPackages', 'PARALLEL_EMERGE_STATUS_FILE': '/tmp/tmp2k6y8c', 'GOMA_MAX_COMPILER_DISABLED_TASKS': '30', 'GOMA_DIR': '/home/chrome-bot/goma', 'GLOG_log_dir': '/tmp/goma_tmp_dir.owbvh_/log_dir', 'GOMA_TMP_DIR': '/tmp/goma_tmp_dir.owbvh_', 'GOMA_DEPS_CACHE_FILE': 'goma.deps'}
02:53:20: ERROR: Packages failed in ./build_packages: chromeos-base/chromeos-chrome
02:53:20: INFO: Translating result Packages failed in ./build_packages: chromeos-base/chromeos-chrome to fail.
02:53:20: INFO: Running cidb query on pid 7026, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x7f33d3dfff90>
02:53:20: INFO: Running cidb query on pid 7026, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x7f33d3dff890>
************************************************************
** Finished Stage BuildPackages - Tue, 03 Apr 2018 02:53:20 -0700 (PDT)
************************************************************
02:53:21: INFO: Running cidb query on pid 7026, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x7f33d3c84b10>

Comment 3 by ihf@chromium.org, Apr 3 2018

Cc: xiaoh...@chromium.org
Components: Infra>Client>ChromeOS>Build
Labels: -Pri-2 M-67 OS-Chrome Pri-1
Looks like goma isn't working right?

Marking a build important doesn't change anything about the build building or not, it just influences later stages of the PFQ.
Thanks for investigating. I tried to repro locally by building Chrome OS packages, and I specified eve to the build, is that correct? Or should I specify eve-arcnext?

Comment 5 by ihf@chromium.org, Apr 3 2018

It should be BOARD=eve-arcnext. Also repo sync to tot first + cros_workon chromeos-chrome to make it build Chrome from source.
Saw this is a failing buildpackages log like https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Feve-arcnext-chrome-pfq%2F54%2F%2B%2Frecipes%2Fsteps%2FBuildPackages%2F0%2Fstdout

chromeos-chrome-67.0.3387.0_rc-r1: FAILED: obj/chromeos/services/assistant/lib/service.o 
<snip>
chromeos-chrome-67.0.3387.0_rc-r1: ../../../../../../../home/chrome-bot/chrome_root/src/chromeos/services/assistant/assistant_manager_service_impl.h:51:71: error: only virtual member functions can be marked 'override'
chromeos-chrome-67.0.3387.0_rc-r1:   void OnShowSuggestions(const std::vector<std::string>& suggestions) override;
chromeos-chrome-67.0.3387.0_rc-r1:                                                                       ^~~~~~~~
chromeos-chrome-67.0.3387.0_rc-r1: 1 warning and 1 error generated.

I saw OnShowSuggestions was added on https://chromium-review.googlesource.com/c/chromium/src/+/986667
Owner: xiaoh...@chromium.org
Status: Started (was: Available)
There was a CL submitted yesterday that caused the breakage. 
https://chromium-review.googlesource.com/c/chromium/src/+/986667

It broke because it depends on an internal repo change, but the internal repo wasn't upreved.  Now the uprev is in and next build should be green.
https://chrome-internal-review.googlesource.com/c/chrome/src-internal/+/601167
Status: Fixed (was: Started)
https://uberchromegw.corp.google.com/i/chromeos/builders/eve-arcnext-chrome-pfq/builds/63

It's green for the last few builds.

Sign in to add a comment