New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 739943 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 739933
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Buildpackages failure was seen on several chrome informational PFQ builds

Project Member Reported by x...@chromium.org, Jul 6 2017

Issue description

The failure was observed on peach_pit-tot-chrome-pfq-informational (https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/peach_pit-tot-chrome-pfq-informational/builds/6828) and cyan-tot-chrome-pfq-informational (https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/cyan-tot-chrome-pfq-informational/builds/3189). 

Selected error message:
waiting for compiler_proxy...
compiler proxy (pid=1086) status: http://127.0.0.1:8088 error: failed to connect to backend servers
Traceback (most recent call last):
  File "/home/chrome-bot/goma/goma_ctl.py", line 2455, in <module>
    sys.exit(main())
  File "/home/chrome-bot/goma/goma_ctl.py", line 2450, in main
    goma.Dispatch(sys.argv[1:])
  File "/home/chrome-bot/goma/goma_ctl.py", line 1114, in Dispatch
    self._action_mappings.get(args[0], self._DefaultAction)()
  File "/home/chrome-bot/goma/goma_ctl.py", line 801, in _RestartCompilerProxy
    self._StartCompilerProxy()
  File "/home/chrome-bot/goma/goma_ctl.py", line 621, in _StartCompilerProxy
    self._GenericStartCompilerProxy(ensure=False)
  File "/home/chrome-bot/goma/goma_ctl.py", line 618, in _GenericStartCompilerProxy
    raise Error('Failed to start compiler_proxy successfully.')
__main__.Error: Failed to start compiler_proxy successfully.
ERROR   : Thu Jul  6 16:01:18 PDT 2017
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 10: ./build_packages '--board=peach_pit' '--accept_licenses=@CHROMEOS' '--withdebugsymbols' '--skip_chroot_upgrade' '--noworkon' '--run_goma' '--withevents' '--eventfile=/mnt/host/source/buildbot_archive/peach_pit-tot-chrome-pfq-informational/R61-9719.0.0-b6828/build-events.json' 'virtual/target-os' 'virtual/target-os-dev' 'virtual/target-os-test' 'virtual/target-os-factory' 'virtual/target-os-factory-shim' 'chromeos-base/autotest-all'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  build_packages:307:main(), called: die_err_trap  
ERROR   : 
ERROR   : Command failed:
ERROR   :   Command '( if [[ "${FLAGS_run_goma}" -eq "${FLAGS_TRUE}" ]]; then
ERROR   :     info "Starting goma compiler_proxy."; goma_ctl="${GOMA_DIR:-${HOME}/goma}/goma_ctl.py"; "${goma_ctl}" restart; trap "'${goma_ctl}' stop" EXIT;
ERROR   : fi; set -o pipefail; sudo -E "${EMERGE_CMD[@]}" "${EMERGE_FLAGS[@]}" "${PACKAGES[@]}" | tee "${tmpfile}" )' exited with nonzero code: 1

@@@STEP_FAILURE@@@
16:01:18: ERROR: ./build_packages failed (code=1)

infra issue? 
 

Comment 1 by x...@chromium.org, Jul 6 2017

Cc: -xiy...@chromium.org xixuan@chromium.org

Comment 2 by x...@chromium.org, Jul 6 2017

Cc: hidehiko@chromium.org
hidehiko@, are you aware of any change that might cause this?

Comment 3 by uekawa@google.com, Jul 7 2017

I think I saw there's a GOMA outage, and started recovering 

Mergedinto: 739933
Status: Duplicate (was: Available)

Comment 5 by uekawa@google.com, Jul 7 2017

One thing is that I thought GOMA outage shouldn't fail the build ?

It depends, as it may hit timeout again like  crbug.com/687643 , which is just delaying problem finding, or the problem is just silently ignored which often makes situation worse.

Aviv, it there any mechanism we can be notified some error (like goma outage) without build failure?

Sign in to add a comment