New issue
Advanced search Search tips

Issue 751740 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Aug 2017
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Builds failing in SetupBoard; multiple versions of LLVM for single slot...

Project Member Reported by cmt...@chromium.org, Aug 2 2017

Issue description

All of the toolchain waterfall builders and the toolchain nightly performance tests have started failing in SetupBoard with the following message:

Starting fast-emerge.
 Building package cross-arm-none-eabi/binutils cross-x86_64-cros-linux-gnu/linux-headers cross-i686-pc-linux-gnu/gdb cross-x86_64-cros-linux-gnu/gcc cross-i686-pc-linux-gnu/gcc cross-x86_64-cros-linux-gnu/gdb cross-arm-none-eabi/gcc sys-devel/llvm cross-x86_64-cros-linux-gnu/glibc cross-i686-pc-linux-gnu/linux-headers cross-x86_64-cros-linux-gnu/go cross-i686-pc-linux-gnu/glibc cross-x86_64-cros-linux-gnu/libcxxabi cross-arm-none-eabi/newlib cross-x86_64-cros-linux-gnu/binutils cross-i686-pc-linux-gnu/binutils cross-x86_64-cros-linux-gnu/libcxx cross-arm-none-eabi/gdb on root
Calculating deps...
[ebuild     U  ] sys-devel/llvm-5.0_pre300080_p20170801 [5.0_pre300080_p20170707-r3]

!!! Multiple package instances within a single package slot have been pulled
!!! into the dependency graph, resulting in a slot conflict:

sys-devel/llvm:0

  (sys-devel/llvm-5.0_pre300080_p20170801:0/5.0::chromiumos, ebuild scheduled for merge) pulled in by
    sys-devel/llvm (Argument)

  (sys-devel/llvm-5.0_pre300080_p20170707-r3:0/5.0::chromiumos, installed) pulled in by
    ~sys-devel/llvm-5.0_pre300080_p20170707[-debug,multitarget,python,static-analyzer,abi_x86_64(-)] required by (sys-devel/clang-5.0_pre300080_p20170707:0/0::chromiumos, installed)
    ^               ^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                           


It may be possible to solve this problem by using package.mask to
prevent one of those packages from being selected. However, it is also
possible that conflicting dependencies exist such that they are
impossible to satisfy simultaneously.  If such a conflict exists in
the dependencies of two different packages, then those packages can
not be installed simultaneously. You may want to try a larger value of
the --backtrack option, such as --backtrack=30, in order to see if
that will solve this conflict automatically.

For more information, see MASKED PACKAGES section in the emerge man
page or refer to the Gentoo Handbook.

cros_setup_toolchains: Unhandled exception:
Traceback (most recent call last):
  File "/mnt/host/source/chromite/bin/cros_setup_toolchains", line 168, in <module>
    DoMain()
  File "/mnt/host/source/chromite/bin/cros_setup_toolchains", line 164, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/mnt/host/source/chromite/lib/commandline.py", line 837, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/mnt/host/source/chromite/scripts/cros_setup_toolchains.py", line 1239, in main
    root=root)
  File "/mnt/host/source/chromite/scripts/cros_setup_toolchains.py", line 686, in UpdateToolchains
    if UpdateTargets(targets, usepkg, root=root) or crossdev_targets or reconfig:
  File "/mnt/host/source/chromite/scripts/cros_setup_toolchains.py", line 516, in UpdateTargets
    cros_build_lib.RunCommand(cmd)
  File "/mnt/host/source/chromite/lib/cros_build_lib.py", line 645, in RunCommand
    raise RunCommandError(msg, cmd_result)
chromite.lib.cros_build_lib.RunCommandError: return code: 1; command: /mnt/host/source/chromite/bin/parallel_emerge --oneshot --update cross-arm-none-eabi/binutils cross-x86_64-cros-linux-gnu/linux-headers cross-i686-pc-linux-gnu/gdb cross-x86_64-cros-linux-gnu/gcc cross-i686-pc-linux-gnu/gcc cross-x86_64-cros-linux-gnu/gdb cross-arm-none-eabi/gcc sys-devel/llvm cross-x86_64-cros-linux-gnu/glibc cross-i686-pc-linux-gnu/linux-headers cross-x86_64-cros-linux-gnu/go cross-i686-pc-linux-gnu/glibc cross-x86_64-cros-linux-gnu/libcxxabi cross-arm-none-eabi/newlib cross-x86_64-cros-linux-gnu/binutils cross-i686-pc-linux-gnu/binutils cross-x86_64-cros-linux-gnu/libcxx cross-arm-none-eabi/gdb
cmd=['/mnt/host/source/chromite/bin/parallel_emerge', '--oneshot', '--update', 'cross-arm-none-eabi/binutils', 'cross-x86_64-cros-linux-gnu/linux-headers', 'cross-i686-pc-linux-gnu/gdb', 'cross-x86_64-cros-linux-gnu/gcc', 'cross-i686-pc-linux-gnu/gcc', 'cross-x86_64-cros-linux-gnu/gdb', 'cross-arm-none-eabi/gcc', 'sys-devel/llvm', 'cross-x86_64-cros-linux-gnu/glibc', 'cross-i686-pc-linux-gnu/linux-headers', 'cross-x86_64-cros-linux-gnu/go', 'cross-i686-pc-linux-gnu/glibc', 'cross-x86_64-cros-linux-gnu/libcxxabi', 'cross-arm-none-eabi/newlib', 'cross-x86_64-cros-linux-gnu/binutils', 'cross-i686-pc-linux-gnu/binutils', 'cross-x86_64-cros-linux-gnu/libcxx', 'cross-arm-none-eabi/gdb']
ERROR   : Wed Aug  2 08:20:21 PDT 2017
ERROR   :  PGID  PPID   PID     ELAPSED     TIME %CPU COMMAND
ERROR   : Arguments of 10: ./update_chroot '--nousepkg' '--toolchain_boards' 'peppy'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :  update_chroot:83:main(), called: die_err_trap  
ERROR   : 
ERROR   : Command failed:
ERROR   :   Command 'sudo -E $(type -p cros_setup_toolchains) "${TOOLCHAIN_FLAGS[@]}"' exited with nonzero code: 1
08:20:21: ERROR: 
return code: 1; command: /b/c/cbuild/repository/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpNYTKZF' 'USE=-cros-debug -tests_cheets_SELinuxTest -tests_network_VPNConnect.l2tpipsec_psk -tests_network_VPNConnect.l2tpipsec_xauth -tests_platform_SyncCrash -tests_power_DarkResumeDisplay -tests_power_DarkResumeShutdownServer -tests_security_OpenFDs -tests_security_SMMLocked chrome_internal next_gcc' 'FEATURES=separatedebug -separatedebug splitdebug' 'CHROME_ORIGIN=LOCAL_SOURCE' -- ./update_chroot --nousepkg --toolchain_boards peppy
cmd=['/b/c/cbuild/repository/chromite/bin/cros_sdk', 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpNYTKZF', u'USE=-cros-debug -tests_cheets_SELinuxTest -tests_network_VPNConnect.l2tpipsec_psk -tests_network_VPNConnect.l2tpipsec_xauth -tests_platform_SyncCrash -tests_power_DarkResumeDisplay -tests_power_DarkResumeShutdownServer -tests_security_OpenFDs -tests_security_SMMLocked chrome_internal next_gcc', 'FEATURES=separatedebug -separatedebug splitdebug', 'CHROME_ORIGIN=LOCAL_SOURCE', '--', './update_chroot', '--nousepkg', '--toolchain_boards', u'peppy'], cwd=/b/c/cbuild/repository, extra env={'PARALLEL_EMERGE_STATUS_FILE': '/tmp/tmpNYTKZF', 'USE': u'-cros-debug -tests_cheets_SELinuxTest -tests_network_VPNConnect.l2tpipsec_psk -tests_network_VPNConnect.l2tpipsec_xauth -tests_platform_SyncCrash -tests_power_DarkResumeDisplay -tests_power_DarkResumeShutdownServer -tests_security_OpenFDs -tests_security_SMMLocked chrome_internal next_gcc', 'FEATURES': 'separatedebug -separatedebug splitdebug', 'CHROME_ORIGIN': 'LOCAL_SOURCE'}

@@@STEP_FAILURE@@@
08:20:21: ERROR: ./update_chroot failed (code=1)


I suspect that any builder that uses the "--latest-toolchain" flag will fail in the same way.  This needs to be fixed ASAP!
 
Owner: manojgupta@chromium.org
Labels: -Pri-0 Pri-2
This is an issue with cross_setup_toolchains. It tries to updates llvm but not clang. 
It should be fixed now since chromiumos-sdk builder has uploaded new prebuilts with latest llvm and clang. https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8057 .

Will keep it open for verification.
CL https://chromium-review.googlesource.com/c/598498 should avoid this issue in future.
Project Member

Comment 4 by bugdroid1@chromium.org, Aug 3 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/f0a602ea6f412020fea499edd1257c9d4ae8fbbf

commit f0a602ea6f412020fea499edd1257c9d4ae8fbbf
Author: Manoj Gupta <manojgupta@google.com>
Date: Thu Aug 03 01:55:47 2017

cros_setup_toolchains: Add clang to list of MANUAL_PKGS.

clang and llvm ebuild version are tied together and need to be
updated simultaneously when --latest-toolchain is used.
When the builds are renamed to a newer version, Currently only llvm
gets updated and emerge fails because the preexisitng clang's dependency
is not satisfied.
e.g https://uberchromegw.corp.google.com/i/chromeos/builders/amd64-gcc-toolchain/builds/338.

BUG= chromium:751740 
TEST=cros_setup_toolchains updates both clang and llvm.

Change-Id: Ifa108740744af47c6f263c928954626fea477762
Reviewed-on: https://chromium-review.googlesource.com/598498
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Trybot-Ready: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/f0a602ea6f412020fea499edd1257c9d4ae8fbbf/scripts/cros_setup_toolchains.py

Status: Verified (was: Untriaged)
Builds are fine today. Closing.
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/3ed47729bc21ea48437c6848528dbc459048fcf4

commit 3ed47729bc21ea48437c6848528dbc459048fcf4
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Aug 09 18:48:07 2017

cros_setup_toolchains: regen cache when script changes

When we update MANUAL_PKGS, that doesn't make it out to the cache file
which leads us to updating an incomplete/old list of packages.  Force
a regen of the cache file whenever the cros_setup_toolchains.py script
itself changes.

BUG= chromium:751740 
TEST=cache is updated when cros_setup_toolchains.py changes, but not when it's the same
TEST=updating host packages works again as the complete list of packages is found

Change-Id: I63551e0e498d71535d119670641ed0d6e390d792
Reviewed-on: https://chromium-review.googlesource.com/606721
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>

[modify] https://crrev.com/3ed47729bc21ea48437c6848528dbc459048fcf4/scripts/cros_setup_toolchains.py

Sign in to add a comment