New issue
Advanced search Search tips
Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Mar 1
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 740549



Sign in to add a comment

Changes to chromeos-base/trunks cause build failures in CQ

Project Member Reported by nya@chromium.org, Feb 20

Issue description

It seems any innocent change to chromeos-base/trunks can be blocked in CQ for dependency resolution failure.

See this failed CQ run:
https://luci-milo.appspot.com/buildbot/chromeos/eve-paladin/2420

It contains only one change crrev.com/c/902902, and it failed in BuildPackages stage:
https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromeos%2Feve-paladin%2F2420%2F%2B%2Frecipes%2Fsteps%2FBuildPackages%2F0%2Fstdout

Important log is:

[ebuild     U  ] chromeos-base/trunks-0.0.1-r2503::chromiumos [0.0.1-r2502::chromiumos] to /build/eve/ USE="cr50_onboard cros-debug {test} -asan -cros_host -ftdi_tpm -fuzzer -profiling -tpm2_simulator" 0 KiB
[ebuild     U  ] chromeos-base/chaps-0.0.1-r2680::chromiumos [0.0.1-r2679::chromiumos] to /build/eve/ USE="cros-debug tpm2 -asan -cros_host -fuzzer -profiling -systemd {-test*} -tpm" 0 KiB
[ebuild     U  ] chromeos-base/tpm_manager-0.0.1-r1651::chromiumos [0.0.1-r1650::chromiumos] to /build/eve/ USE="cros-debug tpm2 -asan -cros_host -fuzzer -profiling {-test*} -tpm" 0 KiB
[ebuild     U  ] chromeos-base/u2fd-0.0.1-r510::chromiumos [0.0.1-r509::chromiumos] to /build/eve/ USE="cros-debug -asan -cros_host -fuzzer -profiling {-test}" 0 KiB
[ebuild     U  ] chromeos-base/attestation-0.0.1-r2369::chromiumos [0.0.1-r2368::chromiumos] to /build/eve/ USE="cros-debug tpm2 -asan -cros_host -fuzzer -profiling {-test*} -tpm" 0 KiB
[ebuild  rR    ] chromeos-base/arc-oemcrypto-0.0.1-r617::chromeos to /build/eve/ USE="cros-debug seccomp -asan -cros_host -fuzzer -profiling -systemd {-test*}" 0 KiB
[ebuild  rR    ] chromeos-base/chromeos-bsp-eve-0.0.1-r39::eve to /build/eve/ USE="-eve-kvm -eve-userdebug" 0 KiB
[ebuild     U  ] chromeos-base/bluetooth-0.0.1-r39::chromiumos [0.0.1-r38::chromiumos] to /build/eve/ USE="cros-debug -asan -cros_host -fuzzer -profiling -seccomp {-test*}" 0 KiB
[ebuild  rR    ] chromeos-base/cryptohome-0.0.1-r2467::chromiumos to /build/eve/ USE="cros-debug direncryption tpm2 -asan -cert_provision -cros_host -fuzzer -profiling -systemd {-test*} -tpm" 0 KiB
...
The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
# required by chromeos-base/chaps-0.0.1-r2680::chromiumos
# required by chromeos-base/cryptohome-0.0.1-r2467::chromiumos
# required by @__auto_rebuild__ (argument)
>=chromeos-base/trunks-0.0.1-r2503 test


This looks like caused by USE dependency in chromeos-base/chaps, introduced in crrev.com/c/565960:

RDEPEND="
...
	tpm2? (
		chromeos-base/trunks[test?]
	)
...
"

IIUC the installed version of chromeos-base/chaps was built with USE=test so it requires chromeos-base/trunks[test], however since they are being rebuilt with FEATURE=-test, emerge failed to resolve?
 
Blocking: 740549
i'm thinking this can only come up on incremental builders.  the scenario:
- unittest stage runs
- packages get rebuilt & installed with USE=test (via FEATURES=test)
- this includes trunks and chaps and cryptohome and such
- next run still have those packages built & installed w/USE=test enabled
- trunks gets upgraded but chaps does not
- we hit the failure situation you describe here

i think we didn't notice this previously because trunks & chaps were in the same git repo, and whenever one uprevved, the other was (more or less) guaranteed to be uprevved too.  now with the subtree work, that's no longer the case.

we could workaround it by having trunks & chaps list each other in their subtree vars ... that way we continue to guarantee they both get rebuilt.

i'm not sure if there's an emerge option atm that'd let us tell emerge it should autorebuild based on changed USE flags when the packages in question haven't been selected.  maybe -DN ?
> i think we didn't notice this previously because trunks & chaps were in the same git repo, and whenever one uprevved, the other was (more or less) guaranteed to be uprevved too.  now with the subtree work, that's no longer the case.

I also guessed this might be an unintended side effect from CROS_WORKON_SUBTREE, but what I don't quite understand is that, in the above log, both trunks and chaps were uprev'ed.


> i'm not sure if there's an emerge option atm that'd let us tell emerge it should autorebuild based on changed USE flags when the packages in question haven't been selected.  maybe -DN ?

I'm afraid it may now work; manpage of emerge(1) has following explanation about --newuse (-N):

 NOTE: This option ignores the state of the "test" USE flag, since that flag has a special binding to FEATURES="test"


===

Anyway, we need a short-term workaround for this issue. How about dropping [test] from those RDEPEND? I'm hesitant to add more dirs to CROS_WORKON_SUBTREE when they are not real dependencies...

I created https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/934107, but I'm not confident if it resolves the issue.

Labels: -Pri-2 Pri-1
Owner: nya@chromium.org
Status: Assigned (was: Untriaged)
This breaks the CQ often enough that it should be at least P1.
Cc: akes...@chromium.org
Project Member

Comment 7 by bugdroid1@chromium.org, Feb 26

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/c3a66566c323a34eced508be6556deb9c526061f

commit c3a66566c323a34eced508be6556deb9c526061f
Author: Shuhei Takahashi <nya@chromium.org>
Date: Mon Feb 26 12:07:02 2018

Avoid runtime USE=test dependencies to trunks.

The purpose of these USE=test dependencies to trunks is to link
libtrunks_test.a, so they need not to be in RDEPEND.

I have not figured out the mechanism of  crbug.com/813791  yet, but
those runtime dependency seem to refuse trunks to be re-emerged
without USE=test.

Also updates EAPI to 5.

BUG= chromium:813791 
TEST=precq

Change-Id: Ie5a5875593370104f31560092603481fcc430383
Reviewed-on: https://chromium-review.googlesource.com/934107
Commit-Ready: Shuhei Takahashi <nya@chromium.org>
Tested-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/c3a66566c323a34eced508be6556deb9c526061f/chromeos-base/tpm_manager/tpm_manager-9999.ebuild
[modify] https://crrev.com/c3a66566c323a34eced508be6556deb9c526061f/chromeos-base/chaps/chaps-9999.ebuild
[modify] https://crrev.com/c3a66566c323a34eced508be6556deb9c526061f/chromeos-base/attestation/attestation-9999.ebuild
[modify] https://crrev.com/c3a66566c323a34eced508be6556deb9c526061f/chromeos-base/cryptohome/cryptohome-9999.ebuild

Status: Fixed (was: Assigned)
I hope the issue is fixed. Please feel free to reopen if we still see the same failure.

Status: Started (was: Fixed)
Yes, it looks possibly related... let me take a look.
Status: Fixed (was: Started)
I looked at the failing test in #c9 and I guess it's real flaky failure in the newly added tests. I'm closing the bug now, but please feel free to reopen if that's not the case.

Sign in to add a comment