New issue
Advanced search Search tips

Issue 900366 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Nov 16
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

crosvm 'cargo test --release' fails to build qcow_utils with cargo 1.28.0

Project Member Reported by dverkamp@chromium.org, Oct 30

Issue description

This seems like it could be a race somewhere (cargo?) between generating the object file and linking it into the final qcow_utils.so file.

The failure seems to be reliably reproducible on my workstation (Lenovo P920 with 72 threads).

The output looks like this:

~/trunk/src/platform/crosvm $ cargo test --release
   Compiling unicode-xid v0.1.0
   Compiling syscall_defines v0.1.0 (file:///mnt/host/source/src/platform/crosvm/syscall_defines)
   Compiling data_model v0.1.0 (file:///mnt/host/source/src/platform/crosvm/data_model)
   Compiling poll_token_derive v0.1.0 (file:///mnt/host/source/src/platform/crosvm/sys_util/poll_token_derive)
   Compiling libc v0.2.40
   Compiling byteorder v1.1.0
   Compiling cc v1.0.15
   Compiling getopts v0.2.17
   Compiling proc-macro2 v0.2.3
   Compiling io_jail v0.1.0 (file:///mnt/host/source/src/platform/crosvm/io_jail)
   Compiling kernel_cmdline v0.1.0 (file:///mnt/host/source/src/platform/crosvm/kernel_cmdline)
   Compiling rand v0.3.20
   Compiling sys_util v0.1.0 (file:///mnt/host/source/src/platform/crosvm/sys_util)
   Compiling quote v0.4.2
   Compiling kvm_sys v0.1.0 (file:///mnt/host/source/src/platform/crosvm/kvm_sys)
   Compiling net_sys v0.1.0 (file:///mnt/host/source/src/platform/crosvm/net_sys)
   Compiling resources v0.1.0 (file:///mnt/host/source/src/platform/crosvm/resources)
   Compiling virtio_sys v0.1.0 (file:///mnt/host/source/src/platform/crosvm/virtio_sys)
   Compiling qcow v0.1.0 (file:///mnt/host/source/src/platform/crosvm/qcow)
   Compiling kernel_loader v0.1.0 (file:///mnt/host/source/src/platform/crosvm/kernel_loader)
warning: variant is never constructed: `OpenGpuBufferDevice`
  --> resources/src/gpu_allocator.rs:16:5
   |
16 |     OpenGpuBufferDevice,
   |     ^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

warning: variant is never constructed: `CreateGpuBufferDevice`
  --> resources/src/gpu_allocator.rs:17:5
   |
17 |     CreateGpuBufferDevice,
   |     ^^^^^^^^^^^^^^^^^^^^^

warning: variant is never constructed: `OpenGpuBufferDevice`
  --> resources/src/gpu_allocator.rs:16:5
   |
16 |     OpenGpuBufferDevice,
   |     ^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

warning: variant is never constructed: `CreateGpuBufferDevice`
  --> resources/src/gpu_allocator.rs:17:5
   |
17 |     CreateGpuBufferDevice,
   |     ^^^^^^^^^^^^^^^^^^^^^

   Compiling syn v0.12.15
   Compiling net_util v0.1.0 (file:///mnt/host/source/src/platform/crosvm/net_util)
   Compiling vhost v0.1.0 (file:///mnt/host/source/src/platform/crosvm/vhost)
   Compiling kvm v0.1.0 (file:///mnt/host/source/src/platform/crosvm/kvm)
   Compiling x86_64 v0.1.0 (file:///mnt/host/source/src/platform/crosvm/x86_64)
   Compiling qcow_utils v0.1.0 (file:///mnt/host/source/src/platform/crosvm/qcow_utils)
   Compiling vm_control v0.1.0 (file:///mnt/host/source/src/platform/crosvm/vm_control)
   Compiling wire_format_derive v0.1.0 (file:///mnt/host/source/src/platform/crosvm/p9/wire_format_derive)
   Compiling msg_on_socket_derive v0.1.0 (file:///mnt/host/source/src/platform/crosvm/msg_socket/msg_on_socket_derive)
   Compiling bit_field_derive v0.1.0 (file:///mnt/host/source/src/platform/crosvm/bit_field/bit_field_derive)
error: linking with `x86_64-pc-linux-gnu-clang` failed: exit code: 1
  |
  = note: "x86_64-pc-linux-gnu-clang" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib64/rust/rustlib/x86_64-unknown-linux-gnu/lib" "/mnt/host/source/src/platform/crosvm/target/release/deps/qcow_utils.qcow_utils.ep1obs6f-cgu.6.rcgu.o" "-o" "/mnt/host/source/src/platform/crosvm/target/release/deps/libqcow_utils.so" "-Wl,--version-script=/tmp/rustcU3W3dx/list" "-Wl,--gc-sections" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/mnt/host/source/src/platform/crosvm/target/release/deps" "-L" "/usr/lib64/rust/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/tmp/rustcU3W3dx/libstd-df4a2aaeab5f3a68.rlib" "-Wl,--end-group" "/usr/lib64/rust/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c51f6ebad2e04d17.rlib" "-Wl,-Bdynamic" "-lc" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared"
  = note: clang-8: error: no such file or directory: '/mnt/host/source/src/platform/crosvm/target/release/deps/qcow_utils.qcow_utils.ep1obs6f-cgu.6.rcgu.o'


error: aborting due to previous error

error: Could not compile `qcow_utils`.
warning: build failed, waiting for other jobs to finish...
error: build failed



Versions of possibly relevant software:
cargo 1.28.0
rustc 1.30.0
Chromium OS 8.0_pre339409_p20180926-r4 clang version 8.0.0 (/var/cache/chromeos-cache/distfiles/host/egit-src/clang.git 6601c8f525499269dba75f75bbd1ee2671aaa262) (/var/cache/chromeos-cache/distfiles/host/egit-src/llvm.git 36f54002c931a026f490f9fb074c11d91e3487a2) (based on LLVM 8.0.0svn)

 
Forgot to mention - this is crosvm commit ac242df107dec314bd047d58f1654537120ee02e (HEAD of current master).
I was not able to reproduce this with cargo 0.26.0, but when I upgraded to cargo 1.28.0, I was able to reproduce this error. I suppose this must be a cargo bug then.

Components: Tools>ChromeOS-Toolchain
Summary: crosvm 'cargo test --release' fails to build qcow_utils with cargo 1.28.0 (was: crosvm 'cargo test --release' fails to build qcow_utils in some cases)
Components: -Tools>ChromeOS-Toolchain OS>Systems>Containers
Labels: OS-Chrome
Owner: za...@chromium.org
Status: Assigned (was: Untriaged)
The latest cargo version is 1.30.0. We should upgrade the version in the chroot before trying to debug this, as it may already have been fixed.
Upgrading to 1.30.0 does not seem to fix it.
Labels: -OS-Chrome
Testing with cargo 1.32.0, I got a confusing error message - it looks like the qcow_utils lib target is getting built twice.  I'm not sure if there is something wrong with our Cargo.toml or if this is a cargo bug.

The error looks like this (note that the two package paths are identical):

warning: output filename collision.
The lib target `qcow_utils` in package `qcow_utils v0.1.0 (/usr/local/google/home/dverkamp/chromiumos/src/platform/crosvm/qcow_utils)` has the same output filename as the lib target `qcow_utils` in package `qcow_utils v0.1.0 (/usr/local/google/home/dverkamp/chromiumos/src/platform/
crosvm/qcow_utils)`.
Colliding filename is: /usr/local/google/home/dverkamp/chromiumos/src/platform/crosvm/target/release/deps/libqcow_utils.so
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future, see https://github.com/rust-lang/cargo/issues/6313


Moving the qcow_utils package from dependencies to workspace in the main Cargo.toml seems to work around this problem. I don't know the root cause though.
Project Member

Comment 10 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/725274f9edd727afc1abed55f587f7ad04e49fa3

commit 725274f9edd727afc1abed55f587f7ad04e49fa3
Author: Zach Reizner <zachr@google.com>
Date: Fri Nov 16 13:01:42 2018

crosvm: add qcow_utils to array of packages

The qcow_utils crate is no longer automatically built because it was
removed from the list of dependencies for the main crosvm crate. This
change adds qcow_utils to the list of explicitly built
packages so installation can work.

CQ-DEPEND=CL:1336737
TEST=emerge crosvm
BUG= chromium:900366 

Change-Id: I015c6074fa8d69b3cef5c30ce2bdca92a295616d
Reviewed-on: https://chromium-review.googlesource.com/1336738
Commit-Ready: Zach Reizner <zachr@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>

[modify] https://crrev.com/725274f9edd727afc1abed55f587f7ad04e49fa3/chromeos-base/crosvm/crosvm-9999.ebuild

Project Member

Comment 11 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosvm/+/674504a3de8c076a4a4607399beb30dba13893b0

commit 674504a3de8c076a4a4607399beb30dba13893b0
Author: Zach Reizner <zachr@google.com>
Date: Fri Nov 16 13:01:41 2018

move qcow_utils to members from dependencies

The qcow_utils crate is not a dependency of crosvm and should not be
built in the same phase as crosvm. Doing so was harmless before the
recent rustc/cargo changes, which seem to be triggering some kind of
race condition. This change works around the bug.

CQ-DEPEND=CL:1336738
TEST=cargo test --release
BUG= chromium:900366 

Change-Id: I01048128b20cf06580e809f6701688ab72e7756d
Reviewed-on: https://chromium-review.googlesource.com/1336737
Commit-Ready: Zach Reizner <zachr@chromium.org>
Tested-by: Zach Reizner <zachr@chromium.org>
Reviewed-by: Dylan Reid <dgreid@chromium.org>
Reviewed-by: Daniel Verkamp <dverkamp@chromium.org>

[modify] https://crrev.com/674504a3de8c076a4a4607399beb30dba13893b0/Cargo.lock
[modify] https://crrev.com/674504a3de8c076a4a4607399beb30dba13893b0/Cargo.toml

Status: Verified (was: Assigned)
With the fix above, cargo test --release passes again, so I think we can call this fixed.

Sign in to add a comment