Issue metadata
Sign in to add a comment
|
VMTest stage fails on amd64-generic-asan builder |
||||||||||||||||||||||||
Issue descriptionSee example build: https://ci.chromium.org/p/chromeos/builds/b8945033862851900736 The VMTest stage fails with: ====================================================================== FAIL: SimpleTestVerify (crostestutils.au_test_harness.au_test.AUTest) Test that only verifies the target image. ---------------------------------------------------------------------- Traceback (most recent call last): File "/b/swarming/w/ir/cache/cbuild/repository/src/platform/crostestutils/au_test_harness/../../crostestutils/au_test_harness/au_test.py", line 232, in SimpleTestVerify self.assertTrue(self.worker.VerifyImage(self)) AssertionError: False is not true ---------------------------------------------------------------------- Ran 1 test in 358.730s FAILED (failures=1) 2018/05/31 01:11:31 - cros_build_lib.py - ERROR : Test harness failed. 2018/05/31 01:11:32 - cros_build_lib.py - ERROR : /b/swarming/w/ir/cache/cbuild/repository/src/platform/crostestutils/au_test_harness/cros_au_test_harness.py --base_image=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/chromiumos_test_image.bin --target_image=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/chromiumos_test_image.bin --board=amd64-generic --type=vm --remote=0.0.0.0 --verbose --jobs=7 --ssh_private_key=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/id_rsa --ssh_port=9228 --verify_suite_name=smoke --test_prefix=SimpleTestVerify --test_results_root=/b/swarming/w/ir/cache/cbuild/repository/chroot/tmp/cbuildbotAeWnok/smoke/test_harness --no_graphics --whitelist_chrome_crashes exited with code 1: None [1;31m01:11:33: ERROR: !!!VMTests failed!!!
,
Jun 1 2018
,
Jun 1 2018
Need help finding an owner for this test.
,
Jun 1 2018
This is the entry point for all VMTest tests. From the logs it looks like the builder was not able to connect to the KVM instance. I don't know if there are more logs but you could run the failed command locally and see if it's reproducible. Also, adding Aviv who might know the better person to ask for help.
`bin/ctest --board=amd64-generic --type=vm --no_graphics --verbose --target_image=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/chromiumos_test_image.bin --test_results_root=/b/swarming/w/ir/cache/cbuild/repository/chroot/tmp/cbuildbotAeWnok/smoke/test_harness --ssh_port=9228 --only_verify --suite=smoke --whitelist_chrome_crashes --ssh_private_key=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/id_rsa`
2018/05/31 01:06:56 - cros_build_lib.py - INFO : RunCommand: ./bin/cros_run_vm_test '--board=amd64-generic' '--image_path=/b/swarming/w/ir/tmp/cbuildbot-tmpx3iz1_/chromiumos_qemu_disk.bin.PkWiRz' --persist '--kvm_pid=/tmp/kvm.9228' '--ssh_port=9228' '--results_dir_root=/tmp/cbuildbotAeWnok/smoke/test_harness/all/SimpleTestVerify/1_autotest_tests' '--verbose=0' suite:smoke --no_graphics --whitelist_chrome_crashes '--ssh_private_key=/b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/id_rsa' in /b/swarming/w/ir/cache/cbuild/repository/src/scripts
INFO : QEMU binary: /b/swarming/w/ir/cache/cbuild/repository/chroot/usr/bin/qemu-system-x86_64
INFO : QEMU version: QEMU emulator version 2.6.0, Copyright (c) 2003-2008 Fabrice Bellard
Starting a KVM instance
INFO : Set kvm_smp to min(8, 32).
INFO : Launching: /b/swarming/w/ir/cache/cbuild/repository/chroot/usr/bin/qemu-system-x86_64 -enable-kvm -m 8G -smp 8 -cpu SandyBridge,-invpcid,-tsc-deadline,check -vga virtio -pidfile /tmp/kvm.9228 -chardev pipe,id=control_pipe,path=/tmp/kvm.9228.monitor -serial file:/tmp/kvm.9228.serial -mon chardev=control_pipe -daemonize -device virtio-net,netdev=eth0 -display none -netdev user,id=eth0,net=10.0.2.0/27,hostfwd=tcp:127.0.0.1:9228-:22 -drive file=/b/swarming/w/ir/tmp/cbuildbot-tmpx3iz1_/chromiumos_qemu_disk.bin.PkWiRz,index=0,media=disk,cache=unsafe
WARNING: Image format was not specified for '/b/swarming/w/ir/tmp/cbuildbot-tmpx3iz1_/chromiumos_qemu_disk.bin.PkWiRz' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
INFO : KVM started with pid stored in /tmp/kvm.9228
INFO : Serial output, if available, can be found in /tmp/kvm.9228.serial
Could not initiate first contact with remote host
Connection timed out during banner exchange
Connection timed out during banner exchange
Failed to connect to virtual machine, retrying ...
INFO : QEMU binary: /b/swarming/w/ir/cache/cbuild/repository/chroot/usr/bin/qemu-system-x86_64
INFO : QEMU version: QEMU emulator version 2.6.0, Copyright (c) 2003-2008 Fabrice Bellard
Persist requested. Use --ssh_port 9228 --ssh_private_key /b/swarming/w/ir/cache/cbuild/repository/src/build/images/amd64-generic/latest-cbuildbot/id_rsa --kvm_pid /tmp/kvm.9228 to re-connect to it.
INFO : QEMU binary: /b/swarming/w/ir/cache/cbuild/repository/chroot/usr/bin/qemu-system-x86_64
INFO : QEMU version: QEMU emulator version 2.6.0, Copyright (c) 2003-2008 Fabrice Bellard
Using a pre-created KVM instance specified by /tmp/kvm.9228.
Could not initiate first contact with remote host
Connection timed out during banner exchange
Connection timed out during banner exchange
Failed to connect to virtual machine, retrying ...
,
Jun 1 2018
Yes, as wonderfly@ said, the machine has problem connecting to its KVM instance. It reminds me of a past issue crbug.com/808045 that it has network ip address collision once builder to GCE instance. Maybe it is related, but at current stage, we should run the failed command locally and check if it's reproducible.
,
Jul 13
VMTest stage is passing currently https://cros-goldeneye.corp.google.com/chromeos/legoland/builderHistory?buildConfig=amd64-generic-asan&buildBranch=master |
|||||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||||
Comment 1 by rahulchaudhry@chromium.org
, May 31 2018