betty-arc64-release fails dev_mode_test in VMTests |
||||
Issue descriptionhttps://luci-milo.appspot.com/buildbot/chromeos/betty-arc64-release/145 " 2017/10/02 07:13:54 - cros_build_lib.py - DEBUG : RunCommand: sudo 'CROS_CACHEDIR=/b/c/cbuild/repository/.cache' 'CROS_SUDO_KEEP_ALIVE=unknown' -- rm -rf -- /b/c/cbuild/repository/chroot/tmp/devserver_wrapper_u4vB0 2017/10/02 07:13:54 - cros_build_lib.py - DEBUG : RunCommand: /b/c/cbuild/repository/src/scripts/bin/cros_stop_vm --kvm_pid /tmp/cbuildbot-tmpJYrHjH/tmpWfmZ9bDevModeTest/kvm.pid 07:14:05: INFO: Refreshing due to a 401 (attempt 1/2) 07:14:05: INFO: Refreshing access_token 07:14:06: INFO: Running VM test cros_vm_test. 07:14:06: INFO: Setting up the VM for testing. 07:14:06: INFO: RunCommand: parted -m /b/c/cbuild/repository/src/build/images/betty-arc64/latest-cbuildbot/chromiumos_qemu_image.bin unit MB print 07:14:44: INFO: VM started at port 37923 07:14:44: INFO: The VM has been successfully set up. Ready to run tests. 07:14:44: INFO: Running test for cros shell. 07:14:44: INFO: Test to use shell command to write a file to the VM device. 07:14:44: INFO: RunCommand: cros shell --no-known-hosts ssh://localhost:37923 -- 'echo "shell command test file" > /tmp/shell-test' 07:14:45: INFO: Test to use shell command to read a file on the VM device. 07:14:45: INFO: RunCommand: cros shell --no-known-hosts ssh://localhost:37923 -- 'cat /tmp/shell-test' 07:14:45: INFO: Test to use shell command to remove a file on the VM device. 07:14:45: INFO: RunCommand: cros shell --no-known-hosts ssh://localhost:37923 -- 'rm /tmp/shell-test' 07:14:46: INFO: Test for cros shell passed. 07:14:46: INFO: Running test for cros debug. 07:14:46: INFO: Test to start and debug a new process on the VM device. 07:14:46: INFO: RunCommand: cros debug --exe /bin/bash ssh://localhost:37923 07:14:54: INFO: Test to attach a running process on the VM device. 07:14:54: INFO: RunCommand: cros debug --pid 1890 ssh://localhost:37923 07:15:02: INFO: Test for cros debug passed. 07:15:02: INFO: Running test for cros flash. 07:15:02: INFO: Test to flash the VM device with the latest image. 07:15:02: INFO: RunCommand: cros flash --no-wipe --no-reboot ssh://localhost:37923 latest 07:17:06: INFO: Refreshing due to a 401 (attempt 1/2) 07:17:06: INFO: Refreshing access_token 07:24:36: ERROR: Failed to flash the VM device. 07:24:36: INFO: ----------- Start of cros flash log ----------- 07:15:03: NOTICE: Preparing to update the remote device localhost 07:15:04: NOTICE: Using image betty-arc64/R63-9995.0.0/chromiumos_test_image.bin 07:23:05: NOTICE: Finalizing image. 07:24:09: NOTICE: Update completed. 07:24:35: ERROR: Stateful update failed. 07:24:36: ERROR: Device update failed. 07:24:36: ERROR: cros flash failed before completing. 07:24:36: ERROR: Failed to perform stateful partition update: return code: 1; command: ssh -p 37923 '-oConnectionAttempts=4' '-oUserKnownHostsFile=/dev/null' '-oProtocol=2' '-oConnectTimeout=30' '-oServerAliveCountMax=3' '-oStrictHostKeyChecking=no' '-oServerAliveInterval=10' '-oNumberOfPasswordPrompts=0' '-oIdentitiesOnly=yes' -i /tmp/cbuildbot-tmpJYrHjH/ssh-tmpiS6l7N/testing_rsa root@localhost -- sh /mnt/stateful_partition/cros-flash/tmp.BgmpQyJQRC/stateful_update /mnt/stateful_partition/cros-flash/tmp.BgmpQyJQRC/stateful.tgz Warning: Permanently added '[localhost]:37923' (ED25519) to the list of known hosts. Warning: Permanently added '[localhost]:37923' (ED25519) to the list of known hosts. Reading local payload /mnt/stateful_partition/cros-flash/tmp.BgmpQyJQRC/stateful.tgz tar: dev_image_new/lib64/python2.7/sre_constants.pyc: Wrote only 4096 of 6195 bytes tar: dev_image_new/lib64/python2.7/rlcompleter.pyo: Cannot write: No space left on device tar: dev_image_new/lib64/python2.7/copy_reg.py: Cannot write: No space left on device tar: dev_image_new/lib64/python2.7/ftplib.pyc: Cannot open: No space left on device tar: dev_image_new/lib64/python2.7/sre.pyo: Cannot open: No space left on device tar: dev_image_new/lib64/python2.7/webbrowser.pyc: Cannot open: No space left on device tar: dev_image_new/lib64/python2.7/rfc822.pyc: Cannot open: No space left on device <snip> " Note that the second attempt is a different failure mode (see Issue 770764 )
,
Oct 30 2017
I think you're right, it should be OK to just increase the filesystem size for betty-arc64. I'll prepare a CL. Regarding crbug.com/770764 , https://chromium-review.googlesource.com/c/chromiumos/platform/crostestutils/+/741684 should fix it.
,
Oct 30 2017
I've been looking into this. It looks like the autotests succeed, but then attempting to re-flash the VM fails. I've reproduced it on my local VM. cros flash downloads a large update file into the stateful partition, then tries to unpack it somewhere and runs out of space.
,
Nov 6 2017
seems to be passing https://uberchromegw.corp.google.com/i/chromeos/builders/betty-arc64-paladin?numbuilds=100 Was something done to fix this, or did it fix itself?
,
Nov 6 2017
It fixed itself. I'm working on a CL to increase the stateful partition size so that this doesn't come back.
,
Nov 18 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/overlays/overlay-betty-arc64-private/+/6cc864fcf91ddd0040852ca11d486d840184afd3 commit 6cc864fcf91ddd0040852ca11d486d840184afd3 Author: Nicolas Norvez <norvez@google.com> Date: Sat Nov 18 21:52:19 2017
,
Nov 20 2017
betty-arc64-release now passes after increasing the size of the stateful partition.
,
Jan 22 2018
,
Jan 23 2018
|
||||
►
Sign in to add a comment |
||||
Comment 1 by teravest@chromium.org
, Oct 30 2017