The amd64-generic-asan builder, on the external waterfall (https://uberchromegw.corp.google.com/i/chromiumos/waterfall) started consistently failing in BuildImage yesterday. When I look at the error logs, I see
chromeos-firewall-1-r1: IOError: [Errno 28] No space left on device
and (at the end):
ERROR : Target image has run out of space:
ERROR : Filesystem 1M-blocks Used Available Use% Mounted on
ERROR : /dev/loop0 1969 1969 0 100% /mnt/host/source/src/build/images/amd64-generic/R58-9311.0.2017_02_23_1227-a1/rootfs
INFO : Unmounting image from /mnt/host/source/src/build/images/amd64-generic/R58-9311.0.2017_02_23_1227-a1/stateful and /mnt/host/source/src/build/images/amd64-generic/R58-9311.0.2017_02_23_1227-a1/rootfs
Cleaning up /usr/local symlinks for /mnt/host/source/src/build/images/amd64-generic/R58-9311.0.2017_02_23_1227-a1/stateful/dev_image
An error occurred in your build so your latest output directory is invalid.
Running in non-interactive mode so deleting output directory.
Deleted /mnt/host/source/src/build/images/amd64-generic/R58-9311.0.2017_02_23_1227-a1
12:30:02: ERROR:
return code: 1; command: /b/cbuild/external_master/chromite/bin/cros_sdk 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpoEZD6b' 'FEATURES=separatedebug' 'CHROME_ORIGIN=LOCAL_SOURCE' -- ./build_image '--board=amd64-generic' --replace '--version=' '--builder_path=amd64-generic-asan/R58-9311.0.0-b17767' '--disk_layout=2gb-rootfs' test
cmd=['/b/cbuild/external_master/chromite/bin/cros_sdk', 'PARALLEL_EMERGE_STATUS_FILE=/tmp/tmpoEZD6b', 'FEATURES=separatedebug', 'CHROME_ORIGIN=LOCAL_SOURCE', '--', './build_image', '--board=amd64-generic', '--replace', '--version=', '--builder_path=amd64-generic-asan/R58-9311.0.0-b17767', '--disk_layout=2gb-rootfs', 'test'], cwd=/b/cbuild/external_master, extra env={'PARALLEL_EMERGE_STATUS_FILE': '/tmp/tmpoEZD6b', 'FEATURES': 'separatedebug', 'CHROME_ORIGIN': 'LOCAL_SOURCE'}
This probably means the rootfs file system on the image needs to be bigger, but it needs to be investigated.
I'm not sure who the correct owner for this is, so I'm filing it against the current sheriffs.
Comment 1 by sha...@chromium.org
, Feb 23 2017