no space left on device for kefka-release during paygenbuild (paygen build) |
||||||||
Issue descriptionhttps://uberchromegw.corp.google.com/i/chromeos/builders/kefka-release/builds/754/steps/PaygenBuildCanary/logs/stdio 23:26:04: INFO: Checking payload integrity 23:26:05: INFO: RunCommand: /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/au-generator/cros_generate_update_payload --outside_chroot --extract --image /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/tgt_image.bin --src_image /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/src_image.bin in /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/au-generator 23:26:12: INFO: Applying delta payload and verifying result 23:26:54: INFO: Uploading payload to gs://chromeos-releases/canary-channel/kefka/9176.0.0/payloads/chromeos_9170.0.0-9176.0.0_kefka_canary-channel_delta_test.bin-3c9a2cf8fd2833734f50cfbe6ed364a9. 23:26:54: INFO: RunCommand: /b/cbuild/internal_master/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil cp /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/delta.bin gs://chromeos-releases/canary-channel/kefka/9176.0.0/payloads/chromeos_9170.0.0-9176.0.0_kefka_canary-channel_delta_test.bin-3c9a2cf8fd2833734f50cfbe6ed364a9 23:26:55: INFO: RunCommand: /b/cbuild/internal_master/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil cp /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/delta.log gs://chromeos-releases/canary-channel/kefka/9176.0.0/payloads/chromeos_9170.0.0-9176.0.0_kefka_canary-channel_delta_test.bin-3c9a2cf8fd2833734f50cfbe6ed364a9.log 23:26:56: INFO: RunCommand: /b/cbuild/internal_master/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil cp /tmp/cbuildbot-tmp203pPj/tmpjCmu2u/paygen_payload.xj_6uU/delta.json gs://chromeos-releases/canary-channel/kefka/9176.0.0/payloads/chromeos_9170.0.0-9176.0.0_kefka_canary-channel_delta_test.bin-3c9a2cf8fd2833734f50cfbe6ed364a9.json 23:26:57: INFO: * Finished payload generation in 0:28:58.622379 23:30:00: INFO: RunCommand: /b/cbuild/internal_master/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' -h x-goog-if-generation-match:1484204265415794 rm gs://chromeos-releases/canary-channel/kefka/9176.0.0/payloads/LOCK_flag 23:30:01: ERROR: Failed: Build definition (board='kefka', version='9176.0.0', channel='canary-channel') @@@STEP_FAILURE@@@ 23:30:01: ERROR: <class 'chromite.lib.paygen.paygen_payload_lib.PayloadVerificationError'>: Resulting rootfs partition corrupted Traceback (most recent call last): File "/b/cbuild/internal_master/chromite/lib/parallel.py", line 602, in TaskRunner task(*x, **task_kwargs) File "/b/cbuild/internal_master/chromite/lib/parallel.py", line 800, in <lambda> fn = lambda idx, task_args: out_queue.put((idx, task(*task_args))) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_build_lib.py", line 267, in _GenerateSinglePayload dry_run=dry_run) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 837, in CreateAndUploadPayload dry_run=dry_run).Run() File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 707, in Run self._drm(self._VerifyPayload) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 45, in __call__ return self.Run(func, *args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 82, in Run return self._Call(func, *args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 86, in _Call return func(*args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 681, in _VerifyPayload self._ApplyPayload(payload, is_delta) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 656, in _ApplyPayload raise PayloadVerificationError('Resulting rootfs partition corrupted') PayloadVerificationError: Resulting rootfs partition corrupted <type 'exceptions.IOError'>: [Errno 28] No space left on device Traceback (most recent call last): File "/b/cbuild/internal_master/chromite/lib/parallel.py", line 602, in TaskRunner task(*x, **task_kwargs) File "/b/cbuild/internal_master/chromite/lib/parallel.py", line 800, in <lambda> fn = lambda idx, task_args: out_queue.put((idx, task(*task_args))) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_build_lib.py", line 267, in _GenerateSinglePayload dry_run=dry_run) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 837, in CreateAndUploadPayload dry_run=dry_run).Run() File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 707, in Run self._drm(self._VerifyPayload) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 45, in __call__ return self.Run(func, *args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 82, in Run return self._Call(func, *args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/dryrun_lib.py", line 86, in _Call return func(*args, **kwargs) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 681, in _VerifyPayload self._ApplyPayload(payload, is_delta) File "/b/cbuild/internal_master/chromite/lib/paygen/paygen_payload_lib.py", line 641, in _ApplyPayload payload.Apply(bspatch_path=bspatch_path, **part_files) File "/b/cbuild/internal_master/src/platform/dev/host/lib/update_payload/payload.py", line 321, in Apply old_rootfs_part=old_rootfs_part) File "/b/cbuild/internal_master/src/platform/dev/host/lib/update_payload/applier.py", line 569, in Run self.payload.manifest.old_rootfs_info) File "/b/cbuild/internal_master/src/platform/dev/host/lib/update_payload/applier.py", line 517, in _ApplyToPartition new_part_file, new_part_info.size) File "/b/cbuild/internal_master/src/platform/dev/host/lib/update_payload/applier.py", line 458, in _ApplyOperations self._ApplyReplaceOperation(op, op_name, data, new_part_file, part_size) File "/b/cbuild/internal_master/src/platform/dev/host/lib/update_payload/applier.py", line 269, in _ApplyReplaceOperation part_file.write(out_data[data_start:data_end]) IOError: [Errno 28] No space left on device
,
Jan 12 2017
The planned reimage of all GCE builders to increase their drive sizes. That may get delayed for a few days, but I'll still redo this one (cros-beefy38-c2) today as soon as the current build finished.
,
Jan 12 2017
And yes, this is a builder space issue, not an image issue.
,
Jan 17 2017
All GCE builders have been reimaged with 2T drives. This problem should be solved.
,
Mar 4 2017
,
Apr 17 2017
,
May 30 2017
,
Aug 1 2017
,
Oct 14 2017
|
||||||||
►
Sign in to add a comment |
||||||||
Comment 1 by h...@chromium.org
, Jan 12 2017Owner: dgarr...@chromium.org