New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 676434 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 698913
Owner: ----
Closed: Apr 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

build_packages failed to fetch items from Google Storage

Project Member Reported by sbasi@chromium.org, Dec 21 2016

Issue description

https://uberchromegw.corp.google.com/i/chromeos/builders/link-paladin/builds/26949/steps/BuildPackages/logs/stdio

Seems to have failed downloading multiple artifacts from GS.


CommandException: Some components of /build/link/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2 were not downloaded successfully. Please retry this download.

cmd=['/mnt/host/source/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'cp', '-v', '--', 'gs://chromeos-prebuilt/board/cyan/chrome-R57-9107.0.0-rc2/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2', u'/build/link/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2'], extra env={'BOTO_CONFIG': u'/mnt/host/source/src/private-overlays/chromeos-overlay/googlestorage_account.boto'}

<class 'chromite.lib.gs.GSCommandError'>: return code: 1; command: /mnt/host/source/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cp -v -- gs://chromeos-prebuilt/board/link/paladin-R57-9091.0.0-rc1/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2 /build/link/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2


('The read operation timed out',)
CommandException: Some components of /build/link/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2 were not downloaded successfully. Please retry this download.

cmd=['/mnt/host/source/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'cp', '-v', '--', 'gs://chromeos-prebuilt/board/link/paladin-R57-9091.0.0-rc1/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2', u'/build/link/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2'], extra env={'BOTO_CONFIG': u'/mnt/host/source/src/private-overlays/chromeos-overlay/googlestorage_account.boto'}
Traceback (most recent call last):
  File "/mnt/host/source/chromite/lib/parallel.py", line 602, in TaskRunner
    task(*x, **task_kwargs)
  File "/mnt/host/source/chromite/lib/parallel.py", line 800, in <lambda>
    fn = lambda idx, task_args: out_queue.put((idx, task(*task_args)))
  File "/mnt/host/source/chromite/scripts/cros_install_debug_syms.py", line 94, in Install
    self._gs_context.Copy(url, archive, debug_level=logging.DEBUG)
  File "/mnt/host/source/chromite/lib/gs.py", line 874, in Copy
    raise
  File "/mnt/host/source/chromite/lib/gs.py", line 851, in Copy
    result = self.DoCommand(cmd, **kwargs)
  File "/mnt/host/source/chromite/lib/gs.py", line 782, in DoCommand
    raise GSCommandError(e.msg, e.result, e.exception)
GSCommandError: return code: 1; command: /mnt/host/source/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cp -v -- gs://chromeos-prebuilt/board/link/paladin-R57-9091.0.0-rc1/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2 /build/link/packages/sys-devel/llvm-3.8.1-r2.debug.tbz2


Hopefully its just GS Flake.
 
It's really weird, though.  It always gets all the way to the end of the download and then just doesn't finish...  ?

Downloading ...ase/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2: 716.79 MiB/717.5 MiB    
Downloading ...ase/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2: 717.5 MiB/717.5 MiB    
CommandException: Some components of /build/link/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2 were not downloaded successfully. Please retry this download.

cmd=['/mnt/host/source/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'cp', '-v', '--', 'gs://chromeos-prebuilt/board/cyan/chrome-R57-9107.0.0-rc2/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2', u'/build/link/packages/chromeos-base/chromeos-chrome-57.0.2958.0_rc-r1.debug.tbz2'], extra env={'BOTO_CONFIG': u'/mnt/host/source/src/private-overlays/chromeos-overlay/googlestorage_account.boto'}

Comment 2 by nxia@chromium.org, Mar 6 2017

Summary: build_packages failed to fetch items from Google Storage (was: link-paladin build_packages failed to fetch items from Google Storage)
https://luci-milo.appspot.com/buildbot/chromeos/auron-paladin/9723
Mergedinto: 698913
Status: Duplicate (was: Untriaged)
#1: I suspect it fails on a *new* download after completing the *first* download, because it downloads the file in multiple parts. See https://bugs.chromium.org/p/chromium/issues/detail?id=698913#c3

Sign in to add a comment