New issue
Advanced search Search tips

Issue 740014 link

Starred by 1 user

Issue metadata

Status: Unconfirmed
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

gs flake when attempting to upload small (metadata) files

Project Member Reported by akes...@chromium.org, Jul 7 2017

Issue description

Numerous slaves of https://luci-milo.appspot.com/buildbot/chromeos/master-paladin/15299 all failed at the same time, in a manner similar to below. This strongly suggests either gs outage (golo connectivity ruled out, as some of these slaves are gce slaves).

Not sure if related to  Issue 451251  which hit the previous cq run.

Copying file:///b/c/cbuild/repository/buildbot_archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json [Content-Type=text/plain]...
Uploading   ...e-paladin/R61-9719.0.0-rc4/partial-metadata.json: 0 B/11.63 KiB    
Uploading   ...e-paladin/R61-9719.0.0-rc4/partial-metadata.json: 11.63 KiB/11.63 KiB    
ServiceException: 503 None
CommandException: 1 file/object could not be transferred.

cmd=['/b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', '-m', 'cp', '-v', '--', u'/b/c/cbuild/repository/buildbot_archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json', u'gs://chromeos-image-archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json'], extra env={'BOTO_CONFIG': '/b/build/site_config/.boto'}
Traceback (most recent call last):
  File "/b/c/cbuild/repository/chromite/lib/failures_lib.py", line 228, in wrapped_functor
    return functor(*args, **kwargs)
  File "/b/c/cbuild/repository/chromite/cbuildbot/commands.py", line 1903, in _UploadPathToGS
    recursive=True)
  File "/b/c/cbuild/repository/chromite/lib/gs.py", line 585, in CopyInto
    **kwargs)
  File "/b/c/cbuild/repository/chromite/lib/gs.py", line 870, in Copy
    raise
  File "/b/c/cbuild/repository/chromite/lib/gs.py", line 847, in Copy
    result = self.DoCommand(cmd, **kwargs)
  File "/b/c/cbuild/repository/chromite/lib/gs.py", line 783, in DoCommand
    raise GSCommandError(e.msg, e.result, e.exception)
GSCommandError: return code: 1; command: /b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' -m cp -v -- /b/c/cbuild/repository/buildbot_archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json gs://chromeos-image-archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json
Copying file:///b/c/cbuild/repository/buildbot_archive/caroline-paladin/R61-9719.0.0-rc4/partial-metadata.json [Content-Type=text/plain]...
Uploading   ...e-paladin/R61-9719.0.0-rc4/partial-metadata.json: 0 B/11.63 KiB    
Uploading   ...e-paladin/R61-9719.0.0-rc4/partial-metadata.json: 11.63 KiB/11.63 KiB    
ServiceException: 503 None
CommandException: 1 file/object could not be transferred.
 
Cc: xixuan@chromium.org
Status: Unconfirmed (was: Untriaged)
Components: Infra>Client>ChromeOS>CI
Components: -Infra>Client>ChromeOS
Hi, this bug has no status updates recently. Please acknowledge the bug and provide status within two weeks (6/22/2018), or the bug will be closed. Thank you.

Sign in to add a comment