GS error during payload signing |
||||
Issue descriptionThis paygen error happened a few times: 12:49:56: WARNING: GS_ERROR: No URLs matched: gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin ... @@@STEP_FAILURE@@@ 13:09:03: ERROR: <class 'chromite.lib.gs.GSCommandError'>: return code: 1; command: /b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' stat -- gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin WARNING 0804 12:51:21.005610 oauth2_client.py] FileSystemTokenCache.PutToken: Failed to create cache file /tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83: [Errno 17] File exists: '/tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83' No URLs matched: gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin cmd=['/b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'stat', '--', 'gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin'], extra env={'BOTO_CONFIG': '/b/build/site_config/.boto'} Traceback (most recent call last): File "/b/c/cbuild/repository/chromite/lib/parallel.py", line 602, in TaskRunner task(*x, **task_kwargs) File "/b/c/cbuild/repository/chromite/lib/parallel.py", line 800, in <lambda> fn = lambda idx, task_args: out_queue.put((idx, task(*task_args))) File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_build_lib.py", line 251, in _GenerateSinglePayload dry_run=dry_run) File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_payload_lib.py", line 827, in CreateAndUploadPayload dry_run=dry_run).Run() File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_payload_lib.py", line 705, in Run self._drm(self._Create) File "/b/c/cbuild/repository/chromite/lib/paygen/dryrun_lib.py", line 45, in __call__ return self.Run(func, *args, **kwargs) File "/b/c/cbuild/repository/chromite/lib/paygen/dryrun_lib.py", line 82, in Run return self._Call(func, *args, **kwargs) File "/b/c/cbuild/repository/chromite/lib/paygen/dryrun_lib.py", line 86, in _Call return func(*args, **kwargs) File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_payload_lib.py", line 560, in _Create _, metadata_signatures = self._SignPayload() File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_payload_lib.py", line 527, in _SignPayload [payload_hash, metadata_hash]) File "/b/c/cbuild/repository/chromite/lib/paygen/paygen_payload_lib.py", line 378, in _SignHashes keysets=self.PAYLOAD_SIGNATURE_KEYSETS) File "/b/c/cbuild/repository/chromite/lib/paygen/signer_payloads_client.py", line 376, in GetHashSignatures if not self._WaitForSignatures(all_signature_uris): File "/b/c/cbuild/repository/chromite/lib/paygen/signer_payloads_client.py", line 280, in _WaitForSignatures while missing_signatures and self._ctx.Exists(missing_signatures[0]): File "/b/c/cbuild/repository/chromite/lib/gs.py", line 1039, in Exists self.Stat(path, **kwargs) File "/b/c/cbuild/repository/chromite/lib/gs.py", line 1093, in Stat res = self.DoCommand(['stat', '--', path], redirect_stdout=True, **kwargs) File "/b/c/cbuild/repository/chromite/lib/gs.py", line 783, in DoCommand raise GSCommandError(e.msg, e.result, e.exception) GSCommandError: return code: 1; command: /b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' stat -- gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin WARNING 0804 12:51:21.005610 oauth2_client.py] FileSystemTokenCache.PutToken: Failed to create cache file /tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83: [Errno 17] File exists: '/tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83' No URLs matched: gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin cmd=['/b/c/cbuild/repository/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil', '-o', 'Boto:num_retries=10', 'stat', '--', 'gs://chromeos-releases/dev-channel/veyron-fievel/9808.0.0/payloads/signing/6166-140527908542272/1.payload.hash.update_signer.signed.bin'], extra env={'BOTO_CONFIG': '/b/build/site_config/.boto'} Link to builder: https://uberchromegw.corp.google.com/i/chromeos/builders/veyron_fievel-release/builds/1368
,
Aug 4 2017
,
Aug 4 2017
,
Aug 4 2017
I believe this is the important error: WARNING 0804 12:51:21.005610 oauth2_client.py] FileSystemTokenCache.PutToken: Failed to create cache file /tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83: [Errno 17] File exists: '/tmp/cbuildbot-tmprrL2BY/oauth2_client-tokencache.1002.534d7de05989f9d65394a17ce0e37362d9e78f83' I don't know why we're seeing it now though.
,
Aug 4 2017
That was a gsutil bug seen a long time ago, but we haven't updated our version in a very long time.
,
Aug 4 2017
I reinstanced the builder, just be paranoid.
,
Aug 9 2017
is this solved by re-initiating the builder?
,
Aug 9 2017
It seems to be fixed, but I'm not sure if I fixed it. |
||||
►
Sign in to add a comment |
||||
Comment 1 by dgarr...@chromium.org
, Aug 4 2017