we had a signing failure:
https://uberchromegw.corp.google.com/i/chromeos/builders/newbie-release/builds/1022

chromeos-signing12.hot.corp.google.com:
> 280218 15:00 DEBUG: Re-signing for recovery
> 280218 15:00 DEBUG: RunCommand: /cros/vboot_reference/scripts/image_signing/sign_official_build.sh recovery /tmp/signer.zWk8r9/recovery_image.bin /cros/keys/NewbiePreMPKeys /tmp/signer.zWk8r9/chromeos_1
> 0446.0.0_newbie_recovery_canary-channel_premp.bin /cros/keys/NewbiePreMPKeys/key.versions
> 280218 15:01 DEBUG: (stdout):
> Parallel unsquashfs: Using 1 processor
> 1584 inodes (7127 blocks) to write
> created 1409 files
> created 314 directories
> created 175 symlinks
> created 0 devices
> created 0 fifos
> 280218 15:01 DEBUG: (stderr):
> sign_official_build.sh: INFO   : Using firmware version: 1
> sign_official_build.sh: INFO   : Using kernel version: 1
> sign_official_build.sh: INFO   : Preparing recovery image...
> sign_official_build.sh: INFO   : Didn't find a firmware update. Not signing firmware.
> sign_android_image.sh: INFO   : Unpacking squashfs image to /tmp/tmp.o1GRlgvoDl/opt/google/containers/android/system.raw.img
> sign_android_image.sh: INFO   : Start signing framework apks
> sign_android_image.sh: ERROR  : Number of re-signed package seems to be wrong

this by itself isn't a problem.  the trouble is that the signer wasn't able to report this back up correctly.

> 280218 15:01 INFO: URL being requested: GET https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest
> 280218 15:01 INFO: Attempting refresh to obtain initial access_token
> 280218 15:01 INFO: Refreshing access_token
> 280218 15:01 INFO: Failed to retrieve access token: {
>   "error" : "deleted_client",
>   "error_description" : "The OAuth client was deleted.",
> ...
> 280218 15:01 DEBUG: Performing Cleanup for [failed]
> 280218 15:01 DEBUG: RunCommand: /cros/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' mv -- gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions gs://chromeos-releases/tobesigned/failed/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions
> 280218 15:01 DEBUG: (stderr):
> Copying gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions [Content-Type=text/plain]...
> / [0 files][    0.0 B/  186.0 B]                                                ^M/ [1 files][  186.0 B/  186.0 B]                                                ^MRemoving gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions...
> -
> Operation completed over 1 objects/186.0 B.                                      
> 
> 280218 15:01 DEBUG: Deleting file gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions.lock
> 280218 15:01 DEBUG: RunCommand: /cros/.cache/common/gsutil_4.27.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' -h x-goog-if-generation-match:1519829947502468 rm gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions.lock
> 280218 15:01 DEBUG: (stderr):
> Removing gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions.lock...
> / [1 objects]                                                                   
> Operation completed over 1 objects.                                              
> 
> 280218 15:01 INFO: Successfully released lock gs://chromeos-releases/tobesigned/50,canary-channel,newbie,10446.0.0,ChromeOS-recovery-R66-10446.0.0-newbie.instructions.lock
> 280218 15:01 INFO: URL being requested: GET https://www.googleapis.com/discovery/v1/apis/gmail/v1/rest
> 280218 15:01 INFO: Attempting refresh to obtain initial access_token
> 280218 15:01 INFO: Refreshing access_token
> 280218 15:01 INFO: Failed to retrieve access token: {
>   "error" : "deleted_client",
>   "error_description" : "The OAuth client was deleted.",
> ...

so it looks like we tried to send a notification, but that notification failed (which is tracked in issue 589670), and that in turn caused us to not update the metadata with the final error.  this caused the release bots to report a timeout instead of seeing the actual ARC++ issue.