New issue
Advanced search Search tips

Issue 913094 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 791061
Owner: ----
Closed: Dec 11
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

MasterSlaveSyncCompletion is flaky. This hurts the postsubmit stability badly.

Project Member Reported by dgarr...@chromium.org, Dec 7

Issue description

In this example postsubmit run, every single build failure is caused by contention while trying to write to manifest-versions.

I'm not sure what exactly is written, or if those writes have any value.

https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8927859469314103216

An example child failure:

https://ci.chromium.org/p/chromeos/builders/luci.chromeos.general/Prod/b8927858680131922384

  14:00:18: INFO: Updating manifest-versions checkout.
  14:00:58: INFO: Pushing to branch (_RemoteRef(remote='origin', ref='refs/heads/master', project_name=None)) with message: Automatic checkin: status=pass build_version 11359.0.0-rc1 for relm-postsubmit 
  14:00:58: ERROR: Failed to update the status for relm-postsubmit during remote command: return code: 1; command: git push origin temp_auto_checkin_branch:refs/heads/master
  To https://chrome-internal.googlesource.com/chromeos/manifest-versions
   ! [rejected]                temp_auto_checkin_branch -> master (fetch first)
  error: failed to push some refs to 'https://chrome-internal.googlesource.com/chromeos/manifest-versions'
  hint: Updates were rejected because the remote contains work that you do
  hint: not have locally. This is usually caused by another repository pushing
  hint: to the same ref. You may want to first integrate the remote changes
  hint: (e.g., 'git pull ...') before pushing again.
  hint: See the 'Note about fast-forwards' in 'git push --help' for details.
  cmd=['git', 'push', 'origin', 'temp_auto_checkin_branch:refs/heads/master'], cwd=/b/swarming/w/ir/cache/cbuild/repository/manifest-versions-internal
  14:00:58: ERROR: Retrying to update the status:  Retry 1/20
 
Mergedinto: 791061
Status: Duplicate (was: Untriaged)
The postsubmit builders have exaggerated an existing problem, but did not create it.

Sign in to add a comment