M52 pfq failing in SyncChrome |
|||||
Issue descriptionLooks like a timeout issue: @@@BUILD_STEP@MasterSlaveSyncCompletion@@@ ************************************************************ ** Start Stage MasterSlaveSyncCompletion - Sun, 10 Jul 2016 23:37:02 -0700 (PDT) ** ** Stage that records whether we passed or failed to build/test manifest. ************************************************************ 23:37:02: INFO: Recording status fail for ['lumpy-pre-flight-branch'] 23:37:02: INFO: RunCommand: /b/cbuild/shared_internal/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cp -v -- /tmp/cbuildbot-tmpKGHctP/cbuildbot-tmp7WJKRS/tmpNPdWHw gs://chromeos-manifest-versions/builder-status/8350.52.0-rc4/lumpy-pre-flight-branch 23:37:03: INFO: Running cidb query on pid 11949, repr(query) starts with 'SELECT deadline >= NOW(), TIMEDIFF(deadline, NOW()) from buildTable where id = 943672' 23:37:03: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Select at 0x6224150; Select object> 23:37:03: ERROR: No status found for build config lumpy-pre-flight-branch. 23:37:03: INFO: Still waiting for the following builds to complete: ['lumpy-pre-flight-branch'] 23:37:03: INFO: 0:00:00 until timeout... 23:37:03: INFO: RunCommand: /b/cbuild/shared_internal/.cache/common/gsutil_4.19.tar.gz/gsutil/gsutil -o 'Boto:num_retries=10' cat gs://chromeos-manifest-versions/builder-status/8350.52.0-rc4/lumpy-pre-flight-branch 23:37:04: ERROR: Not all builds finished before timeout (0 minutes) reached. @@@STEP_LINK@lumpy-pre-flight-branch: The SyncChrome stage failed: <class 'chromite.lib.cros_build_lib.RunCommandError'>: return code: 1; command: /b/cbuild/shared_internal/chromite/bin/sync_chrome --reset --internal --tag 52.0.2743.73 /b/cbuild/shared_internal/.cach@https://uberchromegw.corp.google.com/i/chromiumos.release/builders/lumpy-pre-flight-branch%20release-R52-8350.B/builds/143@@@ @@@STEP_WARNINGS@@@ 23:37:04: WARNING: The following builders failed with this manifest: lumpy-pre-flight-branch Please check the logs of the failing builders for details. @@@STEP_FAILURE@@@ 23:37:04: ERROR: 23:37:04: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x622ef50> 23:37:04: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x6224710> ************************************************************ ** Finished Stage MasterSlaveSyncCompletion - Sun, 10 Jul 2016 23:37:04 -0700 (PDT) ************************************************************ 23:37:04: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Insert object at 0x62245d0> 23:37:04: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x6224b90> 23:37:05: INFO: Running cidb query on pid 11949, repr(query) starts with <sqlalchemy.sql.expression.Update object at 0x6224c90> Build link: https://uberchromegw.corp.google.com/i/chromeos_release/builders/lumpy-pre-flight-branch%20release-R52-8350.B/builds/143 Logs link: https://uberchromegw.corp.google.com/i/chromeos_release/builders/lumpy-pre-flight-branch%20release-R52-8350.B/builds/143/steps/MasterSlaveSyncCompletion/logs/stdio
,
Jul 11 2016
This is failing in SyncChrome. Looks like some kind of git error (perhaps local git corruption). " error: refs/remotes/origin/test_tags1 does not point to a valid object! error: refs/tags/__experimental.1 does not point to a valid object! "
,
Jul 11 2016
Find these errors too: ________ running 'git -c core.deltaBaseCacheLimit=2g fetch origin --verbose' in '/b/cbuild/shared_internal/.cache/distfiles/target/chrome-src-internal/src/v8' error: refs/remotes/origin/test_tags1 does not point to a valid object! error: refs/tags/__experimental.1 does not point to a valid object! error: refs/remotes/origin/test_tags1 does not point to a valid object! error: refs/tags/__experimental.1 does not point to a valid object! error: refs/remotes/origin/test_tags1 does not point to a valid object! error: refs/tags/__experimental.1 does not point to a valid object! How to fix the local git corruption. Do we need to log into the server, and fix the git manually?
,
Jul 11 2016
Yes, that is one option (by deleting the broken repos). A more aggressive option is to do force build with clobber. The drawback is we may run out of gerrit quota if not careful. Don should weigh in on this question.
,
Jul 11 2016
Based on the team discussion with Don, I will do a force build with clobber.
,
Jul 11 2016
Clobbering a single builder is always safe. Its when you wipe out 10 or more that things get to be problematic. As a note, newer branches will auto-clobber Chrome if it fails to sync.
,
Jul 11 2016
Looks like chrome synched ok in current build...thanks!
,
Jul 14 2016
,
Aug 29 2016
,
Aug 29 2016
bulk verified |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by shuqianz@chromium.org
, Jul 11 2016