"Builder: coral factory factory-coral-10122.B" Builds Failed in the InitialCheckout Stage |
||||||||||
Issue descriptionBuild fail link - https://uberchromegw.corp.google.com/i/chromeos.branch/builders/coral%20factory%20factory-coral-10122.B/builds/38 Partial logs for error - fatal: Couldn't find remote ref refs/heads/factory-coral-10122.B-next/chromeos-experimental [W git.go:283] Transient error string identified in STDERR: "fatal: Couldn't find remote ref refs/heads/factory-coral-10122.B-next/chromeos-experimental\n" [W git.go:294] Retrying after 3s (rc=128): transient error string encountered ... fatal: Couldn't find remote ref refs/heads/factory-coral-10122.B-next/chromeos-experimental [W git.go:283] Transient error string identified in STDERR: "fatal: Couldn't find remote ref refs/heads/factory-coral-10122.B-next/chromeos-experimental\n" [W git.go:317] Command completed with rc 128 after 10 transient failure(s). error: Cannot fetch chromiumos/third_party/kernel error: Exited sync due to fetch errors
,
Jan 11 2018
Assign owner to infra deputy first for visibility.
,
Jan 11 2018
more and more factory / firmware branches met the same error including fizz, glados and others. And would expect to get support for upcoming factory build of partners. Thanks. fizz: https://uberchromegw.corp.google.com/i/chromeos.branch/builders/fizz%20pre-flight%20firmware-fizz-10139.B/builds/51
,
Jan 11 2018
master-release release-R64-10176.B is also in the same issue. https://uberchromegw.corp.google.com/i/chromeos_release/builders/master-release%20release-R64-10176.B
,
Jan 11 2018
Similar to bug #801211
,
Jan 11 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/manifest/+/9269bd1ac03fef1c5b622e5aa2b1d283c4245f27 commit 9269bd1ac03fef1c5b622e5aa2b1d283c4245f27 Author: Bernie Thompson <bhthompson@google.com> Date: Thu Jan 11 17:47:10 2018 Remove kernel-experimental branch from R64 manifest BUG= chromium:801062 TEST=None Change-Id: I473611f74240a6f48b6e76d7a641d250698129c4 Reviewed-on: https://chromium-review.googlesource.com/862102 Reviewed-by: Douglas Anderson <dianders@chromium.org> Reviewed-by: YH Lin <yueherngl@chromium.org> Commit-Queue: Bernie Thompson <bhthompson@chromium.org> Tested-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/9269bd1ac03fef1c5b622e5aa2b1d283c4245f27/full.xml
,
Jan 11 2018
Wonder if some entity or person tried to create a branch named "factory-coral-10122.B-next" (and release-R64-10176.B-next and so on). That would mess up git quite nicely.
,
Jan 11 2018
Also, there might be a problem if there was an infrastructure change which uses '/' as delimiter to parse branch names.
,
Jan 11 2018
The following revision refers to this bug: https://chrome-internal.googlesource.com/chromeos/manifest-internal/+/5a30975300b1584044b144bf7be1f327e887d245 commit 5a30975300b1584044b144bf7be1f327e887d245 Author: Douglas Anderson <dianders@chromium.org> Date: Thu Jan 11 18:29:08 2018
,
Jan 11 2018
+jimhoff@ GoB git oncall So removing the kernel-experimental branch did not help, it just fails on the next one. I can check this out locally and it works ok, though I do start seeing it trying to use git gc on a cleanly wiped checkout which makes me nervous that our kernel repo has grown beyond some boundary that may be making us more likely to see corruption? I think we are nearing the end of our expertise here, adding GoB git oncall for thoughts on why we might be failing checkouts intermittently like this. How would one normally debug these sort of 'Transient error string identified in STDERR: "fatal: Couldn't find remote ref refs/heads/factory-coral-10122.B-next/chromeos-experimental'?
,
Jan 11 2018
,
Jan 11 2018
This doesn't sound like a production emergency affecting many customers (what oncall typically handles), but I am having trouble understanding the details of the bug. If it a production emergency, let me know. If it's not, go/gob-ticket is a good way to get support. Also see our docs <https://g3doc.corp.google.com/company/teams/gerritcodereview/support.md?cl=head> on support in general.
,
Jan 11 2018
Adding the chrome trooper; "[W git.go:283]" messages come from a chrome infra tool.
,
Jan 11 2018
The stuff that's failing is mediated by code in Chrome Infra, not Cros Infra. So, roping in a trooper.
,
Jan 11 2018
,
Jan 11 2018
,
Jan 12 2018
After much tribulation, we've found and filed a GoB bug: b/71872654
,
Jan 12 2018
It ain't happenin' no more, and GoB team claims its fixed, so, let's declare victory. |
||||||||||
►
Sign in to add a comment |
||||||||||
Comment 1 by marcochen@chromium.org
, Jan 11 2018