https://chromium.googlesource.com/external/github.com/dart-lang/sdk is behind |
||||||
Issue descriptionIt appears there's some delay from https://chromium.googlesource.com/external/github.com/dart-lang/sdk vs https://github.git.corp.google.com/dart-lang/sdk This is causing breakage on https://build.chromium.org/p/client.dart/console which appears to poll github.com for commits but then fetch them from our mirror :(
,
Mar 6 2017
I'm going to take a look into this because it doesn't seem to be resolving itself.
,
Mar 6 2017
yeah this SHOULD be up to date, but it looks like there's some internal replication lag in googlesource. Filed b/36000031
,
Mar 6 2017
,
Mar 7 2017
Only one of the chromium mirrors, https://us3-mirror-chromium.googlesource.com/external/github.com/dart-lang/sdk , is not syncing correctly. The others are up-to-date. This manifests by the compute-engine builders failing to get the commits, but the golo builders succeeding. Boosting to P1 - the Dart tree is closed while all the builders are failing.
,
Mar 7 2017
They look caught up now. There was a known lag yesterday, so that was probably the cause. On a more important note, it seems like you're asking for pain by having a poller that polls one resource, but then operating on a mirror of that resource. Syncing issues can happen, and they can occasionally be this bad. I *highly* recommend changing your pollers to directly poll the internal mirrors of the GitHub repositories (e.g., https://chromium.googlesource.com/external/github.com/dart-lang/sdk/ ). Even if it lags, at least everything will be internally consistent.
,
Mar 7 2017
,
Mar 8 2017
The problem has come back - us3-mirror-chromium.googlesource.com is out of date, but all other mirrors are correct. I will change the poller to poll the chromium mirror, but this problem won't be fixed by this. I don't think I can make the poller know which of the internal mirrors a builder is hitting.
,
Mar 8 2017
So polling github and then pulling from googlesource is a bad thing, but we should also leave this bug open while googlesource.com actually fixes the copy configs. This particular instance is googlesource having issues keeping all of its replicas in sync.
,
Mar 8 2017
I just checked, and our pollers are only polling the mirrors at chromium.googlesource.com, not github.com, and have been doing so for two years. This issue is only happening because us3-mirror-chromium is not in sync with chromium mirror. But there is nothing for chrome-infra to fix here, I think, so I will set this back to Won't Fix, and leave the GoB bug open. |
||||||
►
Sign in to add a comment |
||||||
Comment 1 by iannu...@google.com
, Mar 6 2017