git cl upload for rietveld broken with SSL error |
|||||||
Issue descriptionGetting an error while running git cl upload $ git clup Using 50% similarity for rename/copy detection. Override with --similarity. milo/appengine/logdog/logDogBuild.go | 51 ++++++++++++++++++++++++++++++++------------------- 1 file changed, 32 insertions(+), 19 deletions(-) Upload server: https://codereview.chromium.org (change with -s/--server) Got exception while uploading -- saving description to /usr/local/google/home/hinoka/.git_cl_description_backup Traceback (most recent call last): File "/usr/local/google/home/hinoka/depot_tools/git_cl.py", line 5379, in <module> sys.exit(main(sys.argv[1:])) File "/usr/local/google/home/hinoka/depot_tools/git_cl.py", line 5361, in main return dispatcher.execute(OptionParser(), argv) File "/usr/local/google/home/hinoka/depot_tools/subcommand.py", line 252, in execute return command(parser, args[1:]) File "/usr/local/google/home/hinoka/depot_tools/git_cl.py", line 4079, in CMDupload return cl.CMDUpload(options, args, orig_args) File "/usr/local/google/home/hinoka/depot_tools/git_cl.py", line 1522, in CMDUpload ret = self.CMDUploadChange(options, git_diff_args, change) File "/usr/local/google/home/hinoka/depot_tools/git_cl.py", line 2163, in CMDUploadChange issue, patchset = upload.RealMain(upload_args) File "/usr/local/google/home/hinoka/depot_tools/third_party/upload.py", line 2479, in RealMain response_body = rpc_server.Send("/upload", body, content_type=ctype) File "/usr/local/google/home/hinoka/depot_tools/third_party/upload.py", line 454, in Send f = self.opener.open(req, timeout=70) File "/usr/lib/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 1222, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open raise URLError(err) urllib2.URLError: <urlopen error [Errno 8] _ssl.c:510: EOF occurred in violation of protocol>
,
Oct 19 2016
,
Oct 19 2016
Investigation shows that this is due to a DNS change which removed the VIP from our CNAME, causing clients to use SNI directly. Theoretically this would have prevented us from ever running into this morning's problems again. However, it turns out that the vast majority of our developer machines -- and our buildbot fleet -- don't actually support SNI (everyone uses Python 2.7.6; Python only added native SNI support in 2.7.9). This is resulting in massive breakage.
,
Oct 19 2016
,
Oct 19 2016
,
Oct 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/6ff1fc0e0163002596edbfbca2335325b043b823 commit 6ff1fc0e0163002596edbfbca2335325b043b823 Author: Aaron Gable <agable@chromium.org> Date: Wed Oct 19 01:23:25 2016 Automatically map urls to their raw appengine forms BUG= 657216 R=dnj@chromium.org Review URL: https://codereview.chromium.org/2431893002 . [add] https://crrev.com/6ff1fc0e0163002596edbfbca2335325b043b823/appengine_mapper.py [modify] https://crrev.com/6ff1fc0e0163002596edbfbca2335325b043b823/third_party/upload.py
,
Oct 19 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal.git/+/00f1a7e4616a55a10e26bbfcd400b42fbb54a4c4 commit 00f1a7e4616a55a10e26bbfcd400b42fbb54a4c4 Author: dnj <dnj@google.com> Date: Wed Oct 19 01:30:47 2016
,
Oct 19 2016
I'm opening this bug up so that external folks can post if they observe continued outage.
,
Oct 19 2016
The depot_tools CL seems to be working for both developers and bots, and the CQs have been restarted to pick up their change and are chugging along. In the mean time, the revert of the DNS change has also landed (much faster than last time), and we're just waiting on that to propagate until we're back at the status quo :)
,
Oct 19 2016
All clear. Just confirmed with friedman@, the rollback has landed. You can confirm by running $ dig @8.8.8.8 codereview.chromium.org CNAME | grep 'codereview.chromium.org. 299' if it shows codereview.chromium.org. 299 IN CNAME ghs-svc-https-c1050.ghs-ssl.googlehosted.com. then the rollback is complete. whereas codereview.chromium.org. 299 IN CNAME ghs-svc-https-c1050.ghs-ssl.googlehosted.com means the rollback is incomplete. There may be a little more flapping due to the nature of DNS.
,
Oct 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/2a932686de7f14f155a66e02e9c21c30aa2926a9 commit 2a932686de7f14f155a66e02e9c21c30aa2926a9 Author: recipe-roller <recipe-roller@chromium.org> Date: Wed Oct 19 03:00:18 2016 Roll recipe dependencies (trivial). This is an automated CL created by the recipe roller. This CL rolls recipe changes from upstream projects (e.g. depot_tools) into downstream projects (e.g. tools/build). More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug (or complain) depot_tools: https://crrev.com/6ff1fc0e0163002596edbfbca2335325b043b823 Automatically map urls to their raw appengine forms (agable@chromium.org) TBR=martiniss@chromium.org,phajdan.jr@chromium.org BUG= 657216 Recipe-Tryjob-Bypass-Reason: Autoroller Bugdroid-Send-Email: False Review-Url: https://chromiumcodereview.appspot.com/2432023002 [modify] https://crrev.com/2a932686de7f14f155a66e02e9c21c30aa2926a9/infra/config/recipes.cfg
,
Oct 19 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/chrome/tools/build_limited/scripts/slave/+/4c79fb660403aeafb708a3c3ee45584d838e4be6 commit 4c79fb660403aeafb708a3c3ee45584d838e4be6 Author: recipe-roller <recipe-roller@chromium.org> Date: Wed Oct 19 03:04:25 2016
,
Oct 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/c0efd536bca76095cf6619fbcd8ec62cb99208a1 commit c0efd536bca76095cf6619fbcd8ec62cb99208a1 Author: recipe-roller <recipe-roller@chromium.org> Date: Wed Oct 19 11:11:44 2016 Roll recipe dependencies (trivial). This is an automated CL created by the recipe roller. This CL rolls recipe changes from upstream projects (e.g. depot_tools) into downstream projects (e.g. tools/build). More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug (or complain) build: https://crrev.com/2a932686de7f14f155a66e02e9c21c30aa2926a9 Roll recipe dependencies (trivial). (recipe-roller@chromium.org) https://crrev.com/4f6540ce526f62da04741be7d86861db8550705e Use signal.SIGTERM instead of signal.SIGKILL to kill process. (yyanagisawa@chromium.org) depot_tools: https://crrev.com/6ff1fc0e0163002596edbfbca2335325b043b823 Automatically map urls to their raw appengine forms (agable@chromium.org) TBR=martiniss@chromium.org,phajdan.jr@chromium.org BUG= 656846 , 657216 Recipe-Tryjob-Bypass-Reason: Autoroller Bugdroid-Send-Email: False Review-Url: https://chromiumcodereview.appspot.com/2436493002 Cr-Commit-Position: refs/heads/master@{#426149} [modify] https://crrev.com/c0efd536bca76095cf6619fbcd8ec62cb99208a1/infra/config/recipes.cfg
,
Oct 19 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/8fa9582d564549815771a7b920c2f4e08e7d6beb commit 8fa9582d564549815771a7b920c2f4e08e7d6beb Author: recipe-roller <recipe-roller@chromium.org> Date: Wed Oct 19 11:22:52 2016 Roll recipe dependencies (trivial). This is an automated CL created by the recipe roller. This CL rolls recipe changes from upstream projects (e.g. depot_tools) into downstream projects (e.g. tools/build). More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug (or complain) build: https://crrev.com/2a932686de7f14f155a66e02e9c21c30aa2926a9 Roll recipe dependencies (trivial). (recipe-roller@chromium.org) https://crrev.com/4f6540ce526f62da04741be7d86861db8550705e Use signal.SIGTERM instead of signal.SIGKILL to kill process. (yyanagisawa@chromium.org) https://crrev.com/d688a695ab02001fa8eb03c371d205725ecc37b7 V8: Bump shards on x87 bot (machenbach@chromium.org) depot_tools: https://crrev.com/6ff1fc0e0163002596edbfbca2335325b043b823 Automatically map urls to their raw appengine forms (agable@chromium.org) TBR=martiniss@chromium.org,phajdan.jr@chromium.org BUG= 656846 , 657216 Recipe-Tryjob-Bypass-Reason: Autoroller Bugdroid-Send-Email: False Review-Url: https://chromiumcodereview.appspot.com/2433003003 [modify] https://crrev.com/8fa9582d564549815771a7b920c2f4e08e7d6beb/infra/config/recipes.cfg
,
Oct 19 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal.git/+/7ec9dd10144c2e66de72b72fd207c58c5e418beb commit 7ec9dd10144c2e66de72b72fd207c58c5e418beb Author: tandrii <tandrii@google.com> Date: Wed Oct 19 14:28:01 2016
,
Oct 20 2016
Ugh. Thanks, Python. What platforms are the holdouts to getting SNI support into depot_tools? It's a little bit disappointing that a spec from 2003 isn't available in our tooling. https://security.googleblog.com/2015/09/disabling-sslv3-and-rc4.html even includes SNI in the minimum standards for TLS clients. I feel like our own tools really ought to meet the rather low bar in that post. :-)
,
Oct 20 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/19348ed63b961635d7e3d8bd725110c512c52ce4 commit 19348ed63b961635d7e3d8bd725110c512c52ce4 Author: Aaron Gable <agable@chromium.org> Date: Thu Oct 20 00:25:34 2016 Rewrite rietveld urls to always show our custom domains Rietveld already has a system for rewriting various ugly urls into pretty ones that we want humans to interact with. Weirdly, it doesn't rewrite "chromiumcodereview.appspot.com" -- it only does the various "-hr" versions. This CL should make it so that humans only ever see "codereview.chromium.org", as desired. BUG= 657216 Change-Id: I7267c8c0d85058108a3581b628e692b1105a0be7 Reviewed-on: https://chromium-review.googlesource.com/400661 Commit-Queue: Aaron Gable <agable@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [modify] https://crrev.com/19348ed63b961635d7e3d8bd725110c512c52ce4/appengine/chromium_rietveld/codereview/common.py [modify] https://crrev.com/19348ed63b961635d7e3d8bd725110c512c52ce4/appengine/chromium_rietveld/codereview/views.py [modify] https://crrev.com/19348ed63b961635d7e3d8bd725110c512c52ce4/appengine/chromium_rietveld/settings.py
,
Oct 23 2016
The following revision refers to this bug: https://chrome-internal.googlesource.com/infra/infra_internal.git/+/9bdca1e6b30838e646cc725fde047b688c94d6b7 commit 9bdca1e6b30838e646cc725fde047b688c94d6b7 Author: tandrii <tandrii@google.com> Date: Sun Oct 23 15:00:00 2016 |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by d...@chromium.org
, Oct 19 2016