err_ssl_version_interference - Fortigate FortiOS 5.0 - disabling TLS 1.3 Fixes
Reported by
dav...@skyfactory.com,
Dec 7 2017
|
||||||||||||||||
Issue descriptionChrome Version : 63.0.3239.84 OS Version: 6.1 (Windows 7, Windows Server 2008 R2) URLs (if applicable) : https://www.google.com/ Other browsers tested: Add OK or FAIL after other browsers where you have tested this issue: Safari: Firefox: OK IE/Edge: What steps will reproduce the problem? 1. Can't access GMail or google.com 2. err_ssl_version_interference 3. Setting TLS 1.3 to disabled under chrome://flags works What is the expected result? web page is loaded What happens instead of that? err_ssl_version_interference Please provide any additional information below. Attach a screenshot if possible. Two people had this problem today. Possibly others once Chrome updates. FortiOS 5.0 firewall (an older version) - I am having other issues related to secure connections but this is the first time I've been able to fix it by disabling an encryption type.
,
Dec 7 2017
+awhalley@ and davidben@, could you ptal and reassign if needed. Thank you.
,
Dec 7 2017
This indicates that you have a buggy firewall or other device. We weren't aware of issues remaining with FortiOS, but they were been quite unresponsive to our repeated contacts all year, so it's plausible there are unexplored issues here. Could you attach a net-internals log per these instructions? Thanks! https://dev.chromium.org/for-testers/providing-network-details Upgrading to the latest versions of your firewall devices may also help with the issue.
,
Dec 7 2017
,
Dec 7 2017
OK, I have two captures, chrome-net-export-log-draft-tls1.3.json TLS 1.3 set to enabled (problem setting) chrome-net-export-log-no-tls1.3.json TLS disabled (works fine)
,
Dec 7 2017
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 7 2017
I can also confirm that TLS 1.3 works if I punch out of the firewall using a VPN.
,
Dec 7 2017
Oh! What happens if you set it to "Enabled (Experiment2)"? We're not actually shipping the draft-18 version because it trips loads of bugs in middleboxes, including FortiOS. The middleboxes are the ones that messed up but, alas, the breakage was too high. We're actually shipping the somewhat misleadingly-named "Experiment2" setting which works around the bugs we know of. (At the time, the workarounds were still experimental.)
,
Dec 7 2017
Additionally, do other HTTPS sites work? (E.g. https://www.facebook.com.) Normally we'd have expected this to only affect Gmail, not www.google.com.
,
Dec 7 2017
At least www.hbo.com doesn't work (it seems to enforce https). I didn't try more than a couple of sites. I tried the Experiment2 setting and captured a log file. It fails in the same way.
,
Dec 7 2017
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 7 2017
Fascinating. What's your FortiOS firmware version? Could you upgrade it? We've had previous reports with Fortinet devices that worked with Experiment2, so perhaps there was some bug in an older revision. I also notice that your log shows secure.livechatinc.com working, so not quite everything is broken? This is puzzling because most of the Google sites you list aren't running TLS 1.3 yet. Thanks for your responsiveness, by the way!
,
Dec 7 2017
Oh, also could you include a net-internals that tries to talk to https://tls.ctf.network/? (That's one of our test servers.) Thanks!
,
Dec 7 2017
FG100D-5.00-build271 - built circa 2014 We don't have an active support contract with FortiNet right now so I can't upgrade the firmware, I'm vacillating back and forth between replacing the firewall with a Tomato-firmware Asus AC3200 or reactivating our support contract, hoping to fix the existing FortiGate fw, which is frankly well beyond the needs of our company. I'm not sure if this date means anything to you, but I had some automated emails sent from my Ubuntu server here locally that stopped on precisely August 4th of this year. Poking in /var/log/mail.log shows this error: Dec 7 14:44:47 air2 postfix/smtp[19817]: F07485447A0: to=<davidr@skyfactory.com>, relay=smtp.gmail.com[74.125.70.108]:587, delay=369924, delays=369903/0.03/21/0, dsn=4.7.5, status=deferred (Cannot start TLS: handshake failure) I'm sure this issue began on August 3rd or 4th, I'm 90% certain this is the same issue as we experienced in Chrome today, and I'm 99% sure the FortiGate is to blame. As to your question about not everything being broken... Is it possible that some TLS connections could persist temporarily across restarts? Changing the setting to the "experiment2" setting, I had to stop and restart the capture, because it was working... Then after a minute or so it stopped and that was when I logged the experiment 2 capture. So, perhaps the secure.livechatinc.com was working using a previously-established connection, in a way the firewall doesn't interfere with?
,
Dec 7 2017
Thank you for providing more feedback. Adding requester "davidben@chromium.org" to the cc list and removing "Needs-Feedback" label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Dec 7 2017
Here's a log of my attempted visit to https://tls.ctf.network/? with Experiment 2 enabled.
,
Dec 7 2017
Thanks! It looks like, from logs on the test server, that the TLS 1.3 ClientHellos aren't making it out of the FortiGate device at all, which is consistent with it affecting all sites, not just those that support TLS 1.3, though not something we'd seen before. (TLS works by having the client advertise everything it supports, and the server picking something. So a TLS 1.3 ClientHello is also a valid TLS 1.2 ClientHello. TLS 1.2 servers will then pick TLS 1.2, and TLS 1.3 servers pick TLS 1.3.) It does appear to kind of inconsistently block things, so I'm not really sure what's going on there. One of your logs even shows a Google site getting through. TLS connections and resumption information don't persist across restarts, so that wouldn't be it. Perhaps the device is stateful and once it gets offended by something, it starts blackholing things? I don't think the smtp.gmail.com thing could be related? We never deployed TLS 1.3 to that service, and presumably the client there is postfix or so, not Chrome, so they wouldn't be offering TLS 1.3, or whatever else caused them troubles.
,
Dec 7 2017
By the way, I forgot to mention: you can use the SSLVersionMax administrative policy to disable TLS 1.3 in your organization in the meantime. Though it does sound like there is indeed *something* weird going on with your device that we ought to get to the bottom of. https://www.chromium.org/administrators/policy-list-3#SSLVersionMax
,
Dec 12 2017
,
Dec 13 2017
davidr@ - Could you please try the SSLVersionMax administrative policy as per comment #18 and please let us know your observations. Thanks...!!
,
Dec 13 2017
davidr: Sorry about that. Please disregard the previous comment. krajshree: No, there is no interesting observation needed from SSLVersionMax. It is known what affect that will have. That is a temporary workaround for the bug in the firewall.
,
Jan 12 2018
,
Jan 19 2018
As this issue is not reproducible from TE end, adding TE-NeedsTriageHelp label. Could someone from Internals>Network>SSL team please have a look at this. Thanks!
,
Oct 18
Are you still running into issues here?
,
Oct 18
Interesting but fairly unsatisfying conclusion to this issue, we finally replaced our firewall but that did not resolve the issue for us. Our local ISP determined that our problem was the result of a bad port that we were connected through. They swapped the port we were on, problem solved.
,
Oct 18
Thank you for providing more feedback. Adding the requester to the cc list. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Oct 18
Thanks for following up! I'll go ahead and close this then. |
||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||
Comment 1 by dav...@skyfactory.com
, Dec 7 2017